Wesley Grignani
email: wesley.grignani@edu.univali.br

Douglas A Santos

Luigi Dilillo
email: luigi.dilillo@umontpellier.fr

Felipe Viel

Douglas R Melo

A Low-Cost Hardware Accelerator for CCSDS 123 Lossless Hyperspectral Image Compression

Keywords: Systems-on-Chip, Hardware Accelerators, Image Processing, Hyperspectral Images, CCSDS 123.0-B-2

Several remote sensing applications that collect specific data in the space environment use images capable of providing a large volume of information, known as hyperspectral images. Given the amount of data, one of the most critical issues in applications that use hyperspectral images is the demanded for compression, which also affects restrictions on the storage capacity and processing in space applications. This work aimed to implement hyperspectral image compressors, considering the standard by the CCSDS (Consultative Committee for Space Data Systems). The solutions were implemented using a High-Level Synthesis tool (HLS) and a manual description in a Hardware Description Language (HDL). Results show that, compared to the software solution, the HLS and the HDL implementation accelerated the application by 1.6× and 4×, respectively. For images up to 512 spectral bands, the HLS solution presented a throughput of 9.11 MSa/s, while the HDL solution can process 21.47 MSa/s, which meets the real-time requirements of the standard. The HDL solution uses about 3× fewer LUTs (Look-Up Tables) LUTs and 8× fewer FFs (Flip-Flops) than the HLS implementation. Due to the low cost observed in the results, we intend to harden the accelerator and integrate it into a future multi-core satellite system.

I. INTRODUCTION

Space applications that collect data about the Earth use remote sensing techniques for this task. One technique is the hyperspectral images (HSIs), which can be used for Earth image collection, climate analysis, and monitoring forest environments [START_REF] Zhu | A review: Remote sensing sensors[END_REF]. HSIs are three-dimensional structures of pixels, where each pixel is represented in a (x, y, z) coordinate system. Each z-axis layer represents a single image at a given electromagnetic spectrum. The dimensions can sometimes be hundreds or thousands of bands [START_REF] Lopez | FPGA implementation of the ccsds-123.0-b-1 lossless hyperspectral image compression algorithm prediction stage[END_REF].

Given the large volume of data in hyperspectral images, a reduction in their size becomes desirable, aiming to minimize impacts on performance in transmission and processing techniques. A well-known compression algorithm for this type of image is CCSDS 123, developed by the Consultative Committee for Spatial Data Systems [START_REF]The consultative committee for space data systems[END_REF]. Recently, a new version of the standard entitled CCSDS 123.0-B-2 was released. This version introduces some modifications compared to the previous version, such as changes in the predictor equations and changes in the header structure in the encoder. In addition, this standard introduces a near-lossless compression mode for use with the new hybrid entropy encoder.

The existence of many bands to compute HSIs turns it into a data-intensive processing structure for space systems. It affects the ability to store, process, and even transmit this imagery to a ground station. For example, the HyspIRI sensor developed by the National Aeronautics and Space Administration (NASA) can produce up to 5 Terabytes of data per day [START_REF] Hook | Nasa 2014 the hyperspectral infrared imager (hyspiri)[END_REF].

Using reconfigurable architectures, such as FPGAs, has seen an increasing adoption for creating high-performance systems for image processing applications. However, algorithm implementation for a hardware design is more complex than software implementation. Therefore, adopting High-Level Synthesis (HLS) tools has begun, enabling hardware development using programming languages [START_REF] De Fine Licht | Transformations of high-level synthesis codes for high-performance computing[END_REF].

High-level synthesis tools allow the creation of a hardware design from a high-level programming language. The hardware creation process starts from the code specification, and the HLS tool is responsible for creating and allocating resources and generating the hardware description representing the design at the Register-Transfer Level (RTL) [START_REF] Coussy | An introduction to high-level synthesis[END_REF].

Some works have implemented hardware accelerators using HDL (Hardware Description Language) or HLS techniques for HSI compression. The works [START_REF] Tsigkanos | A 3.3 Gbps ccsds 123.0-b-1 multispectral & hyperspectral image compression hardware accelerator on a space-grade SRAM FPGA[END_REF], [START_REF] Pereira | A low-cost hardware accelerator for ccsds 123 predictor in FPGA[END_REF] and [START_REF] Orlandić | A parallel FPGA implementation of the ccsds-123 compression algorithm[END_REF] implemented a hardware accelerator in the CCSDS 123.0-B-1 standard, however, [START_REF] Pereira | A low-cost hardware accelerator for ccsds 123 predictor in FPGA[END_REF] implemented only the prediction step. The work [START_REF] Barrios | Shyloc 2.0: A versatile hardware solution for on-board data and hyperspectral image compression on future space missions[END_REF] presents a versatile solution that implements both CCSDS 123.0-B-1 and CCSDS 121.0-B-2 standards. The works [START_REF] Gonzalez | A real-time FPGA implementation of the ccsds 123.0-b-2 standard[END_REF] and [START_REF] Barrios | Hardware implementation of the ccsds 123.0-b-2 near-lossless compression standard following an hls design methodology[END_REF] implemented the complete compressor of the CCSDS 123.0-B-2 standard, in which [START_REF] Gonzalez | A real-time FPGA implementation of the ccsds 123.0-b-2 standard[END_REF] implemented using VHDL and [START_REF] Barrios | Hardware implementation of the ccsds 123.0-b-2 near-lossless compression standard following an hls design methodology[END_REF] using an HLS tool.

In this work, we present the implementation of accelerators in HDL and HLS for the lossless compression of HSI compatible with the CCSDS 123.0-B-2 standard. The accelerators were designed to reduce development complexity and low resource utilization for applications in space systems, like the new nano/cube satellite segment that introduces stringent constraints in terms of power and resources.

II. CCSDS 123 ACCELERATOR DESIGN

In this work, we extend a previous project [START_REF] Grignani | A high-level synthesis compressor of hyperspectral images based on ccsds 123.0-b-2[END_REF] by implementing the accelerator in HDL and optimizing the HLS version. We present a detailed characterization of the cost and performance and compare it with related works. The compression standard is composed of the predictor and the encoder blocks. Each block has several calculation steps, which can be seen as follows or in more detail in the specification [START_REF] Ccsds | Low-complexity lossless and near-lossless multispectral and hyperspectral image compression[END_REF]. Fig. 1 presents an overview of the prediction design.

A. Prediction

The prediction calculation that involves part of the lossless compression process is separated by sub-steps such as local sum, local difference, weights and local difference vector, central prediction, and mapped quantizer index.

1) Local sum: The first step of the compressor consists of a local sum between samples from the same spectral band. Each sample is represented by s z,y,x through the coordinate system or by the index t, defined as t = yN x + x. There are three options for performing local sum, wide or narrow neighbor-oriented, and column-oriented.

2) Local difference: The local difference is performed considering the local sum calculated earlier. The local difference considers the current pixel and performs the local directional differences in N , N W , and W coordinates.

3) Weights and local difference vectors: The local difference vector is used to store P z user-defined local differences, which are later used to calculate the predicted value of the sample. On the other hand, the weight vector stores the P z values that will be used in the prediction by multiplying them by the vector of local differences. The P z represents the number of preceding bands that predict a sample.

4) Predict: For the prediction calculation, the local difference prediction is performed first, being the multiplication between the weight vector and the local difference vector. Then, the high-resolution predicted sample, the doubleresolution predicted sample, and the predicted sample are calculated. These calculations are necessary to obtain the mapped quantizer index, which represents the output of the predictor.

5) Mapped quantized index: After the previous steps, the output value of the predictor is obtained, called the mapped quantized index. For this, a quantizer index is used, calculated by considering the difference between the value of the current sample and the value of the predicted sample, called the prediction residual.

B. Encoding

The of the encoder (Fig. 2) follows the sample adaptive encoding model and receives the residual prediction the previous component as the input value. The output form of the variable length words can change according the calculations performed by the compressor. The encoder is responsible for receiving samples from the predictor component and encoding them using one of the methods presented in the specification. The encoder should generate a file with a header and body structure of variable sizes specified by the user.

The header contains information about the specification of the method used to perform the prediction and encoding, as well as information about the image that is being compressed. The body of the compressed image should contain the data generated by the entropy encoder used for the implementation.

This work implemented the sample-adaptive entropy encoder because it shows better compression results in the reduced and lossless prediction mode. Each mapped quantized index must be encoded using a variable-length binary codeword. The encoder has an accumulator and counter vector that stores information from each spectral band of the image.

III. HARDWARE IMPLEMENTATION

The lossless compressor implementation in this work is compatible with the new standard CCSDS 123.0-B-2. This CCSDS standard retains all features available in the previous B-1 standard, and the compressed image header of the recommended standard has been designed to be backward compatible with the old version. Thus compressed images produced by a compressor compliant with B-1 will also be compliant with the B-2 [START_REF]Lossless multispectral and hyperspectral image compression informational report[END_REF].

A. Design and Implementation Choices

We made some design choices to reduce the complexity of the implementation and have the lowest use of logic resources. Table I presents the design choices made between the allowed modes of implementation in the standard. As can be seen in Table I, several implementation options are allowed in the standard. All these options have an impact on the image compression ratio but also have a significant impact on implementation complexity and resource utilization.

1) Compression mode: The near-lossless model requires the additional implementation of the sample representatives s ′′ z,y,x component and the fidelity control method m z (t), not needed for the lossless model. Setting m z (t) = 0 we reduce the complexity because the quantization calculation is removed, which eliminates the integer division operations in quantization and mapped steps. In addition, we also reduce the hardware complexity by eliminating the sample representative calculation by setting the parameters to 0 for all bands.

2) Prediction mode: The reduced prediction mode does not require additional calculations of the directional local differences present in the full mode. Moreover, the storage of these values in the weights and local difference vectors increases in the full mode because of these directional differences. The reduced mode was selected for our implementation.

3) Local sum mode: The local sum considers neighboring pixels for the calculation. According to the order of processing, Band Sequential (BSQ), Band Interleaved by Line (BIL), or Band Interleaved by Pixel (BIP) implies the size of the buffers needed to store the samples to calculate the local sum. The column-oriented local sum presents the least dependence on neighboring pixels for the calculation compared to the other options. In this work, the compressor does not buffer the samples to do the local sum but receives the sample and neighbor as input.

4) Processing order: The processing order directly affects the size of the local differences to be stored. The prediction in the spectral band z depends specifically on the local differences, computed in bands z -1, z -2, .., z -P z and stored in the local differences vector. In the BIP method, the size of the local difference vector is only based on the number of previous bands considered for prediction, since this method goes through all the bands of a (x, y) position at a time. Fig. 3 illustrates the BIP storage process for P z = 3 previous local differences. Running through the BSQ or BIL, the vector would need to store all the differences calculated in that band, so when it moves to the next spectral band, the local difference values in the previous bands for each pixel would be stored for the prediction. Thus, the size of the vector in BIP mode is implemented based only on the previous P z bands and does not need to consider the N x and N y dimensions of the image, which are needed in BSQ or BIL methods [START_REF]Lossless multispectral and hyperspectral image compression informational report[END_REF].

These design choices made in the local sum step and processing order make the resource utilization of the compressor change only based on the N z bands of the input image, and not on the N x and N y spatial resolution.

B. High-Level Synthesis

In the HLS implementation, each compression step was implemented in a function using the C-based HLS language. The HLS tool has optimization directives that can be used to inform the synthesis tool on generating the hardware in that specific part. The tool also has libraries for manipulating variables with a fixed bit size, which is crucial for optimized resource usage. The main directives that are widely used are:

• Pipeline: allows the hardware creation in parallel and with specific pipeline levels so that the startup interval of a new calculation is reduced, increasing the throughput. • Unroll: enables the operations in a repetition loop to be executed in a single clock cycle, usually using more logic resources. • Array Partition: forces the compiler to partition the memory vectors into smaller blocks and allocate them in registers, allowing simultaneous access to the data. The HLS tool automatically uses the described directives to generate hardware with better performance. On the other hand, these directives can significantly increase resource usage. Each C-based function that implements a compression step is called sequentially. No optimization directive was manually added at first to check the behavior of the HLS tool in generating the hardware since it automatically applies the optimization directives previously mentioned. The implementation uses the ap fixed library from the HLS tool, which allows us to specify the size of a variable in bits according to the standard specification. To reduce the complexity, calculations with powers of base 2 were done using shifters, avoiding the use of mathematical library functions.

C. Hardware Description Language

In the HDL solution, each compression step was implemented using VHDL. The prediction and encoding components were designed to work in parallel to obtain higher throughput. In addition, some steps have been parallelized to reduce the number of cycles to process a sample. It was observed that some steps did not have data dependency on previous steps or presented simple calculations that could be executed in parallel using a more sophisticated controller.

The local sum and local difference were allocated to execute in the same cycle. The predicted sample and the quantizer index were also organized to run in the same cycle after the double resolution. Besides that, we observed that the new local difference in the vector could be stored in any step after the central prediction, and it was allocated to run together with the weight update in the last cycle.

In addition, the local difference and weights vectors were created using registers to store the values. This way, all values can be read in a single cycle instead of reading all values from RAM sequentially. In the encoder block, there is no need to access all the accumulator and counter values simultaneously, enabling block RAMs to implement these vectors, which tends to reduce logical resource usage.

D. SoC Design

We used a Zedboard Zynq-7000 with an ARM Cortex-A9 to prototype the developed compressor. Fig. 4 illustrates the system integrated with the compressor via an AXI4-Lite communication interface. In addition, the compressor has an interrupt signal connected to the global interrupt controller of the processor to inform when a sample has been compressed. The ARM processor reads the image in BIP mode from an SD card and buffers the input samples to correctly send the sample and neighbor values according to the column-oriented local sum mode. This buffering is done according to the size of the image and is not implemented internally in hardware. As the accelerator each image sample, the interrupt signal is active, telling ARM processor that the compressed sample can and back memory.

RESULTS

A pattern test image from AVIRIS sensor, provided by CCSDS, was used for the compressor validation. The final compression result was compared with the reference implementation of the Empord à software, developed by [START_REF] Gici | Gici emporda ccsds 123.0-b-1[END_REF]. The synthesis and performance results were collected using the Xilinx Vivado 2022.2 for the HDL solution and Xilinx Vitis HLS 2022.2 for the HLS solution.

Results for both solutions in terms of resource utilization, maximum frequency, and dissipated power were obtained for different values of bands for a hyperspectral image, considering the range for sensors up to 512 spectral bands, covering hyperspectral sensors like AVIRIS-NG (480 bands).

A. HLS Results

Table II presents the results obtained from the HLS solution for different N z bands. The HLS tool generated a hardware solution that takes 11 cycles to process a sample. Thus, the throughput for different band values is approximately 9 MSa/s and varies a little due to the fact that the frequency also changes. The use of LUTs (Look-Up Tables) and FFs (Flip-Flops) has increased as the number of bands has also increased, but no Block Randomic Access Memorys (BRAMs) were allocated. This happened because the tool automatically inferred registers instead of BRAMs. In the prediction block, the use of local difference vectors and weights was synthesized as registers to have access to all its values in a single clock cycle. On the other hand, in the encoding block, only one value of the vector is accessed at a time, justifying the increase in resource utilization as the number of bands increases.

To get around this, the bind storage directive can force the tool to allocate these vectors to use BRAMs since only one value of each vector is accessed at a time. The results in Table III show that by applying the directive we can keep the same throughput achieved previously and make use of the BRAMs. When comparing the solutions for N z = 2 9 bands, we observe a reduction of 8× in the use of LUTs and 7× in the use of FFs, besides a reduction in the power dissipated from 262 mW to 149 mW, making the solution more energy efficient. We can highlight that the force use of BRAMs instead of registers does not bring differences in terms of reliability, since they have similar sensitiveness to single events. On the other hand, the results have substantial outcomes in terms of resources and power overhead. However, aiming to have a solution using fewer resources, we forced the tool not to apply the unroll and pipeline directives that are being applied. Table IV presents the results for N z = 2 9 for the standard configuration in comparison with the previous optimized solution in Table III. When forcing the tool to not apply directives in the solution, we observe that there is a reduction in the use of resources in terms of LUTs and FFs. However, there is an increase in the number of DSPs used, and the solution has a throughput about 3× lower because the HLS standard solution takes 36 cycles to process a sample.

With these results, we can point out that even using unroll and pipeline directives, which tend to increase resource utilization, may not increase the entire resource utilization of the solution as seen in Table IV. We could observe an increase in LUTs and FFs but a reduction in the use of DSPs, besides getting a higher throughput, 9.11 against 3.37 MSa/s. This way, the application of optimization directives are good alternatives to be explored aiming at a trade-off between performance and resources.

B. HDL Results

Table V presents the resource utilization obtained from the HDL solution for different spectral band values. The HDL solution uses almost the same DSPs and BRAMs as the HLS. However, the use of LUTs and FFs does not increase so much as the number of bands also increases, in contrast to the HLS implementation. Moreover, the HDL solution considering the higher number of bands N z = 2 9 , has a throughput of 21.47 versus 9.11 MSa/s and uses about 3× fewer LUTs and 8× fewer FFs than the HLS solution. This reduction of resources, especially regarding FFs, represents a good result regarding reliability facing Single Event Upsets (SEUs) since FFs are one of the most sensitive parts of circuits in the space environment. Moreover, unlike Single Event Transients (SETs), in FFs, the bit-flips remain stored and potentially create problems up to the FF update [START_REF] Sorin | Fault Tolerant Computer Architecture[END_REF].

The achieved throughput of 21.47 MSa/s was possible because the solution takes 5 cycles to process a sample. This is due to the HDL architecture design considering the parallelization of some steps that did not have a dependency and a control unit that made the predictor and encoder blocks run simultaneously.

C. SoC Results

The compressor was also tested on the Zynq-7000 SoC to measure the acceleration of the compression application, as shown in Fig. 4. The system was initially configured to perform image compression only by the ARM Cortex-A9 processor in a software solution. Then, the ARM processor is used only to send the samples to the accelerator, which controls the entire compression step.

The results in Table VI highlight that the compressor in HLS and HDL accelerated the HSI compression application by 1.6× and 4× compared to the ARM processor running at 667 MHz. The solution accelerates the application by moving the routines from software to dedicated hardware even when using the AXI4-Lite communication interface, which requires the processor to send and receive samples directly to the compressor over a single channel, which slows the communication. Furthermore, while an HDL implementation performs better than an HLS implementation, the design time also differs. The HDL implementation took approximately 5× longer than the HLS implementation for this project, which could favor projects looking for fast verification and design iteration instead of optimal performance.

D. Comparison with Related Works

Table VII presents the results of the HLS and HDL implementations with the related works, where the implementations of this work considered those obtained with the higher number of spectral bands N z = 2 9 .

The HDL implementation has the lowest resource utilization of any other related works, even versions that implement lossless or near-lossless compression. The HDL solution of this work uses about 12× fewer LUTs and 40× fewer FFs than [START_REF] Gonzalez | A real-time FPGA implementation of the ccsds 123.0-b-2 standard[END_REF], and 13× fewer LUTs and 30× fewer FFs than [START_REF] Barrios | Hardware implementation of the ccsds 123.0-b-2 near-lossless compression standard following an hls design methodology[END_REF]. On the other hand, these works have a higher throughput than this work and implement the near-lossless version of the standard.

When compared with [START_REF] Pereira | A low-cost hardware accelerator for ccsds 123 predictor in FPGA[END_REF] and [START_REF] Orlandić | A parallel FPGA implementation of the ccsds-123 compression algorithm[END_REF], which also implement the same lossless version, we observe that the HDL solution has a resource utilization about 2× lower and higher throughput than [START_REF] Pereira | A low-cost hardware accelerator for ccsds 123 predictor in FPGA[END_REF], which implements only the prediction step of the compressor. Compared to [START_REF] Orlandić | A parallel FPGA implementation of the ccsds-123 compression algorithm[END_REF], the HDL solution uses 11× fewer LUTs and 33× fewer FFs, but has a much lower throughput since [START_REF] Orlandić | A parallel FPGA implementation of the ccsds-123 compression algorithm[END_REF] focuses on a highly parallelized implementation to obtain higher throughput.

Works [START_REF] Tsigkanos | A 3.3 Gbps ccsds 123.0-b-1 multispectral & hyperspectral image compression hardware accelerator on a space-grade SRAM FPGA[END_REF] and [START_REF] Barrios | Shyloc 2.0: A versatile hardware solution for on-board data and hyperspectral image compression on future space missions[END_REF] achieved a throughput about 10× and 7× higher than our HDL solution, since these works have a high data rate bus interface and an internal buffering architecture capable of processing one sample per cycle.

The implementations in this work show the use of resources lower than the related works but also a lower throughput. However, it is important to highlight that even though the related works have a better energy efficiency due to their high throughput, the total dissipated power of the implementation is about 5 to 7 times higher than this work, which may not meet requirements for low power applications.

V. CONCLUSION This paper presents lossless HSI compressors compatible with the CCSDS 123.0-B-2 standard described using HLS and manually in VHDL. The implementations were designed to reduce development complexity and low resource utilization for applications in space systems.

We compared the achieved performance with works developed in HDL and HLS. The HDL implementation presented a higher performance and lower resource utilization than the HLS solution. Directives applied to the HLS solution showed good results regarding resources and power overhead. The HDL solution shows good reliability when facing SEUs due to the low utilization of resources compared to the HLS solution and related works. In addition, both solutions accelerated the compression application, enabling the use of accelerators.

For future work, we plan to use an AXI4-Stream interface with a DMA system to speed up the compression application and integrate the compressor into a multi-core system. We also plan to apply fault tolerance techniques to the implemented compressor for further fault injection campaign evaluation.

Fig. 1 .

 1 Fig. 1. Predictor component design.

Fig. 2 .

 2 Fig. 2. Encoder component design.

Fig. 3 .

 3 Fig. 3. Processing order and local difference vector storage process.

Fig. 4 .

 4 Fig. 4. SoC design overview.

TABLE I COMPRESSOR

 I DESIGN AND IMPLEMENTATION CHOICES.

	Parameter	Specification	Implementation
	Compression mode Lossless, Near-lossless	Lossless
	Prediction mode	Full, Reduced	Reduced
	Local sum mode	Wide, Narrow, Column	Column
	Processing order	BSQ, BIL, BIP	BIP
	Entropy encoder	Block, Sample, Hybrid	Sample-Adaptative

TABLE II HLS

 II RESOURCE UTILIZATION.

	Nz	LUTs	FFs DSPs BRAMs	Fmax (MHz)	Power (mW)	Troughtput (MSa/s)
	2 5	4,139	3,832	4	0	101.01	159	9.18
	2 6	5,813	5,058	4	0	101.73	164	9.25
	2 7	9,842	6,825	4	0	103.23	170	9.38
	2 8 16,123 12,780	4	0	101.78	200	9.25
	2 9 30,272 22,843	4	0	101.75	262	9.25

TABLE III HLS

 III RESOURCE UTILIZATION USING bind storage DIRECTIVE

	Nz LUTs	FFs DSPs BRAMs	Fmax (MHz)	Power (mW)	Troughtput (MSa/s)
	2 5 2,887 2,604	4	1	101.36	144	9.21
	2 6 3,038 2,702	4	1	107.52	141	9.77
	2 7 3,140 2,832	4	2	104.64	141	9.51
	2 8 3,373 3,089	4	2	103.10	143	9.37
	2 9 3,819 3,091	4	4	100.20	149	9.11

TABLE V HDL

 V UTILIZATION RESOURCES FOR DIFFERENT NUMBER OF BANDS.

	Nz LUTs FFs DSPs BRAMs	Fmax (MHz)	Power (mW)	Troughtput (MSa/s)
	2 5 1,314 378	4	1	107.15	119	21.43
	2 6 1,320 382	4	1.5	107.52	120	21.50
	2 7 1,322 384	4	2	107.32	120	21.46
	2 8 1,320 384	4	2.5	107.12	119	21.42
	2 9 1,321 386	4	4	107.34	120	21.47

TABLE VII SYNTHESIS

 VII AND PERFORMANCE RESULTS IN COMPARISON WITH RELATED WORKS.

	Work	Implementation	CCSDS	FPGA	LUTs	FFs DSPs BRAMs	Fmax (MHz)	Power (mW)	Troughtput (MSa/s)	Energy * (mJ)
	[7]	VHDL	lossless	Virtex-5 FX130T	9,462	9,990	6	83	213	-	213.00	-
	[8]	VHDL	lossless	Zynq-7000	2,244	630	3	-	142	106	20.40	5.22
	[9]	VHDL	lossless	Zynq-7035	14,709 12,830	-	37	150	515	750.00	0.68
	[10]	VHDL	lossless	Zynq-7035	4,619	2,765	8	74	151	-	151.00	-
	[11]	VHDL	near-lossless	Virtex-7 VC709	16,458 15,707	30	187	250	732	250.00	2.93
	[12]	HLS	near-lossless	Kintex XCKU40	17,185 11,915	63	85	125	500	12.50	40.00
	HLS std.	HLS	lossless	Zynq-7000	2,577	2,428	15	4	121	145	3.37	43.15
	HLS opt.	HLS	lossless	Zynq-7000	3,819	3,091	4	4	100	149	9.11	16.39
	HDL	VHDL	lossless	Zynq-7000	1,321	386	4	4	107	120	21.47	5.56

* Maximum estimated energy consumed to process 1MSa.

This work was supported in part by the Foundation for Support of Research and Innovation, Santa Catarina (FAPESC-2021TR001907), the Brazilian National Council for Scientific and Technological Development (CNPq -processes 138179/2021-2, 140368/2021-3 and 350794/2023-5), and the Region d'Occitanie and the École Doctorale I2S from the University of Montpellier (contract 20007368/ALDOCT-000932).


Benjamin W Mezger

Douglas A Santos

Luigi Dilillo
email: luigi.dilillo@umontpellier.fr

Douglas R Melo

Hardening a Real-Time Operating System for a Dependable RISC-V System-on-Chip

Keywords: Systems-on-Chip, Fault Tolerance, Software Reliability, RISC-V, FreeRTOS

In safety-critical systems, solutions that rely on redundancies at the hardware and software levels allow these systems to operate in radiation-harsh environments. In the literature, software-based techniques for enhancing reliability in critical systems are less developed when compared to hardware-based fault-tolerant techniques. In this context, we implemented error correction and detection techniques through software, along with a custom RISC-V processor-core implementation. RISC-V is an open instruction set architecture that is gaining popularity due to its modular design, as well as the FreeRTOS kernel, which is a well-established Real-Time Operating System. Our implemented proposal has shown to be able to reduce the number of execution failures by four times when compared to the standard version at the cost of increased execution time and energy consumption. The experimental results may guide designers in choosing the best trade-offs between reliability and resource constraints in complex contexts such as space applications.

I. INTRODUCTION

RISC-V is an open standard architecture based on the Reduced Instruction Set Computer (RISC) principles, initially developed by Berkeley University but maintained by the RISC-V foundation. RISC-V provides a base instruction set with several optional extensions, reducing software costs and enabling implementations of large-and small-scale architectures [START_REF] Mezger | A survey of the risc-v architecture software support[END_REF]. Semico Research Corp. estimates that in 2025, the market will have around 62.4 billion RISC-V cores worldwide [START_REF] Mixon | Semico Forecasts Strong Growth for RISC-V[END_REF].

RISC-V's software support is increasing in multiple computing fields, such as Operating System (OS), on which the Linux kernel [START_REF] Torvalds | RISC-V Port for Linux 4.15 v9[END_REF] and Zephyr [START_REF] Lockwood | Zephyr Project Blog Post: Zephyr RTOS Featured In RISC-V Getting Started Guide[END_REF] already have support, as well as compilers and simulation software. In addition to a general-purpose OS, systems with additional characteristics, such as real-time systems provided by FreeRTOS [START_REF] Barr | RISC-V Support in the FreeRTOS Kernel[END_REF] are also available. However, approaches focusing on delivering reliability are not near the same number in RISC-V.

An operating system may run in environments that demand reliability techniques. These systems can experience external interference, impacting the system's availability, security, integrity, or maintainability [START_REF] Avizienis | Basic concepts and taxonomy of dependable and secure computing[END_REF]. For these systems to operate correctly in the presence of a fault, the system may provide reliability at the hardware or software levels.

Several works in the literature tackle reliability characteristics at the software level, such as the work [START_REF] Laurent | Fault injection on hidden registers in a risc-v rocket processor and software countermeasures[END_REF], where the authors inject single bit-flips to simulate faulty behavior in the RISC-V multiplier unit. The work [START_REF] Bohman | Microcontroller compiler-assisted software fault tolerance[END_REF] implements reliability techniques for software by adding fault mitigations to instructions and/or data at the compiler level. The work [START_REF] Adelt | A scalable platform for qemu based fault effect analysis for risc-v hardware architectures[END_REF] proposes the simulation of errors in a set of RISC-V compiled C programs to evaluate the impact of software execution when injecting permanent and transient faults. Moreover, the work [START_REF] Wali | Analyzing the impact of the operating system on the reliability of a risc-v fpga implementation[END_REF] evaluates the impact of injecting single bit-flips in the configuration memory in a Linux-based system. Finally, [START_REF] Sio | Seu evaluation of hardened-by-replication software in risc-v soft processor[END_REF] analyses the pros and cons of hardened-by-replication software applications running on soft-core microprocessors.

Solutions explore redundancy or rely on fault-tolerant hardware components to guarantee reliability for safetycritical applications. Hardened hardware components are not always cost-effective, nor do they always provide the specificity a system needs. Therefore, this work explores software-based reliability techniques and proposes applying a technique at OS level on a RISC-V architecture. Furthermore, we propose a set of modifications of a RISC-V processor to monitor and communicate the OS of a detected fault.

Given this context, we port FreeRTOS to a reliable RISC-V processor and implement the required exceptions and interrupts for supporting the OS. We implement process-level Triple Module Redundancy (TMR), which blocks the running tasks and waits until all task values have been compared. Further, we implement fault notification through a custom RISC-V exception to allow the OS to log, reset and do any required post-processing when a fault is detected.

II. BACKGROUND

Compilation of

FreeRTOS Generate file

Load generated file before simulation

Simulate

Analyze results

Fig. 1. HARV and FreeRTOS compilation and simulation process.

HArdened RISC-V (HARV) [START_REF] Santos | Reliability analysis of a fault-tolerant risc-v system-on-chip[END_REF] is a low-cost fault-tolerant, single-cycle RISC-V processor based on the RV32I instruction set of the non-privileged specification. The HARV processor is described in VHDL without any vendorspecific Intellectual Property (IP) blocks. Furthermore, this processor has been thoroughly analyzed in irradiation facilities and provides detailed information on radiation-induced errors [START_REF] Santos | Enhancing fault awareness and reliability of a fault-tolerant risc-v system-on-chip[END_REF]. Until now, HARV was only used in specific and dedicated applications running in bare-metal.

FreeRTOS is a real-time OS for microcontrollers designed to execute user-defined tasks within a strict period. FreeRTOS's architecture is structured into three main layers: (i) the tasks layer, which deals with creating userdefined tasks and scheduling maintenance, (ii) the communication layer, allowing tasks and interrupts to send data to each other and signal resources, and (iii), the shim layer, which acts between the hardware-independent and -dependent code [START_REF] Brown | The Architecture of Open Source Applications: Elegance, Evolution, and a Few Fearless Hacks[END_REF].

Since HARV did not provide support to any OS, we ported FreeRTOS to increase its software support. We used the existing FreeRTOS RISC-V port available in the main repository and modified the source code to adapt to HARV's current architecture. The RISC-V port facilitated the modification and validation of the necessary parts of the source code while hiding details that were not the main focus of this work.

To simulate the compiled binary and validate our port, we ran a script against the binary file to convert it from a binary to a text with the instruction format and data in hexadecimal format. The simulation then parses all lines of the generated file and executes the instructions. This flow is illustrated in Fig. 1.

Furthermore, given that HARV implements the RV32I instruction set, this work used the open-source FreeRTOS RISC-V implementation [START_REF] Amazon | FreeRTOS Kernel[END_REF] as the baseline of the port to make the necessary modifications.

III. OPERATING SYSTEM HARDENING

To add support to FreeRTOS in the HARV architecture, modifications were made to support basic functionalities of the FreeRTOS kernel, both in hardware (HARV) and in software (FreeRTOS). We identified three structures of Amazon's FreeRTOS port that required changes to adapt to the HARV processor:

1) The vPortSetupTimerInterrupt to set up the timer interrupt;

2) portcontextSAVE_INTERRUPT_CONTEXT to handle a context switch and save the current context;

3) The linker script provided by FreeRTOS source demo for the linking phase.

The FreeRTOS vPortSetupTimerInterrupt function required changes on the mtime register, given that in RISC-V specification, it is memory-mapped and in HARV is an actual Control Status Register (CSR). A change to portcontextSAVE_INTERRUPT_CONTEXT procedure was required to support the mstatus CSR, given that mstatus was not recognized by the compiler during the compilation phase, which required us to change the CSR register to its correct address in HARV. Finally, the linker script required CSR, and Read-Only Memory (ROM) addresses modification to find the initial address of the program.

Our findings later showed that Amazon's RISC-V version of FreeRTOS distribution was out of date along with the FreeRTOS kernel repository. The changes were later implemented along with refactoring on trap handling and context switching1 . To overcome such problems, we copied the new refactor of the FreeRTOS Kernel source code to the FreeRTOS distribution source. This source code replacement was enough to get HARV to execute the demo software provided by FreeRTOS.

While the RISC-V specification defines a range of exception codes, HARV had no support for software traps and timer interrupts. Four traps, shown in Table I, were identified to be relevant to HARV to support FreeRTOS.

To implement the trap support, a new set of CSRs is required, including (i) mie CSR for read/write of interrupt enable bits, (ii) memory-mapped mtime real-time cycle counter, (iii) mcause for indicating the event that caused a trap and (iv ) mtval for assisting software with exception-specific information. The implemented CSRs, and their privilege level are shown in Table II.

The new trap encoder unit is connected to the Instruction Fetch (IF), Control, and ALU unit. The trap encoder handles timer interrupt by comparing if the current timer (mtime) value is greater or equal to the value set in

A. Fault trap support

Given the HARV processor has support for fault detection and correction through means of TMR and Hamming, notifying the OS of a fault enables handling fault mitigation at the software level or running any post-processing mechanism. Post-processing mechanisms can include logging fault information, fault correction, or running safety checks to decide whether the system should halt or continue running.

To support fault mitigation in HARV at the software level, we add support to a new RISC-V custom asynchronous exception. We use the exception code 18 because it is designated for platform use. We modify HARV's Hamming and TMR component to raise an asynchronous exception and set mcause to value 18 to notify the OS of a hardwaredetected fault. Since not all designs require trapping on every fault, we can enable or disable this feature during compilation by defining the FAULT_TRAP_HANDLER flag with the function's name.

B. Software-based fault tolerance

To further harden the system, we applied TMR to correct errors based on different task output results. In our proposal, if the process has no redundancy mechanism, we run the process and continue without any modification.

Otherwise, our task-based TMR runs a copy of three processes with independent memory space and then compares the output of all three tasks byte-by-byte. We run an optionally provided post-processing mechanism if a fault is detected during the voting procedure.

To synchronize the output of the tasks, a private queue is used to store the computed values. Each task replica is responsible for inserting the shared value into the queue and blocking it until all other tasks have inserted their value. When all processes have synchronized, we compare the values inserted in the queue byte-by-byte and trigger an optional exception handler if two values do not match. Further, our solution ensures that the value address is within the memory range if a fault changes the address value to an out-of-range address.

Fig. 3 shows a flowchart of the task's synchronization and voting procedures, including inserting a value into the queue, blocking, and context switching to another task. Finally, when the queue is full, we compare the results and trigger a fault trap if enabled by the user and unblock any blocked tasks waiting for the result. III, with functions and constants for users to synchronize tasks and check for faults affecting data.

IV. RESULTS

This section presents the results related to hardware and software cost, performance, and dependability obtained through the fault injection campaign.

A. Cost

We used Xilinx Vivado 2021.1 to synthesize the HARV processor targeting the Xilinx 7000 FPGA. Table IV presents the cost in terms of resources for the synthesis of the HARV before and after the addition of the trap encoder. The hardware upgrade increased 56.31% in the number of FFs and led to an increase of 46.03% in LUTs compared to the baseline, which ended up degrading the operational frequency in 0.8%. To analyze software resources, we used FreeRTOS standard toolchain (GNU GCC) to compile and evaluate binary segment size. Table V presents the cost of binary size (in bytes) for the standard and hardened FreeRTOS versions. We had an increase of 3.78% in terms of code (section .text) and an increase of 18.87% in read-only data (.rodata). Considering the total size of the binary, the hardened version of the FreeRTOS presents an increase of 1.90% in the segment with respect to the standard version.

B. Performance

We used Xilinx Vivado's simulator to evaluate the modified HARV in terms of performance for standard and hardened versions of FreeRTOS. The simulation gave as result the total number of cycles for each execution. Taking into account the frequency at which HARV operates (detailed in Table IV), the performance results in Table VI show that the total execution time for the hardened increases approximately two times compared to the standard FreeRTOS version. A similar analysis can be made for energy consumption, considering the total power presented in Table IV. The standard version does not execute vTmrInit and vTmrCompare functions, as it does not initialize any TMR tasks, and neither calls the TMR compare to check all task values. The call to iTmrInsertValue has the highest number of cycles, given it blocks the process until all values are fed to the queue and compared, thus blocking each process at each call in case the queue is not filled. Further, the vTmrCompare function is dependent on the size of the value that is being compared, while iTmrInsertValue is dependent on the number of cycles vTmrCompare requires.

C. Experimental Setup

To simulate the fault occurrence for evaluating the effectiveness of our software-based hardening proposal, we made a fault injection campaign that inserts Single-Event Upset (SEU) faults in random flip-flops at random times. We used Intel ModelSim SE 2019.01 to simulate faults through a TCL script. To run the simulations, we used a server running CentOS, with 512 gigabytes RAM and a 96-core Intel (Haswell) CPU, with a 64-bit architecture.

The first step of the simulation is the execution of a golden run, which executes once the entire application without fault injections. This run provides baseline indicators for the comparison with the subsequent simulations, i.e., execution time, system flip-flops, and UART output. These indicators are also used as parameters for the fault injections.

The fault injection is based on a neutron-characterized flip-flop FIT NYC (Failure In Time for a billion hours in the New York City's neutron flux at sea level) of 248 and a neutron flux of 5 × 10 10 n/cm 2 /s. The computation of the fault injections is presented in Fig. 4, in which the script steps over the execution time in random increments calculating a fluence estimation for each time delta. We use the total fluence to calculate the error rate for each flip-flop during this period, which uses uniform randomization to decide whether the fault should be injected.

Regarding workload, we implemented three FreeRTOS tasks that iterate until a given number and multiply the number by 2 through adder, multiplier, and bit-shifting. Since the tasks that we implemented are simple operations, instead of keeping the result of the multiplication stored in a variable, we iterated through an array of 512 positions and store the generated value in all array positions, to simulate an intensive computational operation and increase the chances of getting an error propagation.

D. Fault injection campaign

We ran each simulation with a set of 1000 executions. Each execution creates a uart.log log file with all the output and the simulation time that were next used to analyze the simulation results. At the end of all 1000 executions, we ran a Python script for all the log files to analyze the results obtained for each simulation. The Python script verifies which execution finished correctly and which did not and how many faults were detected and corrected by the software.

version takes approximately two times more for its execution in comparison to the standard version, exposing the system to more errors compared to the standard version in the simulation. Considering the detected errors, the hardened FreeRTOS detected 765 errors from 933 executions with errors. Among the 765 executions with errors, 22 errors were not only detected but also corrected.

Considering the reliability analysis, the standard version had 694 executions with errors propagated into failure because of the absence of detection and detection mechanisms. The hardened FreeRTOS presented only 16.8% of executions with failure, in which the system was not able to neither detect nor correct the error. Despite the execution taking twice as long exposing the system to twice the number of injected errors, the experiments show that the hardened version of FreeRTOS was able to reduce the number of execution failures by 4 times, since it was able to detect the error before it would provoke a failure.

E. Discussion

To achieve the goals of this work, modifications to the processor and OS were necessary. These modifications had a significant increase in the logical resources and a small increase in terms of binary size when compiling the OS with hardened features.

Concerning performance, we noticed the execution time for the hardened version doubled. We also observed that the execution time increase is mainly due to the TMR usage, which blocks all tasks until it finishes executing. Moreover, the input size that will be compared by the TMR module will impact the execution time since the module will compare byte-by-byte of each input.

Concerning reliability, we made a fault injection campaign that randomly inserted SEUs. We ran 1000 executions for each simulation, in which we observed that the hardened version presented more executions with errors due to the increase in execution time. Besides this drawback, the hardened FreeRTOS presented a reduction of the number of executions failure by four times. In applications where the main requirement is reliability, this hardened version presents a good trade-off between execution time and fault tolerance.

V. CONCLUSION

This work aimed to provide a software-level hardening solution using FreeRTOS in a RISC-V architecture. For this purpose, we modified the HARV processor core to provide trap handling and fault notification through a custom exception. Further, we added TMR support for FreeRTOS tasks, including registering a fault trap handler in case the HARV detected a fault.

Fig. 3 .

 3 Fig. 3. Task-level TMR flow. The TMR functionality further provides N-version support by enabling the programmer to specify different versions of the function for redundant execution. With support for N-version functions, one can benefit from TMR by running the

TABLE I EXCEPTIONS

 I AND INTERRUPTS IMPLEMENTED. Furthermore, the trap encoder writes any pending interrupts before encoding the trap matching any exception shown in TableIand writing the mcause if applicable. The CSR components were extended to support trap-related CSR. Fig.2illustrate the HARV organization after the inclusion of the trap encoder.

			Exception type Exception name
			Synchronous	Environment call from Machine-Mode
			Synchronous	Load access fault
			Synchronous	Store access fault
			Asynchronous	Machine timer interrupt
					TABLE II
				IMPLEMENTED CSRS IN HARV.
	Number	Privilege	Name	Description
	0x304	Machine R/W	mie	Interrupt Enable
	0x342	Machine R/W	mcause	Trap Cause
	0x343	Machine R/W mtval	Trap Value
	Memory-mapped Machine R/W mtime and	Timer
			mtimecmp
	Memory mtimecmp. Instruction	Instruction Fetch		Control Register File	ALU Trap Encoder	Data Memory
						CSR
			Fig. 2. HARV organization with trap support.

TABLE V BINARY

 V SIZE OF FREERTOS STANDARD AND HARDENED.

	Segment (in bytes) FreeRTOS standard FreeRTOS hardened
	.text	69880	72524
	.rodata	1992	2368
	.data	28	28
	.bss	73188	73188
	.stack	12284	12284
	Total	158164	161184

TABLE VI PERFORMANCE

 VI RESULTS.Table VII presents the performance overhead of cycles introduced by each TMR function to provide fault tolerance. For this measurement, the cycle count is fetched before and after entering the function call.

	FreeRTOS	Cycles	Execution time (s) Energy (mJ)
	Standard	8.54 × 10 7	1.61	380.95
	Hardened	1.73 × 10 8	3.27	771.24

TABLE VII CYCLE

 VII COUNT PER API FUNCTION WITH TMR ENABLED.

	Function	Cycle start	Cycle end	Total
	vTmrInit	1,583,504	1,854,083	270,579
	iTmrInsertValue	6,478,322 11,992,678 5,514,356
	vTmrCompare	7,072,730 11,783,849 4,711,119

This work was supported in part by the Foundation for Support of Research and Innovation, Santa Catarina (FAPESC-2021TR001907), and the Brazilian National Council for Scientific and Technological Development (CNPq -process 350794/2023-5), and the Region d'Occitanie and the École Doctorale I2S from the University of Montpellier (contract 20007368/ALDOCT-000932).

Pull Request can be found at https://github.com/FreeRTOS/FreeRTOS-Kernel/pull/444

The experiments showed the hardened FreeRTOS capability to reduce executions failure four times lower than the standard one. This performance was obtained at the price of increased energy consumption and execution time. This trade-off can be considered suitable for applications where reliability has a higher priority than other metrics.

In future work, we intend to submit the proposed implementation to a fault injection campaign in particle accelerators and compare fault-tolerant techniques at the hardware level and techniques implemented at the software level.

Fetches register signals

A full simulation (1000 executions) took 120-150 hours to complete. In total, the simulations took around 600 hours (25 days). The results of all simulations are shown in Table VIII. Of the 1000 executions for each simulation, we observed the standard version had 694 executions with errors, while the hardened version had 933 executions with errors and an increase of 34.43%. This is because the hardened


Douglas A Santos

M P Mattos

Douglas R Melo

Luigi Dilillo
email: luigi.dilillo@umontpellier.fr

Characterization of a Fault-Tolerant RISC-V System-on-Chip for Space Environments

Keywords: RISC-V, System-on-Chip, Fault Tolerance, Radiation Effects, Single-Event Effects, Total Ionizing Dose, Protons

Processing units are sensitive to the harsh radiation conditions present in space applications. Thus, radiation testing is a mandatory step toward high reliability for these systems. Notably, as an emerging processor architecture, RISC-V has attained interest for its utilization in these applications. For this reason, in previous work, we presented a faulttolerant RISC-V System-on-Chip (SoC) with enhanced fault awareness, known as HARV-SoC, to attend to this increasing demand. The SoC includes structures to detect, correct, and report radiation-induced errors. In this work, we characterize HARV-SoC under proton irradiation, enabling the proposition of space applications based on a low-cost and open-source RISC-V solution. We observed and reported the effects of Single Event Effects (SEEs) and Total Ionizing Dose (TID) in the entire SoC, individually evaluating the HARV core, SoC elements, and external memories, with a detailed analysis of error propagation.

I. INTRODUCTION

Designing systems for space applications is a challenging task due to various strict requirements and harsh operational conditions. In orbit, a serious threat to these systems is the degradation caused by radiation. Ionizing particles constantly interact with the system's components and materials, leading to a plethora of effects. In some cases, these particles are capable of inducing faults, either due to single interactions, a Single Event Effect (SEE), or caused by their accumulative effect, Total Ionizing Dose (TID) and Displacement Damage (DD). The induced faults can be transient, intermittent, or permanent [START_REF] Yang | Fault-tolerance techniques for spacecraft control computers[END_REF].

In order to understand these effects and enable mitigation techniques to be applied, tests are prepared to characterize these systems under irradiation [START_REF] Coronetti | Radiation hardness assurance through system-level testing: Risk acceptance, facility requirements, test methodology, and data exploitation[END_REF]. For that, radiation conditions are recreated in particle accelerators, where users have the ability to change important parameters in a controlled environment for accelerated experimental analysis [START_REF] Rajkowski | Radiation qualification by means of the system-level testing: Opportunities and limitations[END_REF]. These facilities usually provide specific particle species and spectra that allow reliability assessment of applications targeting a certain environment. For instance, for representative SEE characterization for space applications, proton and heavy-ion characterization is suggested. In space standards [START_REF]Single Event Effects Test Method and Guidelines, European Space Components Coordination (ESCC) Std. ESCC Basic Specification[END_REF], [START_REF]Space product assurance: Radiation hardness assurance -EEE components[END_REF], this recommendation is further defined, and its applicability is discussed.

An emerging trend in the last years has been the employment of Commercial Off-The-Shelf (COTS) components in modern space developments with higher risk acceptance [START_REF] Budroweit | Risk assessment for the use of COTS devices in space systems under consideration of radiation effects[END_REF]. This movement is mainly motivated by the high cost, long lead times, and limited performance of traditional radiation-hardened components. However, to mitigate the several reliability challenges raised by this approach, developers often exploit risk acceptance by applying fault tolerance techniques on their COTS-based systems. They apply strategies and techniques that usually rely on redundancy, using temporal, spatial, and informational characteristics of a system [START_REF] Sorin | Fault tolerant computer architecture[END_REF].

Processing units are a central element of complex space systems, for which COTS alternatives have been applied and investigated. In particular, RISC-V [START_REF]RISC-V: The free and open RISC instruction set architecture[END_REF] is an emerging processor architecture that has attained an important role in various application domains, reaching even space systems as a promising alternative for future developments [START_REF] Furano | A european roadmap to leverage RISC-V in space applications[END_REF]. For this reason, we started the development of a RISC-V processor prospecting space applications in prior work [START_REF] Santos | A low-cost fault-tolerant RISC-V processor for space systems[END_REF]. Then, the processor was extended to become a robust SoC (known as HARV-SoC) with enhanced fault observability and monitoring for proper experimental validation, in which extensive tests were performed in various irradiation campaigns [START_REF] Santos | Enhancing fault awareness and reliability of a fault-tolerant RISC-V system-on-chip[END_REF].

In this work, we continue the validation effort by extending HARV-SoC characterization with high-energy protons. Thus, we present the experimental preparation, acquired results, and post-analysis, contributing to a more comprehensive understanding of the SoC under harsh radiation conditions. As aforementioned, the characterization of protons is an important step towards enabling the usage of systems in space applications. Therefore, we aim to present to the community the robustness of HARV-SoC as an alternative COTS solution targeting in-orbit applications.

This paper is organized as follows. The fault-tolerant RISC-V SoC is presented in Section II. Section III describes the irradiation facility and the proposed experiment. Section IV presents results for the proton irradiation. In Section V, we discuss these results. Finally, Section VI provides the conclusions.

II. FAULT-TOLERANT RISC-V SYSTEM-ON-CHIP

The RISC-V System-on-Chip tested in this work (HARV-SoC) focuses on reliability for harsh environments [START_REF] Santos | Enhancing fault awareness and reliability of a fault-tolerant RISC-V system-on-chip[END_REF]. The SoC is based on HARV (Hardened RISC-V) processor core [START_REF] Santos | A low-cost fault-tolerant RISC-V processor for space systems[END_REF], which applies hardening techniques on the architectural level to allow detecting, correcting, and reporting faults. HARV is a multi-cycle RISC-V-based processor core supporting the integer instruction set (RV32I), Control and Status Registers (CSRs), interrupts, and exceptions. The multi-cycle architecture is composed of five stages: instruction fetch, execution, memory access, write back, and trap handler. HARV applies techniques such as Error-Correcting Code (ECC) and Triple-Modular Redundancy (TMR) for the detection and correction of errors in its internal components. The register file, the program counter, and the instruction register are hardened using Single-Error Correction and Double-Error Detection (SECDED). The control, Arithmetic-Logic Unit (ALU), and critical CSRs use TMR for hardening. HARV also provides error reporting capabilities, which are provided through exceptions that synchronously interrupt the application flow and report detailed information on the detected error, such as the error type, application context, and corrupted data. These features improve the reliability of the SoC and support fault awareness at the application level.

HARV-SoC is composed of the HARV processor core and various peripherals (e.g., communication interfaces, memory controllers) integrated into an architecture targeting reliability with flexibility and low hardening overheads. This reliability is achieved by hardening critical internal components and providing recovery schemes, e.g., data memory with ECC, bus access timeout, reset controller, watchdog timer reset, and application checksum. For the data memory, the SoC uses an external SDRAM memory, accessed through a customized controller that enables SECDED for each 32-bit word. For access to peripherals, the bus controller provides a timeout flag that triggers an exception when a peripheral is not responding. With the bus timeout, the application receives information about the faulty peripheral. The application may use this information to perform actions to restore the functionality of a peripheral. For that, the reset controller is used, which enables resetting specific peripherals. Besides the timeout from the bus, the system also provides a watchdog timer that resets the SoC when a system timeout is reached, preventing hang processor failures.

Besides these hardening strategies, an application integrity verification is performed by the startup routine using a Cyclic Redundancy Check (CRC) algorithm. This enables the detection of errors while loading the program from external non-volatile memories. Furthermore, the reporting capabilities of the error monitor enable the application to perform actions according to the detected error. If the error is not correctable, the application uses the available information to decide, for example, to ignore it and continue the execution or even force a system reset to avoid faulty system behavior. Fig. 1 presents an overview of the SoC architecture, including the fault-tolerant HARV processor, its interconnection with peripherals, and the error monitor module, responsible for detection and reporting.

III. RADIATION EXPERIMENT

To evaluate the SoC reliability, we performed a proton irradiation campaign. The following subsections provide a description of the irradiation facility, experimental setup, and test modes. PARTREC [START_REF] Gerbershagen | Establishment of the new particle therapy research center (PARTREC) at UMCG groningen[END_REF] is a dedicated research facility located in Groningen, Netherlands. It includes an accelerator that is used for nuclear physics research and radiation-hardness testing of electronic systems. This facility operates a large superconducting cyclotron capable of generating high-energy protons up to 190 M eV with fluxes up to 1 × 10 8 p/cm 2 /s depending on the beam area. For this experiment, the beam configuration had a homogeneity uncertainty under 10 %. This type of irradiation provides the necessary conditions for triggering representative SEEs in the SUTs (Systems Under Test), the primary phenomena under investigation in this work. However, another effect is also observed: the accumulation of damage in the SUTs, known as TID. Both phenomena are presented and discussed in Section IV.

B. Experimental Setup

The experimental setup consisted of various boards hosting the SoC in flash-based FPGA devices and the auxiliary circuitry for operation (e.g., power supply, logging interfaces). the FPGAs and external SDRAM memory were irradiated, leaving the remaining devices outside the direct beam sight. In particular, the FPGA is the Microchip's SmartFusion2 M2S010 [START_REF]IGLOO2 and SmartFusion2 Datasheet, DS0128[END_REF], and the SDRAM memory is the Winbond W9864G6JT [START_REF]G6JT Datasheet[END_REF]. The FPGA configuration memory is based on flash cells, which are inherently more robust against SEEs than traditional FPGAs with SRAMbased cells, as shown by the manufacturer [START_REF]Single Event Effects -A Comparison of Configuration Upsets and Data Upsets, WP0203 White Paper[END_REF], [START_REF]IGLOO2 and SmartFusion2 65nm Commercial Flash FPGAs[END_REF]. Despite that, other internal structures of the chosen FPGA are still sensitive to SEEs, such as the Block RAMs (BRAMs), which are based on SRAM technology, and D-type Flip-Flops (DFFs), as shown in [START_REF]IGLOO2 and SmartFusion2 65nm Commercial Flash FPGAs[END_REF], [START_REF] Dsilva | Neutron SEE testing of the 65nm SmartFusion2 flash-based FPGA[END_REF]. For logging the experimental data, interfaces were implemented using a serial protocol available in the SoC architecture, in which each board had a direct and independent connection with a host computer. To achieve a representative characterization, i.e., a statistically significant number of radiation-induced effects, we used six boards for the irradiation. The boards were attached in frames without superposition and as close as possible to the center of the defined beam area. The supply current was individually monitored and limited in case of faulty behavior.

C. Test Modes

Each SUT executed a benchmark: EEMBC's CoreMark [START_REF] Gal-On | Exploring CoreMark a benchmark maximizing simplicity and efficacy, Embedded Microprocessor Benchmark Consortium[END_REF] and the open-source Embench [START_REF] Bennett | Embench: An evolving benchmark suite for embedded IoT computers from an academic-industrial cooperative[END_REF]. The usage of multiple benchmarks intends to improve the test stimuli, as discussed in [START_REF] Quinn | Using benchmarks for radiation testing of microprocessors and FPGAs[END_REF]. CoreMark is a consolidated performance benchmark that includes four main workloads: list processing, matrix manipulation, state machines, and CRC. The CRC algorithm is used as a workload and provides a self-checking scheme for the inner steps of the benchmark execution. CoreMark is executed in iterations and provides a final result check. Embench is a free and open-source performance benchmark and intends to provide a broader set of algorithms, resulting in twenty-two different workloads. We implemented a combination of these algorithms in a single test application, which uses a custom set of these algorithms due to memory size constraints.

For the proposed radiation experiment, we assigned the first three boards with CoreMark and the other three with Embench. The execution is performed in a cycling test mode, in which the benchmark runs a short number of iterations and outputs the final results, then the SoC performs a soft reset before starting a new cycle. Every cycle runs the processors with a different hardening configuration, i.e., reporting and corrections enabled or just the reporting. In this experiment, we have two hardening configurations: baseline, with correction disabled, except the SECDED in the external memory, and hardened, with all the available protections enabled. We kept the SECDED enabled in the baseline to reduce the number of memory-related errors, as observed to be the most contributing error in all the previous characterizations, such as [START_REF] Santos | Enhancing fault awareness and reliability of a fault-tolerant RISC-V system-on-chip[END_REF]. Moreover, these experiments consistently proved the importance of ECC in the system memory, leaving the opportunity for exploring other reliability aspects in this work.

It is worth mentioning that, for the first beam runs of this experiment, we experimented with different proton fluxes to define the most adequate for testing HARV-SoC. We performed the test in two stages: in the first, we irradiated four boards (SUTs #1, #2, #4, and #5) until unrecoverable failure, as deeply discussed in Section IV, with fluxes between 1 × 10 6 and 1 × 10 8 p/cm 2 /s, then we tested the remaining two boards (SUTs #3 and #6) following the same approach, but mostly with a flux of 2 × 10 8 p/cm 2 /s. Both stages included the same setup and procedures, with the exception that the second run used a single flux for most of the experiment instead of different ones based on the initial verification.

IV. RESULTS

This section reports the results and analysis of the test campaign, providing details on the observed proton-induced SEE and TID effects on the SUTs. We perform an initial investigation regarding the impact of TID on the SUTs and device failures to ensure that no bias is present in the SEE analysis due to accumulative effects. Then, we detail the detected errors and their cross sections, including an error propagation analysis.

A. Device Failure Analysis

During the test campaign, the SUTs accumulated a significant dose, which led to unrecoverable device failures. For this reason, the first analysis that we performed was to understand the impact of the accumulative effect on each SUT. However, it is important to note that this work focuses on the RISC-V SoC analysis regarding SEE effects as the opposite of device (FPGA) characterization, in which the TID analysis only is performed to ensure proper filtering of possibly biased data. Besides that, to report a more insightful result, we investigated the most probable cause for the TID limitation of the device.

In Fig. 2, we show the accumulative number of HARV Fault Tolerance Exceptions (HFTE) over the received dose in the test campaign. HFTEs are exceptions triggered in the processor related to the fault-tolerant structures in the HARV implementation. The curves are plotted until the last reported HFTE and the dose values are estimations since only doses per beam run were reported by the facility. As expected, an abrupt increase is observed at the end of the irradiation for most SUTs since faulty and critical behavior was noticed during the test campaign. After 25 krad, SUT #2 presented a rapid increase in the number of HFTEs, followed by SUT #6 and SUT #5 at 29 krad and 31 krad, respectively. The remaining SUTs shortly presented a similar behavior. Hence, due to a clear dose dependency for the number of HFTEs, we decided to adopt a worst-case approach, in which we defined the threshold for SEE analysis at 25 krad, avoiding biases and reducing experimental uncertainties. As mentioned, we further investigated an important behavior that may point to the root cause of the observed TID limitation. In most cases, moments before reaching an unrecoverable device failure, the SoC reported many CRC errors used to guarantee the integrity of the application code. As described, the integrity verification is executed during the startup routine, which is always called after a soft or hard reset, meaning a watchdog recovery attempt or a manual power cycle, respectively. Fig. 3 presents the accumulated number of CRC errors over the received dose in the test campaign. Similar to Fig. 2, the SUTs present a significant increase in CRC error at approximately 30 krad, corroborating the previous proposition of setting a conservative threshold for SEE analysis at 25 krad. It is worth mentioning that the curves were plotted until the last reported CRC error, which may differ from the last reported HFTE in Fig. 2. Since the application code is stored in an internal flash memory connected to the HARV through complex interfaces, our main hypothesis is that an element in this interface or in the controlling logic of this memory block failed with the accumulated doses.

Thus, to define the fluence and dose of the discussed unrecoverable device failure, we set the condition as the last execution of the SUT when even a power cycle was ineffective in restoring the operation. Table I presents these results for each SUT. Besides that, for comparison, we included the fluence used as a threshold for SEE analysis.

Finally, an evaluation of device failure throughout the test campaign was performed. The SUTs experienced failures that required external intervention with a complete power cycle. It is important to note that these failures are not recovered with soft reset attempts triggered by the SoC watchdog timer. Table II details the number of device failures per SUT. For the SUTs #1, #2, #4, and #5, we noticed a similar number of failures, while SUTs #3 and #6, which were tested in the second batch of the experiment, presented fewer failures.

B. Error Distribution

Before investigating the reported errors, an analysis was performed to verify the distribution of errors for each SUT. Table III summarizes the reported errors per board considering both benchmarks used in the experiment. From this

C. Error Classification

We used HARV reporting features to classify the detected HFTEs. This classification is shown in Table IV, which reports the number of each type of HFTE occurrence (#HFTE) and its specific cross sections (HFTE XS). It is worth mentioning that the results presented in this table consider the experiment before the permanent damage caused by the TID, defined as 25 krad.

The most frequent HFTE is single-bit upset from the SDRAM memory, representing 57.9% of all HFTEs, while memory double-bit upsets account for 2.7% of the total. Although all single-bit upsets from the memory are corrected, these results show that the memory is the weakest structure of the SoC. Following that, several upsets were reported from the register file, mostly single-bit upsets, which account for 22.4% of all exceptions and are corrected only in the hardened HARV experiments. Furthermore, the results show that the third most common HFTE is the timeout load access fault, which is detected by the bus master block. This HFTE is reported when the bus master attempts to access a peripheral, but the peripheral does not respond before a specified time limit, likely due to errors either in the bus interconnection structures or in the peripherals' controllers. When this HFTE is triggered, the HARV hardening structures attempt to recover access to the peripherals by resetting the entire bus, including the bus master, interconnects, and peripherals, through a specific reset signal.

Despite the most frequent HFTEs, some appeared only once during the experiment, namely the program counter single-and double-bit upset, instruction register double-bit upset, and load access faults. Out of those, single-bit upsets in the program counter are corrected for the hardened HARV configuration, and double-bit upsets and load access faults are the most critical because they are not correctable.

D. Error Propagation

In order to explore the effectiveness of the HARV-SoC's hardening structures, we analyze the propagation of each HFTE for each HARV-SoC configuration separately, as shown in Table V. This table presents the number of reports for each HFTE type (#HFTE) and the number of those that propagated to cause an execution error and did not finish correctly (#Propagated HFTEs). Although the HFTEs propagated to cause execution errors, these were still identified by the watchdog timer, which is always enabled and recovers the processor operation to continue the experiment. Furthermore, we present the cross section for HFTE propagations, which is normalized by the fluence during the execution of each HARV-SoC configuration.

Most of the propagated HFTEs were timeout load access faults, which were most likely non-recognized faults in unprotected structures that led to the peripherals being unresponsive. At times, the faulty state due to these HFTEs was actually recovered by the peripheral reset mechanism, which we left intentionally enabled for baseline configuration to improve its error reporting, which would simply leave the processor in a stuck state. Although these HFTEs did not always recover the processor operation, the application is still provided with valuable information that aids in recovering the processor operation through software strategies, such as rollbacks, task restarting, and soft resets.

The HFTEs from the register file were mostly single-bit upsets, which were corrected in the hardened HARV-SoC and ignored for the baseline. Out of those, only one propagated to cause an execution error, which in turn propagated to cause a timeout load access fault.

Fig. 1 .

 1 Fig.1. HARV-SoC architecture[START_REF] Santos | Enhancing fault awareness and reliability of a fault-tolerant RISC-V system-on-chip[END_REF].

Fig. 2 .

 2 Fig. 2. Accumulated number of HFTEs vs TID of each board.

Fig. 3 .

 3 Fig. 3. Accumulated number of CRC errors per TID.

TABLE I ACCUMULATED

 I FLUENCE AND DOSE UNTIL UNRECOVERABLE DEVICE FAILURE PER SUT. , we did not identify a clear and consistent distinction between both benchmarks. Hence, for classifying and analyzing the propagation of the reported errors, we combined the results of both benchmarks. Some fluctuations in the number of errors per SUT were observed, mainly concerning SUT #3 and SUT #4, as expected of a stochastic phenomenon since SUTs had the same experimental conditions.

	SUT	TID [krad] Fluence [p/cm 2 ]	Fluence [p/cm 2 ] (TID < 25 krad)
	SUT #1	38.44	5.94 × 10 11	3.87 × 10 11
	SUT #2	35.52	6.17 × 10 11	4.04 × 10 11
	SUT #3	37.20	4.97 × 10 11	3.38 × 10 11
	SUT #4	39.96	5.89 × 10 11	4.03 × 10 11
	SUT #5	39.50	5.66 × 10 11	4.05 × 10 11
	SUT #6	37.19	5.92 × 10 11	4.05 × 10 11

table

TABLE III ERROR DISTRIBUTION FOR TID LESS THAN 25 krad Benchmark SUT # Errors 1 Error XS 1 [cm 2 /device]

 III

		SUT #1	13	3.36 × 10 -11
	Coremark	SUT #2	15	3.71 × 10 -11
		SUT #3	7	2.07 × 10 -11
		SUT #4	7	1.74 × 10 -11
	Embench	SUT #5	16	3.95 × 10 -11
		SUT #6	18	4.44 × 10 -11
	1 Analysis considering before device failure at 25 krad.

TABLE II CROSS

 II SECTION OF DEVICE FAILURES PER SUT.

	SUT	# Device failures 1	Failure XS 1 [cm 2 /device]
	SUT #1	11	1.85 × 10 -11
	SUT #2	11	1.78 × 10 -11
	SUT #3	8	1.61 × 10 -11
	SUT #4	12	2.04 × 10 -11
	SUT #5	11	1.94 × 10 -11
	SUT #6	5	8.44 × 10 -12
	1 Analysis considering accumulated fluence until unrecoverable failure.

TABLE V ERROR

 V PROPAGATION PER HFTES FOR EACH HARV-SOC CONFIGURATION.

VI. CONCLUSION

This work presented the effects of high-energy proton irradiation in a fault-tolerant RISC-V SoC, reporting a detailed error analysis and providing insightful discussions on the system's reliability. We discussed error propagation on the SoC, compared the baseline and hardened implementations, and identified the impact of accumulative effects on the SEE analysis and the device failure. The fault tolerance techniques applied to the SoC showed significant improvement in reliability for high-energy proton irradiation. Moreover, the available reporting capabilities of HARV-SoC greatly supported our analysis. Finally, the proton characterization extends the evaluated environments of HARV-SoC, enabling the proposition of applications targeting space systems. In future work, we intend to further explore the impact of using different benchmarks as test stimuli and improve the protection for the application code.

This study has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement no. 101008126, the Region d'Occitanie and the École Doctorale I2S from the University of Montpellier (contract no. 20007368/ALDOCT-000932), the Foundation for Support of Research and Innovation, Santa Catarina (FAPESC-2021TR001907), and the Brazilian National Council for Scientific and Technological Development (350794/2023-5).

Configuration HFTE Type

#HFTE 1 #Propagated HFTEs Considering the HFTEs from the memory, all single-bit upsets were mitigated in both HARV-SoC configurations. The double-bit upsets, on the other hand, were not corrected, but they affected data that was not critical for the benchmark execution, which finished correctly. Similar behavior was observed for the least common HFTEs in the hardened configuration, in which the affected data was not critical for the benchmark execution.

As a result of the error propagation, the hardened HARV-SoC improved the reliability by reducing the number of propagations by approximately 16% based on the HFTE propagation cross section.

V. DISCUSSION

In Subsection IV-A, we investigated device failures to enable proper SEE analysis since unrecoverable failures were observed at the end of the test campaign. Notably, we provided a hypothesis for the observed behavior, indicating the application code, stored in internal flash memory, as the root cause. The failure affected the read operation performed to fetch instructions for the processor in an intermittent manner since in some cases hard resets temporally restored the functionality. Despite that, this behavior was consistently observed for the SUTs after a certain dose, as shown in Fig. 3, leading to unpredictable execution as supported by Fig. 2. We suspect that the failure source is related to the CMOS logic, known to be sensitive to TID [START_REF] Sajid | Analysis of total ionizing dose effects for highly scaled CMOS devices in low earth orbit[END_REF], in the path to access the memory cells and not the flash cells themselves, which are more robust to this effect, as shown by the manufacturer [START_REF]Single Event Effects -A Comparison of Configuration Upsets and Data Upsets, WP0203 White Paper[END_REF], [START_REF]IGLOO2 and SmartFusion2 65nm Commercial Flash FPGAs[END_REF]. To access this memory block, HARV uses an unprotected interface implemented in the device, leaving a margin for failures. Moreover, the controlling logic of the memory is not hardened by design, in which buffers, encoders/decoders, multiplexers, and many other structures are susceptible to dose effects.

We explored the execution of different benchmarks to extend the test stimuli and improve the characterization. Despite not finding a consistent distinction between both benchmarks, as discussed in Subsection IV-B, a difference is expected for distinct workloads. In this case, both CoreMark and Embench have a broad set of algorithms with different processor and memory utilization profiles, minimizing variations.

Finally, the error classification and propagation analysis provide insights into the improvements achieved with the fault tolerance techniques implemented in HARV-SoC. In Table V, we can observe that the hardened configuration is highly capable of avoiding the propagation of errors since the hardened version is able to reduce the error propagation by 16%.