Hung Quoc

Lu

Soheib Fergani
email: sfergani@laas.fr

Carine Jauberthie

A new scheme for fault detection based on Optimal Upper Bounded Interval Kalman Filter

Keywords:

. Application on a vehicle bicycle model highlights the efficiency of the proposed approach. Comparisons with other efficient estimation and fault detection strategies are provided to discuss the accuracy of the obtained results.

INTRODUCTION

Fault detection and diagnosis are important for all system engineering problems. Several methods for fault detection in dynamic systems are mentioned in [START_REF] Willsky | A survey of design methods for failure detection in dynamic systems[END_REF], including the innovation-based method in which a χ 2statistic hypothesis testing was used. This last method is applied appropriately with the standard Kalman filter [START_REF] Kalman | A new approach to linear filtering and prediction problems[END_REF]) to process the linear dynamic system with deterministic coefficient matrices. In [START_REF] Sainz | Fault detection and isolation of the three-tank system using the modal interval analysis[END_REF], for example, an approach to generate envelopes based on interval techniques of the modal interval analysis is proposed. In [START_REF] Puig | Fault diagnosis and fault tolerant control using set-membership approaches: Application to real case studies[END_REF], set-membership methods in fault diagnosis (FD) and fault tolerant control (FTC) are reviewed, these methods aim at checking the consistency between observed and predicted behaviour by using simple sets to approximate the exact set of possible behaviour.

In [START_REF] Tran | Cadre unifié pour la modélisation des incertitudes statistiques et bornées -Application à la détection et isolation de défauts dans les systèmes dynamiques incertains par estimation[END_REF], a fault detection approach is presented and this approach combines a χ 2 -statistics hypothesis test with the Upper Bound Interval Kalman Filter (UBIKF). An upper bound for all positive semi-definite matrices belonging to an interval matrix is proposed. This upper bound aims to overcome the singularity of the inverse of interval matrices. Based on the concept of UBIKF, a filter using optimal upper bounds for this class of problems, named OUBIKF, is introduced in [START_REF] Lu | Optimally bounded interval kalman filter[END_REF]. The contribution of the later consists in a rigorous mathematical framework for an optimal upper bound (in the sense of operator norm minimization) providing tight estimate intervals containing all actual states of the dynamic system.

In this paper, an adaptive hypothesis test method is developed to detect sensor faults applied to a linear discrete time dynamic system with assumptions requiring the use of interval computations. The proposed scheme for fault detection combines the developed hypothesis test (based on a χ 2 -statistic with degrees of freedom (d.f.) adaptively chosen thanks to amplifier coefficients) and the OUBIKF.

The paper is organized as follows: Section 2 provides some preliminaries including principles and OUBIKF algorithm. In Section 3, the proposed scheme for fault detection is developed and some evaluation indicators are given. An application in simulation of the proposed approach is presented in Section 4. The application model comes from automotive domain. A comparison with other method (see [START_REF] Raka | Fault detection based on robust adaptive thresholds: A dynamic interval approach[END_REF]) is given. Conclusions and some ideas for future works are provided in Section 5.

PRELIMINARY

Optimal upper bound of all positive semi-definite matrices belonging to an interval matrix

A real positive semi-definite matrix M is denoted by M 0. Denote also S(n) = M ∈ R n×n : M = M T and S + (n) = {M ∈ S(n) : M 0}.
Let M, N be two real squared matrices of the same size. An order between M and N denoted by N M is defined if and only if M -N 0. M is called an upper bound of N , or says N is dominated by M or M dominates N . For Hermitian matrices, this order is known as the Loewner (partial) order [START_REF] Pukelsheim | Optimal design of Experiments[END_REF]; [START_REF] Zhan | Matrix Inequalities[END_REF]). This partial order is extended to the notion of bounds for a non empty set Ω of real squared matrices as follow. If a real squared matrix K satisfying M K, ∀M ∈ Ω then K is an upper bound of Ω, denoted Ω K. If K and L are two upper bounds of Ω, then K is said better than L if K ≤ L depending on the choice of matrix norm . .

th column). Write M ∈ [M ] to indicate a punctual matrix M belonging to [M ], i.e. ∀i, j, M ij ∈ [M ] ij . Let [M ] be an n × n real interval matrix. Denote: a) S([M ]) = M ∈ [M ] : M = M T , b) S + ([M ]) = {M ∈ S([M ]) : M 0}, c) BS + ([M ]) = {K ∈ S + (n) : S + ([M ]) K}.
Other standard notations and computations related to interval analysis are referred to [START_REF] Jaulin | Applied Interval Analysis, with Examples in Parameter and State Estimation, Robust Control and Robotics[END_REF].

For any n × n matrix A, we use the notations σ i (A), λ i (A) (i = 1, ..., n) to indicate respectively the singular values and eigenvalues of A among which σ max (A) and λ max (A) are the corresponding maximum values. It is referred to [START_REF] Zhan | Matrix Inequalities[END_REF] for vector and matrix norm definitions.

In the following two theorems, let [M ] be an n × n real interval symmetric matrix and assume that S + ([M ]) = ∅. Theorem 1. (Existence of Optimal upper bounds). The following properties hold:

i) α * = sup M ∈S+([M ]) {λ max (M )} < ∞ and S + ([M ]) αI if and only if α ≥ α * . ii) α * I is the optimal upper bound of S + ([M ]) in the set BS + ([M ]
) in the sense of operator norm minimization.

iii

) Let Ω = K ∈ BS + ([M ]) : n -1 n i=1 λ i (K) ≥ α * . α * I is the optimal upper bound of S + ([M ]) in Ω in the sense of nuclear norm minimization.
α * is said the optimal value of BS + ([M ]). Theorem 2. (Bounds of Optimal value α * ). The following properties hold:

i) E = {M ∈ S + ([M ]) : diag(M ) = diag(sup([M ]))} is
the non empty set of maximal elements of S + ([M ]). ii) There exists a matrix

N * ∈ E s.t. λ max (N * ) = α * . iii) Let Max([M ]) = (Max ij ) be a matrix determined by Max ij = sup([M ]) ij , if mid([M ]) ij ≥ 0 inf([M ]) ij , otherwise . (1) then α * ≤ sup M ∈[M ] { M F } ≤ Max([M ]) F . If Max([M ]) 0 then λ max (Max([M ])) ≤ α * ≤ Max([M ]) F . Remark 1. α * I is the simplest upper bound of S + ([M ]
) to use both in practice and theory. Its existence and optimality are stated in Theorem 1. Theorem 2 provides a simple way to localize the optimal value α * (via Max([M ])).

More details and proofs can be found in [START_REF] Lu | Optimally bounded interval kalman filter[END_REF].

2.2 Mixed uncertainty linear discrete-time dynamic system for state estimation and fault detection Consider the following linear discrete time dynamic system

x k = A k x k-1 + B k u k + w k , y k = C k x k + D k u k + v k , k ∈ N * , (2)
] ([r k ] = y k -[ŷ k ], [ŷ k ] = [C][x k|k-1 ] + [D]u k )
where [x k|k-1 ] is the predicted state estimate and [ŷ k ] the measure estimate.

Algorithm 1 OUBIKF ALGORITHM Initialization:

[x 0|0 ], P 0|0 , [A], [B], [C], [D], [Q], [R], p, u k , y k , k = 1, 2, ..., N . Find n 0 the number of non zero radius of [C] Find γ s.t. S + ([R]) γI for k = 1, 2, 3, ...N do Prediction step: [x k|k-1 ] = [A][x k-1|k-1 ] + [B]u k [P k|k-1 ] = [A]P k-1|k-1 [A] T + [Q] Find α k s.t. S + ([P k|k-1 ]) α k I Correction step: β k = 1/(2α p k ) ; σ k = β k /n 0 V k = mid([C])mid([C]) T + γ k α k (1+n0/β k ) I + β k +n0σ k 1+n0/β k diag{rad([C])rad([C]) T } K k = mid([C])V -1 k [x k|k ] = (I -K k [C])[x k|k-1 ] + K k (y k -D k u k ) P k|k = (I -K k mid([C]))α k (1 + n 0 /β k ) end for where S + ([P k|k ]) P k|k , K k = argmin{tr(P k|k )}.
In this paper, fault detection approach is based on the one hand Algorithm 1 and on the other hand a statistical hypothesis testing. Therefore, it is vital to investigate carefully the stochastic properties of related terms of the system (2), otherwise the fault detection test for the interval case cannot be derived. In this study, it can be proved that state x k , measure output y k , estimator xk|k , estimation error k and residual r k are all Gaussian vectors, in which

r k ∼ N (0, S k ), r k is a function of {x 0 , w 1:k , v 1:k } and S k
is the covariance matrix of estimation error.

Key property (K):

Assuming S k is non singular and let η k = S -1/2 k r k = (η k,1 , ..., η k,ny ). Then η k ∼ N (0, I), that is η k,i 's are N (0, 1)-distributed and independent each other. Furthermore, being a transformation of r k , η k is then a function of {x 0 , w 1:k , v 1:k }, write η k = η k (x 0 , w 1:k , v 1:k ).
Therefore, for any 1 ≤ t ≤ s, η t = η t (x 0 , w 1:t , v 1:t ) and η s = η s (x 0 , w 1:t , w t+1:s , v 1:t , v t+1:s ) are not independent. The same holds for r t and r s .

Sensor fault detection using the χ 2 -statistics in the literature

The measure y k affected by an additive fault vector f s k is given by

y k = C k x k + D k u k + v k + f s k ,
where f s k is a sensor fault vector belonging to R ny . The fault vector f s k can be composed by multiple or single faults:

-first type: some (or all) sensors are faulty, -second type: only one sensor is faulty.

In the literature, using the χ 2 -statistics test for sensor fault detection is a kind of Innovation approach mentioned in [START_REF] Mehra | An innovation approach to fault detection and diagnosis in dynamic systems[END_REF]. In [START_REF] Willsky | Adaptive filtering and self-test methods for failure detection and compensation[END_REF] and [START_REF] Willsky | Two self-test methods applied to an inertial system problem[END_REF], this method is applied for fault detection problems in which the following statistic is used

ν k = k i=k-W +1 η T i η i = k i=k-W +1 r T i S -1 i r i , (3)
where W is a window size (W ≤ k). This statistic is used as χ 2 -distributed random variable with W n y degrees of freedom. A rule for the fault detection test was established:

(H 0 ) ν k ≤ δ, no error occurred; (H 1 ) ν k > δ,
an error occurred, where δ is a threshold determined by P(χ 2 (W n y ) > δ) = α with α a chosen significance level (P is a probability measure).

Clearly, any η T k η k follows the distribution χ 2 (n y ) since, by property (K), η T k η k is the sum of n y independent Gaussian components. Also by property (K), η t and η s are not independent for all t = s, so

k i=k-W +1 η T i η i ∼ χ 2 (W n y )
does not hold exactly. Nevertheless, this test worked rather well as an alarm (detection) to faults and was a simple one among others tests. This statistic ν k might be approximated reasonably by a χ 2 (W n y ). We distinguish a statistic T following exactly a distribution F and being approximated by another statistic T with distribution F . Any statistic can be used as estimator for a quantity of interest with different accuracies.

SENSOR FAULT DETECTION BASED ON OUBIKF AND ADAPTIVE DEGREES OF FREEDOM

χ 2 -STATISTICS Considering system (2) and assumptions (H). Measures

y k and interval matrices [A], [B], [C], [D], [Q], [R] are known.
The following terms are obtained by computation:

measure estimate intervals [ŷ k ] = [C][x k|k-1 ] + [D]u k , residual intervals [r k ] = y k -[ŷ k ] and the interval matrix [S k ] = ([C][A])[P k-1|k-1 ]([C][A]) T +[C][Q][C] T +[R]
which contains all residual covariances S k .

Description of the approach

In the literature, to use the χ 2 -statistics test, a standard normal distribution form (

η k = S -1/2 k r k ∼ N (0, I)) is needed.
A similar form but for the interval vector [r k ] is meant to match our goals, and thus the singularity problem of [S k ] is an impediment. To overcome this impact, it is proposed in [START_REF] Tran | Cadre unifié pour la modélisation des incertitudes statistiques et bornées -Application à la détection et isolation de défauts dans les systèmes dynamiques incertains par estimation[END_REF] to use the upper bound of S + ([S k ]) instead of [S k ] and a better choice of this upper bound is applied thanks to properties in Section 2.1.

The following strategy is proposed in the present work:

+ Find Σ k such that S + ([S k ]) Σ k . This upper bound matrix is of the form Σ k = a k I (a k ∈ R * + ) using properties in Section 2.1. + Compute: [η k ] = Σ -1/2 k [r k ] = [r k ]/ √ a k , [ξ k ] = [η k ] T [η k ] = [r k ] T Σ -1 k [r k ] = [r k ] T [r k ]/a k . + Compute abs([ξ k ]), the absolute operator applying for [ξ k ], since ξ k = ηT k ηk ≥ 0, ∀ξ k ∈ [ξ k ] whilst during interval computations, most of the time 0 ∈ [ξ k ]. abs([a , b]) = [min(|a|, |b|) , max(|a|, |b|)] , 0 / ∈ [a , b] [0 , max(|a|, |b|)] , 0 ∈ [a , b]. + Let U k = sup(abs([ξ k ]))
. The statistic U k will be used in hypothesis testing for which it is approximated by a χ 2 (κ k n y ) random variable (some details are given in the next paragraph). κ k is called an adaptive amplifier coefficient.

Some remarks can be made immediately as follows: ∀k ≥ 1,

• 0 ≤ ξ k ≤ η T k η k ∼ χ 2 (n y ) (since S k Σ k for any S k in S + ([S k ])), • 0 ≤ ξ k ≤ U k , • E[χ 2 (n y )] = n y U k most of the time.
It is reasonable to consider ξ k as a χ 2 -distributed random variable with a d.f. smaller than n y , but this statistic is actually unknown. The statistic U k is obtained by computation. Based on previously mentioned remarks, it is proposed to approximate this statistic U k by a χ 2distributed random variable with an adaptive d.f. κ k n y (κ k > 1) where κ k is an adaptive amplifier coefficient (a.a.c.). Thanks to this a.a.c., adaptive thresholds are built and help to detect faults.

The rule for the fault detection test is similar to this one presented previously: (H 0 ) U k ≤ δ k , no error occurred; (H 1 ) U k > δ k , an error occurred. δ k is an adaptive threshold determined by P(χ 2 (κ k n y ) > δ k ) = α with α is a chosen significance level.

The proposed approach is summarized in Algorithm 2.

Algorithm 2 FAULT DETECTION ALGORITHM Initialization:

[x 0|0 ], P 0|0 , [A], [B], [C], [D], [Q], [R], p, α, u k , y k , k = 1, 2, ..., N . for k = 1, 2, 3, ...N do Implementation: Use Algo.1 to get : [x k|k ], P k|k , [x k|k-1 ], [P k|k-1 ]. [r k ] = y k -[C][x k|k-1 ] -[D]u k [S k ] = [C][P k|k-1 ][C] T + [R] Find a k according to theorem 2 s.t.: S + ([S k ]) a k I. U k = sup{abs([r k ] T [r k ]/a k )} κ k = mean{sup([r k ]) -inf([r k ])} Find δ k s.t.: P(χ 2 (κ k n y ) > δ k ) = α Detection signal : π k = I(U k > δ k ) end for
where I(x) equal 1 if x holds true and null otherwise.

After test running, an adjustment procedure is proposed to obtain detection signals more accurately. That is, in a window of size w, if the number of consecutive error occurrences is smaller than w, we consider that these errors (if exist) don't cause serious effects and will be dismissed. Furthermore, since error is often detected with a delay, all detection signals will be shifted to the left w/2 steps ( . is the floor function).

Evaluation indicators

To evaluate the fault detection performance, some indicators are introduced. Assume that system(2) is applied for N iterations among which faults occur in a region R with length l (0 ≤ l ≤ N ). The region R may be a range or union of ranges. For simplicity, hereafter we call R an error range. Knowing that the detection signal has value 1 or 0, we call right detected signal the 1-value detection signal situated inside the error range and false detected signal the 1-value detection signal situated outside the error range. More details on indicators (with slight differences) can be found in Chen and Patton (1999).

Choice of the a.a.c. κ k

A χ 2 distributed random variable has the cumulative distribution function with d.f. k:

F (x, k) = P(χ 2 k ≤ x) = x/2 0 t k 2 -1 e -t dt ∞ 0 t k 2 -1 e -t dt . (4)
In the literature, k is a positive integer. However, from the analysis point of view, F (x, k) is a continuous function of k (k > 0) at any positive value of x (since the Gamma function Γ(z) = ∞ 0 t z-1 e -t dt is continuous for all z > 0). Consequently a positive real d.f. κ k n y is used.

For an accurate choice of a.a.c. κ k , some conditions are required:

• It must be sensitive to the fault occurred.

• It must be large enough to get a small FAR (e.g. 5%) in the fault free case. • Being a distribution parameter of statistic U k , it is highly recommended that the chosen κ k is related to the U k 's construction.

Concretely, by writing residual intervals in the form [r

k ] = mid([r k ]) + [-1 2 , 1 2 ] * width([r k ]), the statistic U k is ex- pressed as U k = width([r k ]) + 2 * abs(mid([r k ])) 2 /(4a k ) ,
(5) a function of mid([r k ]) and width([r k ]) where the later is more sensitive to the fault than the former. The residual width is a major factor influencing the U k 's computation and, furthermore, reflects the performance of the model and algorithm. Consequently it is reasonable to chose κ k as a function of residual width.

Which function of residual width will be chosen is a hard problem due to many impacts, for instance:

• no further information about width([r k ]) and mid([r k ]) is available,

• κ k and corresponding threshold δ k are both unknown; it exists only a relation represented via the quite complex function (4) so that F (δ k , κ k n y ) ≥ 1 -α, • the yielded δ k (by κ k ) must satisfy the fault detection constraint: δ k ≥ U k when no error occurs and δ k < U k otherwise.

Therefore, an additional requirement for the chosen κ k is that it must (while being sufficiently large in the fault free case as aforementioned) not increase as fast as U k when an error occurs and affects on the width([r k ]).

Combining all constraints and noticing that, in general, identifying analytically degrees of freedom for a test problem is always not evident, the following a.a.c. is proposed:

κ k = ny i=1 (sup([r k ]) i -inf([r k ]) i ) /n y . ( 6
)
Simulation results in section 4.2 favored this choice by showing that it provides a small FAR and also satisfies all other requirements aforementioned.

Being not unique, the a.a.c. can be chosen differently, e.g. by a scale of ( 6) so that it becomes κk = λ k κ k , where λ k > 0 is a scale parameter. This choice would be a scope in our perspective research.

APPLICATION

In this section, the strategy developed previously is applied in simulation to a model taken from automotive domain [START_REF] Fergani | Robust multivariable control for vehicle dynamics[END_REF]). This model is a nonlinear continuoustime model which has been discretized/linearized and thus given under the form (2). The results are compared with those obtained by the method proposed in [START_REF] Raka | Fault detection based on robust adaptive thresholds: A dynamic interval approach[END_REF] (called method B in the next).

Simulation procedure

A discretization phase with a sampling time T = 0.05s is applied to the considered continuous model to get matrices

A d , B d , C d , D d (
0 = (0, 0) T . At each step k, generate A k , B k , C k , D k , Q k , R k according to uniform distribution in corresponding interval matrices and so that Q k ∈ S + (n x )
and R k ∈ S + (n y ). Then w k and v k are simulated. Finally, {x k , y k } k∈1:N are computed according to system (2).

Sensor faults are generated in term of bias vector b k ∈ R ny added to y k for all k in a range R with length l. Each sequence of y k 's components, e.g. {y k (i)} k∈1:N for some i = 1, ..., n y , is called a chain. In this simulation, b k = b.1 where 1 is the all-ones vector in R ny .

Apply Algorithm 2 for N steps. The following choices are applied inside the algorithm: starting point [x 0 ] = ([-0.5, 0.5], [-0.5, 0.5]) T , initial error covariance bound

P 0|0 = max{diag(sup([Q]))}I, p = 3, upper bounds ω k I of any set S + ([M ]) identified by ω k = Max([M ]) F .
The window size is fixed at w = 5 (see 3.1).

Using the previously defined sampling period T , the following discrete matrices are obtained:

A d = 0.9126 0.0446 -0.17

Simulation results

The following simulations are to be considered as first results and a preliminary study to highlight the consistency and the efficiency of the proposed method by simulating two scenarios. Other insights are being investigated in the authors ongoing studies. Table 1 shows that DR has ascending trend as well as b increases while FAR is rather stable in [1.0 , 1.5](%). This means that the larger the fault value b, the better the fault detection procedure is performed and, conversely, the b change hardly affects the false alarm rate FAR. This also means that the current choice of a.a. Table 2 shows that the adjustment procedure eliminates almost all FAR indexes (at least 73% comparing to those in Table 1). Additionally, this procedure yields a positive effect with large fault value (b > 15) and a negative effect otherwise for EFF indexes.

Scenario 2: Fix error range R. For each fault value b, do L times variable simulations to get measurements y k . This scenario aims to show the effects of different measurement samples {y k } s (k = 1 : N, s = 1 : L) on the fault detection procedure for a given error range R. Specifically, these effects come from random noises existing inside of y k since the later is a function of {x 0 , w 1 : w k , v k }. used for our method. The setting is that: 1-dimension multiplicative and additive disturbances (q = 1, δ t = d t = sin(2πt)) are used, a bias sensor fault with value b = 20 is added to all chains of measurements y t in a time range [te1, te2]. This error range is determined correspondingly to the discrete error range [300 : 350] used in the simulation of our method. Some remarks can be pointed out:

(i) method B takes the fault detection chain by chain.

(ii) to compare with the method presented in this paper, detection signals of method B in all chains are combined in one, i.e. new detection signal is 1 if there is any detection signal in any chain getting value 1. (iii) Table 5 shows that the proposed method obtains much better indexes vis-a-vis method B in the context of the implementation simulation.

CONCLUSION AND PERSPECTIVE

A new scheme for fault detection combining OUBIKF and an hypothesis test method using χ 2 -statistics with adaptive degrees of freedom is proposed in this paper.

Theoretical framework is developed. The proposed approach is then applied to a bicycle vehicle model and it is compared to two others methods. The results obtained in simulations highlight the potential of this approach. Simulation results show that the current choice of a.a.c. κ k , in (6), is appropriate for fault detection eliminating well false alarms and dismissing almost all non clear signs of error existence (a prudent fault detection). The possibility of adjusting the a.a.c. according to different purposes of fault detection is a scope of future works. Furthermore, the modified EFF index proposed by EFF = c 1 ×DR-c 2 ×FAR with c 1 , c 2 two constants in [0, 1] can be applied to control the importance of the two indexes DR and FAR so that a compromise between them is achieved. This index, in turn, might affect the choice of the a.a.c.

Also, the great flexibility of this method by adjusting tuning factors makes it suitable to multiple applications.

A

  real interval matrix, denoted by [M ], is a matrix with real interval components (of the type [a, b] with a, b ∈ R and a ≤ b). The components of [M ] are denoted [M ] ij (i th row and j

+

  Furthermore, Detection Rate (DR) is determined by the number of right detected signals over the length l of error range. + No Detection Rate (NDR) is determined by NDR = 1 -DR. + False Alarm Rate (FAR) is determined by the number of false detected signals over N -l, the cardinal of the region outside the error range. + The Efficiency (EFF) of the detection is determined by EFF = DR -FAR.

  non interval and independent of time instant k) according to equations in (2). Then, interval matrices [A], [B], [C], [D] are generated as follow: for M ∈ {A d , B d , C d , D d }, let M = mid([M ]) and choose the radii rad([M ]) at random in [0 , max rad] with max rad = 0.5. The covariance matrices [Q] and [R] are generated in the same way, their diagonal elements being intervals of positive real numbers. Inputs u k are simulated according to a dynamic change for N = 864 iterations. The initial state is chosen at x

  General implementation: Let b k = b.1 be identical for all k in an error range R with length l = 50. Let b vary in 0 : 5 : 30. According to each scenario and for each value of b, do L = 100 times of fault detection. Indicators are computed for each of L simulation times and their means are yielded afterward. Remark 2. Let τ = b/Max width where Max width is the maximum width of the diagonal elements of [Q] and [R].This quantity gives an idea of how large is the actual fault value b w.r.t. some known quantity causing the fault and propagating according to the dynamic system, that is the maximum covariance of noises.Scenario 1: Fix variable simulation {y k } k∈1:N . For each fault value b, choose randomly error range R and do L times error generations. This scenario helps us to consider the method performance in term of fault values b and the positions at which errors occur (in R) w.r.t. a given measurement sample {y k } k∈1:N .

Fig. 1 .

 1 Fig. 1. Method B -Residual for the system with disturbances and bias sensor fault b = 20.

  in which the notations are usual for the standard Kalman filter: x k ∈ R nx and y k ∈ R ny represent state variables and measures respectively,u k ∈ R nu inputs, w k ∈ R nx state noises, v k ∈ R ny measurement noises. Matrices A k , B k , C k ,D k are unknown, deterministic and belonging to a given bounded interval matrices [A], [B], [C], [D] respectively. w k , v k are centered Gaussian vectors with covariance matrices Q k and R k belonging respectively to given interval matrices [Q] and [R]. The initial state x 0 is also Gaussian with mean µ 0 and covariance matrix P 0 . In addition, x 0 , {w 1 , ..., w k } and {v 1 , ..., v k } are assumed to be mutually independent.

	Assumptions (H): Toward achieving efficient sensor fault detection, OUBIKF
	has been introduced and Algorithm 1 is the devoted
	algorithm. The filter focuses on state estimation of system
	(2) and can be used to get residual intervals [r k

Table 1 .

 1 Fault detection table for scenario 1 with a fixed data {y k } k=1:N .

	b	τ	DR%	NDR%	FAR%	EFF%
	0	0	1.26	98.74	1.03	0.23
	5	13.8	5.34	94.66	1.04	4.30
	10 27.5	21.12	78.88	1.04	20.08
	15 41.3	67.28	32.72	1.17	66.11
	20 55.0	94.94	5.06	1.26	93.68
	25 68.8	98.64	1.36	1.36	97.28
	30 82.5	99.84	0.16	1.42	98.42

Table 2 .

 2 c. κ k is appropriate for a fault detection eliminating well false alarms and dismissing almost all non clear signs of error existence (a prudent fault detection). For different purposes of fault detection, κ k can be adjusted. Adjusted fault detection table for scenario 1 with a fixed data {y k } k∈1:N .

	Seeing more, EFF represents the effectiveness of the fault
	detection procedure taking into account both DR and
	FAR. It has also ascending trend according to b. Starting
	at b = 15 (≈ 41×Max width) EFF begins to achieve
	remarkable value (66.11%).
	b	τ	DR%	NDR%	FAR%	EFF%
	0	0	0	100	0	0
	5	13.8	3.30	96.70	0.01	3.29
	10 27.5	17.70	82.30	0.02	17.68
	15 41.3	63.54	36.46	0.05	63.49
	20 55.0	96.36	3.64	0.06	96.30
	25 68.8	99.96	0.04	0.15	99.81
	30 82.5	100	0	0.38	99.62

Table 3 .

 3 Fault detection table for scenario 2 with a fixed error range. Fig.1presents the simulation of this method applying to bicycle model with an as similar as possible setting with that

	b	τ	DR%	NDR%	FAR%	EFF%
	0	0	3.34	96.66	1.90	1.44
	5	13.8	3.08	96.92	2.36	0.72
	10 27.5	19.24	80.76	2.41	16.83
	15 41.3	82.48	17.52	2.18	80.30
	20 55.0	94.74	5.26	2.48	92.26
	25 68.8	98.66	1.34	2.28	96.38
	30 82.5	99.88	0.12	2.37	97.51
	In Table 3, DR and EFF indexes are not necessarily in-
	creasing functions w.r.t. b but their main trends are always
	ascending. The FAR index is also stable in [1.9 , 2.5](%). In
	comparison with the one in Table 1 (FAR ∈ [1.0, 1.5](%)),
	we see that FAR is rather greater in scenario 2 than in
	scenario 1. This means that FAR is more affected by
	random noises than by the position of error range. The
	adjustment procedure eliminates more than 18% of FAR
	indexes comparing Table 4 and Table 3. It has also positive
	effect or negative effect for EFF index according to the
	fault value b being greater or smaller than 15.
	Comparison with method B: The method B consists
	in applying interval observer for linear continuous time
	dynamic system with additive and multiplicative distur-
	bances to compute adaptively upper bounds (ub t ) and
	lower bounds (lb

t ) of residuals r t , and the fault detection rule is that a fault is detected if 0 / ∈ [lb t , ub t ].

Table 4 .

 4 Adjusted fault detection table for scenario 2 with a fixed error range.

	b	τ	DR%	NDR%	FAR%	EFF%
	0	0	2.8	97.20	1.44	1.36
	5	13.8	2.16	97.84	1.84	0.32
	10 27.5	14.06	85.94	1.97	12.09
	15 41.3	82.42	17.58	1.62	80.80
	20 55.0	96.96	3.04	1.82	95.14
	25 68.8	99.8	0.20	1.71	98.10
	30 82.5	100	0	1.89	98.11

Table 5 .

 5 Method B of[START_REF] Raka | Fault detection based on robust adaptive thresholds: A dynamic interval approach[END_REF] versus the Proposed FD scheme.

		DR%	NDR%	FAR%	EFF%
	Method B	8.14	91.86	5.64	2.50
	Proposed method	98	2	0.12	97.88


Claire Labit-Bonis
email: claire.labit-bonis@actia.fr

Jérôme Thomas
email: jerome.thomas@actia.fr

Frédéric Lerasle
email: lerasle@laas.fr

Visual and automatic bus passenger counting based on a deep tracking-by-detection system

In this paper, we address the industrial constraints of automatic passenger counting in city buses through a deep architecture able to deal with images taken from low cost 2D cameras placed above the doorstep, from a zenithal point of view. The challenge is then to handle highly variable scenes due to passengers appearance (hair color, hats, height), bus population density at rush hour and changes in scene illumination. The scientific breakthrough related to deep learning applied to computer vision as well as the system embedding requirements for this task motivate us to integrate in this context a lightweight convolutional multiobject tracker which was especially designed for embedded applications and performed well on the MOT Challenge. We here evaluate it in an industrial context on our large scale in-situ dataset, labelled for detection, multi-target tracking and counting, and present a complete and embedded counting system meeting the requirements of our application.

Introduction

In the literature, people counting is divided into two main areas: "region of interest" or ROI counting, and "line of interest" or LOI counting. The former consists in estimating the number of people present in an image at a given time, while the latter considers the individual and temporal counting of people crossing an artificial line in the image. For a public transport operator, counting passengers on buses allows both to establish statistics on bus lines occupancy and to estimate the fraud rate w.r.t. ticketing. To get the accurate and individual count required by these tasks, we place ourselves from the LOI counting perspective.

In order to avoid all the problems of occultation that can occur with ordinary perspective view surveillance systems, the existing precision counting sensors are generally placed from a zenithal point of view i.e. above each of the doors to be monitored. The technologies used are mainly either stereoscopic vision sensors with depth reconstruction from two images taken from different points of view (passive 3D sensors), or infrared sensors that pulse light beams on the scene and calculate the return time of flight of the beam to estimate the depth of the elements in the observed scene (active 3D sensors). Despite the ease of acquisition of RGB images which would make the 2D modality a very attractive alternative as presented by Sun et al. [30] in their study on LOI counting, 2D image sensors are rarely used for passenger counting on the public transport commercial market. However, the technological breakthrough related to deep learning methods and convolutional neural networks (CNNs) applied to computer vision in recent years motivate us to develop a low cost system based on 2D cameras and offering counting performance equivalent to the market.

In addition to counting performance, we must meet industrial requirements such as (i) a reasonable cost for the client, but also (ii) on-board and near real-time processing -the videostream of passengers getting in/out of the bus can be analyzed between two bus stops. From this perspective, the recent and constant evolution of embedded hardware platforms designed for deep learning approaches is a real game changer: we can imagine a whole material architecture suited for this kind of method, able to handle several doorsteps/cameras at the same time, in parallel, as opposed to existing decentralized solutions with independent sensors and processing units for each doorstep to consider -thus multiplying the final costs.

One way to address the 2D visual counting problem is then to perform a frame-by-frame visual analysis of the scene in a multi-object tracking (MOT) framework. The vision community is very active on this problematic, and the detection-based tracking paradigm is widely used to tackle this challenge. A single image counting process based on one-shot detection is not suited for LOI counting because single image detection cannot infer if people are getting Figure 1. Online multi-object tracking-by-detection applied to passenger counting. A detector locates every object in the scene on a frame-by-frame basis, then a multi-object tracker links these detections together in order to reconstruct the distinct passengers trajectories. Finally, a tracklet crossing the doorstep line is counted in/out depending on its direction. in/out of the bus -to this aim, knowing the passenger direction is essential. Moreover, a temporal analysis i.e. targets tracking, is more robust since it can catch up on the single image detector mistakes by filtering false positives and filling false negative gaps over time. On another note, where conventional motion flow analysis methods such as optical flow would have difficulty distinguishing passengers at rush hour when they are all moving together, trackingby-detection allows to precisely detect each target at each instant, and to link these detections through time to reconstruct individual trajectories as it is illustrated by figure 1.

In this context and based on the industrial embedded constraints of our overall counting application, we integrate and evaluate the compact siamese multi-object tracker proposed by Labit-Bonis et al. [START_REF] Labit-Bonis | Compact and discriminative multiobject tracking with siamese cnns[END_REF]. Because their work was specifically motivated by the performance/speed trade-off required by such embedded applications, it is well-suited to our context as they proposed a lightweight architecture, benefiting from CNNs and which performed great in terms of tracking performance and speed on the well-known public benchmark for multiple people tracking, namely the MOT Challenge [START_REF] Leal-Taixé | Towards a benchmark for multi-target tracking[END_REF].

As a recap, our contributions are three-fold, with:

• the integration of a recent, compact and embeddable CNN-based siamese tracker which performed well on the MOT Challenge, into a 2D visual counting system thus benefiting from the recent breakthrough of deep learning for computer vision tasks;

• the description of the large scale bus passenger in situ dataset that we acquired in a city bus under operational circumstances both at off-peak and rush hour, at different times of the day, and that we annotated for detection, tracking and counting;

• a tracking and counting performance evaluation of the overall application on this dataset, along with em-bedded considerations and preliminary results for the whole algorithm porting.

In the following, we propose a review of existing 2D visual methods applied to people counting in section 2. In section 3, we then describe the CNN-based siamese multiobject tracker integrated in the overall system [START_REF] Labit-Bonis | Compact and discriminative multiobject tracking with siamese cnns[END_REF], before delving into the industrial context of passenger counting with the presentation of our dataset in section 4. Finally in section 5, we evaluate our approach both in terms of tracking and counting metrics on this dataset, after what we show preliminary results towards the embedding of the final counting system.

Related work

2D vision and deep learning in LOI counting is rarely used by bus line operators. However, the considerable gains brought by these techniques on the problem of multi-target tracking and object detection show a real interest in evaluating their contribution in the industrial context of passenger counting. Several visual methods exist, though, and have been applied to passenger counting for image preprocessing, person detection or tracking and counting algorithms, but few propose to use deep learning and convolutional neural networks. In this section, we propose a review of the existing 2D vision systems for people counting in the literature, as well as an overview of the online multi-object tracking-by-detection techniques.

Visual passenger counting with 2D images

In the context of city buses. In the operational context of city buses, works are usually carried out on private image databases, in zenithal view. In 2008, Chen et al. [8] performs segmentation and region merging based on motion vectors extracted from the image. In 2012, Chen et al.

[9] opt for circular shape detection using Canny filtering and a Hough transform, as well as colorimetric filtering on hair color. The detected circles are then compared with those of nearby instants to infer the passengers trajectories. In 2016, Perng et al. [START_REF] Perng | The design and implementation of a visionbased people counting system in buses[END_REF] apply background subtraction as well as morphological operators to focus on moving areas of interest, before performing the actual person detection via the correspondence between the region under consideration and an upstream defined head/shoulder geometric profile. In 2017, Liu et al. [START_REF] Liu | Passenger flow estimation based on convolutional neural network in public transportation system[END_REF] use a mixture of Gaussians and background subtraction for target pre-localization, and then apply a CNN for automatic extraction of passenger visual features. They also mitigate target tracker drift by progressively updating a pheromone map capturing the memory of passing trajectories. In 2019, Nakashima et al.

[24] achieve a 93.5% (resp. 36.1%) counting accuracy for people getting on (resp. off) of a bus on their dataset taken from a perspective view, by using a tracking-by-detection approach based on YOLO [START_REF] Redmon | You only look once: Unified, real-time object detection[END_REF] as the detector and DeepSORT [START_REF] Wojke | Simple online and realtime tracking with a deep association metric[END_REF] as the tracking method. They robustify their counting accuracy by adding a Random Forest Regressor dealing with additional data based on GPS and vehicle speed.

Out of bus context. Other works in perspective and/or zenithal view propose for example to manually extract visual features through HOG and LBP [START_REF] Zeng | Robust head-shoulder detection by pcabased multilevel hog-lbp detector for people counting[END_REF]16,[START_REF] Ahmed | A robust algorithm for detecting people in overhead views[END_REF] before passing them to an SVM classifier. In addition, Zeng et al. [START_REF] Zeng | Robust head-shoulder detection by pcabased multilevel hog-lbp detector for people counting[END_REF] then use a particle filter for target tracking, Kocamaz et al.

[16] associate detections with existing trajectories based on the spatial proximity and motion fluxes of the targets, García et al. [START_REF] García | Directional people counter based on head tracking[END_REF] use a Kalman filter to predict the position of the targets prior to the measurements acquisition.

Online multi-object tracking-by-detection

Visual and online MOT considers sequences of successive 2D images. The targets positions are generally represented by a bounding box around them at each image instant, expressed in pixel coordinates e.g. [x, y, width, height] and to which a unique identifier related to the considered target is assigned. The tracking-bydetection paradigm is commonly used in 2D visual tracking and consists in (i) locating the targets present in each image via a dedicated detection algorithm and (ii) linking these inter-image observations by specifically identifying them. The tracking objective is twofold: to ensure the spatio-temporal coherence of the targets' trajectories, and to compensate for the errors of the detection agorithm which can generate false positives (object detected but not present in the image), false negatives (object present in the image but not detected) and imperfect measurements. In this context, detectors and trackers are generally evaluated separately on dedicated benchmarks such as ImageNet [28] and the MOT Challenge [START_REF] Leal-Taixé | Towards a benchmark for multi-target tracking[END_REF]. Detection. For a long time and before the advent of deep learning, object detection was done by means of a window of fixed size browsing the entire image in small regular steps and with overlap, which were each one going through a feature extraction and classification process before the gathering and filtering of every confident window to produce the final bounding box predictions. Since the formalization of the detection task within CNNs and the evolution of graphic cards specifically well designed for this kind of structure, all detectors today replace this slow sliding window process by using "region-based" and "one-shot" convolutional architectures such as Faster R-CNN [27], SSD [START_REF] Liu | Ssd: Single shot multibox detector[END_REF] or YOLO [START_REF] Redmon | You only look once: Unified, real-time object detection[END_REF]. They take an image as input and process it once, in its entirety, to directly produce as network outputs the box coordinates and classification scores for every detected object. SSD and YOLO both stand out from the rest by their architecture compacity and their speed of execution, well suited for our application requirements, while still providing for good detection performance.

Tracking. At each image instant and after the detector is applied, detection outputs are given to the tracking algorithm, which creates new trajectories when objects are detected, predicts the current position of existing tracklets, and associates them with the remaining detections. Various methods exist for predicting, associating, and managing the tracklets status. For prediction, some of them use simple linear velocity models [4], Kalman filters [START_REF] Chen | Real-time multiple people tracking with deeply learned candidate selection and person re-identification[END_REF][START_REF] Wojke | Simple online and realtime tracking with a deep association metric[END_REF] or single object convolutional trackers [START_REF] Chu | Famnet: Joint learning of feature, affinity and multi-dimensional assignment for online multiple object tracking[END_REF][START_REF] Feng | Multi-object tracking with multiple cues and switcher-aware classification[END_REF]. The vast majority of the approaches submitted to the MOT Challenge also use CNNs for appearance description within the association process [START_REF] Chen | Real-time multiple people tracking with deeply learned candidate selection and person re-identification[END_REF][START_REF] Wojke | Simple online and realtime tracking with a deep association metric[END_REF][START_REF] Chu | Famnet: Joint learning of feature, affinity and multi-dimensional assignment for online multiple object tracking[END_REF][START_REF] Feng | Multi-object tracking with multiple cues and switcher-aware classification[END_REF][START_REF] Karthik | Simple unsupervised multi-object tracking[END_REF]35]. However, even for methods using CNNs for position prediction or appearance feature extraction, networks are trained and executed separately for the two different tasks, thus multipliying the execution time of the overall process. To overcome this issue, Labit-Bonis et al. [START_REF] Labit-Bonis | Compact and discriminative multiobject tracking with siamese cnns[END_REF] proposed a tracking method meeting the industrial requirements presented in section 1, by combining these two tasks into a single, unified and compact siamese convolutional architecture for one-shot regression and reidentification, demonstrating great speed and tracking performance on the MOT Challenge w.r.t. the litterature. The tracker formalization will be further summarized in section 3. This paper focuses on the behavior of this tracker within our industrial passenger counting system, coupled with two recent detectors of the literature: SSD, and YOLO. As a baseline to characterize the contribution of the tracking method, we also compare these couplings with Deep-SORT [START_REF] Wojke | Simple online and realtime tracking with a deep association metric[END_REF] in place of the tracking technology brick: this popular 1 method uses a Kalman filter for position prediction and a CNN for appearance feature extraction.

Publicly available datasets

Since 2019, a large public database of bus passenger counting sequences, PCDS (for Passenger Counting DataSet) has been made available to the scientific community by Sun et al. [30]. However to our knowledge, most of the image databases dedicated to tracking or counting concern plunging or frontal views [START_REF] Chan | Counting people with lowlevel features and bayesian regression[END_REF]6,[START_REF] Ma | Counting people crossing a line using integer programming and local features[END_REF][START_REF] Ma | Crossing the line: Crowd counting by integer programming with local features[END_REF] and there is still no public dataset annotated for the three tasks of detection, multi-object tracking and counting within this applicative context in a zenithal view. In section 4, we focus on our large scale in situ dataset, on which we evaluate the tracking system in section 5. [START_REF] Labit-Bonis | Compact and discriminative multiobject tracking with siamese cnns[END_REF]. From two images cropped around targets position at t -1 and t, the tracker both regresses their position offset within the search area, and a reidentification score between the current target appearance and the predicted one, which can be interpreted as the prediction confidence.

Tracker overview

Labit-Bonis et al.'s work start from the two observations mentioned before. First, within the MOT context, many approaches use CNNs to efficiently describe targets, but only a few use them for the position prediction phase, even though CNN-based single object trackers (SOT) are among the best on the VOT Challenge [START_REF] Kristan | The seventh visual object tracking vot2019 challenge results[END_REF]. Second, when trackers do use CNNs both for prediction and re-identification within MOT [START_REF] Chu | Famnet: Joint learning of feature, affinity and multi-dimensional assignment for online multiple object tracking[END_REF][START_REF] Feng | Multi-object tracking with multiple cues and switcher-aware classification[END_REF], they show great tracking performance but their processing still remains cumbersome: different networks are used for both tasks, they are trained independently and executed in sequence.

All-in-one architecture. This single object tracker for position prediction is integrated within the MOT context and enriched with a reidentification purpose in a single, compact and unified architecture by taking advantage of Siamese networks. These structures are multi-input architectures with layers sharing all or part of their weights on each entry. Initially formulated for similarity/dissimilarity feature learning for verification applications [START_REF] Taigman | Deepface: Closing the gap to human-level performance in face verification[END_REF][START_REF] Chen | Automatic head detection for passenger flow analysis in bus surveillance videos[END_REF], they are now widely used by the visual tracking community for target visual motion prediction and have shown great progress on the VOT Challenge in particular.

The Siamese-based tracker proposed in the paper [18] consists of three parts: the joint visual feature extraction, the inter-frame target offset prediction and the prediction similarity computation via the generation of a reidentification descriptor. Figure 2 illustrates the high-level architecture and highlights the joint processing of regression and reidentification in a two-headed system.

From two enlarged t -1 and t image patches cropped around the target position at time t -1, the network first infers the target position offset between the two instants through the SOT branch, and then produces a reidentification score between the current target appearance and the one at the predicted position after retrieving the feature maps 1 ≈ 1000 citations. subsets thanks to a ROI Align layer. Based on this prediction confidence score, they are then able to adopt a strategy which keeps confident targets alive even if no detection has been made or associated. They thus fill the gaps produced by missing detections and show a good robustness to occlusions.

Tracking performance on the MOT Challenge. In 2021, the authors presented state-of-the-art tracking performance of their siamese-based tracker on the MOT Challenge SDP test set, comparatively to the most recent approaches of the literature at the time such as MOTDT [START_REF] Feng | Multi-object tracking with multiple cues and switcher-aware classification[END_REF], FAMNet [START_REF] Chu | Famnet: Joint learning of feature, affinity and multi-dimensional assignment for online multiple object tracking[END_REF], Tracktor++ [START_REF] Bergmann | Tracking without bells and whistles[END_REF] or DASOT [START_REF] Chu | Dasot: A unified framework integrating data association and single object tracking for online multi-object tracking[END_REF]. Beyond tracking performance, they also exhibited a 2 to 17 times speedup over the approaches with the best MOTA.

Based on these promising results both in terms of tracking performance and speed, we want to integrate and evaluate it within the industrial context of passenger counting on our dataset we present hereafter.

Large-scale in-situ passenger dataset

Dataset description. For our evaluations, we recorded a video database by placing a GoPro camera above the central door of a bus in operation. These videos are acquired at different times of the day to capture the variability of the perceived scenes (day/night, bus congestion, shape/appearance of passengers). On these recordings, only some scenes contain people (at the terminus, the bus is waiting for its departure with closed doors, without passengers on board): only these sub-sequences are used for detection and tracking. For the counting evaluation, we then select only the moments between opening and closing of the bus doors at the stops. In addition to this segmentation, all videos go through a calibration phase to linearize the distortion induced by our onboarded fish-eye camera.

Detection, Tracking, and Counting Annotations. We annotate our database for the three evaluation objectives of detection, multi-target tracking, and counting. Detection and tracking are annotated using the VATIC tool [START_REF] Vondrick | Efficiently scaling up crowdsourced video annotation[END_REF]; all the sequences details are given in Table 1, and illustrated by Figure 3. We estimate the annotation time for detection and tracking alone to be about 70 effective hours. Counting is evaluated only on the sequences of opening and closing of doors i.e. 100 sub-sequences accounting for 118 ascents and 239 descents. For each sub-sequence, we store the ground truth in a .txt file indicating the number of ascents and descents.

Dataset splits. For each experiment, the sequences Seq-2, Seq-3 and Seq-4 are used for training, Seq-1 and Seq-5 for validation, and finally Seq-6 for testing. 1. Passenger dataset description: there are several scenarios with more or less congestion (standing and sitting passengers) as well as changing illumination (day, night, with or without artificial lighting). This distribution is chosen in order to have sequences of all types both in training and validation/testing sets i.e. with different illumination conditions and scene clutter.

Evaluations & discussion

Implementation details

The counting prototype in its first version (SSD + Deep-SORT, as described in section 2) is embedded and functional on an NVIDIA Jetson TX2 development kit. At the time of the porting choice, NVIDIA was the only manufacturer on the market and the TX2 had the necessary capabilities for handling three video streams in parallel and running the considered networks.

The system in its latest version i.e. with the YOLO detector and the tracker from [START_REF] Labit-Bonis | Compact and discriminative multiobject tracking with siamese cnns[END_REF], is currently running on a fixed development PC equipped with a Titan Xp. Even if the official branch of YOLO is the one developped by Alexey Bochkovskiy [5], we chose the implementation of "YOLOv5", released in August 2020 by Ultralytics 2 because of its ease of use and portability. Precisely, its porting is in progress and the arrival of potential processing units such as Intel Myriad V2 or Google Coral motivates us to consider a new platform, better adapted than the Jetson TX2 to the industrial constraints stated above. We will further discuss and show preliminary results of the counting system on these platforms at the end of this section.

Tracking performance analysis

Tracking metrics

As it is done in the MOT Challenge, we evaluate the tracker with the CLEARMOT [3] metrics: false positives FP, false 2 Source: https://github.com/ultralytics/yolov5. negatives FN, identity switches IDS, percentage of mostly tracked (MT) and mostly lost (ML) targets, fragmentations (FM) and most importantly the accuracy of the multi-object tracking MOTA which combines these metrics and gives relevant insight into the overall tracking performance.

Tested configurations

In order to quantify the contribution of our method, we compare several variants in Table 2, in an incremental way:

(A) SSD + DeepSORT: as stated in section 2, this coupling is used as a reference;

(B) SSD + [START_REF] Labit-Bonis | Compact and discriminative multiobject tracking with siamese cnns[END_REF]: to illustrate the contribution of the lightweight siamese tracker, the DeepSORT tracker is replaced by [START_REF] Labit-Bonis | Compact and discriminative multiobject tracking with siamese cnns[END_REF];

(C) YOLOv5 + DeepSORT: replacing SSD by YOLOv5 and keeping DeepSORT as a tracker shows the impact of the detection performance;

(D) YOLOv5 + [START_REF] Labit-Bonis | Compact and discriminative multiobject tracking with siamese cnns[END_REF]: finally, we replace the tracker by [START_REF] Labit-Bonis | Compact and discriminative multiobject tracking with siamese cnns[END_REF] again to show its influence.

Results interpretation

Tracker [START_REF] Labit-Bonis | Compact and discriminative multiobject tracking with siamese cnns[END_REF] contribution: (A) vs. (B) Considering exactly the same detections from SSD, the application of the new tracker instead of DeepSORT gains respectively +1.6%, +6.4% and +4.5% MOTA points on the three sequences, the most notable contribution being on the most crowded sequence, Seq-5. For SSD, which generates many false negatives, the trajectory keep-alive strategy presented in section 3 reduces them considerably (-431, -6581 and -937 decrease on FNs compared to SSD + DeepSORT). The visual position prediction strategy also Importance of the detector for our application: (C) vs. (D) The evaluation of the system against DeepSORT with YOLOv5 in detection instead of SSD shows the importance of the detection quality in the context of a zenithal view. Compared to the MOT Challenge for which the authors show the effectiveness of their approach in these complex situations, the zenithal view drastically reduces the number of occlusions: the application of a powerful detector like YOLOv5 mechanically reduces the potential gain brought by the tracker. However, we still observe a reduction of fragmentations, identity switches and a better coverage of the tracked targets.

Counting performance analysis

Counting metrics

As for detection and tracking, an automatic passenger counting system can generate two types of errors within a full counting sequence at a bus stop during the doors opening: false positives (FP) and false negatives (FN). When the automatic count of people is greater (resp. lower) than it should be, it generates FP (resp. FN). Few studies formalize precisely the metric used for passenger counting, but it is nevertheless common to see two types of errors advertised on the industrial market:

• the "compensated" error, which is based on the overall number of people on the bus, and for which the errors can cancel each other out3 ;

• the "uncompensated" counting error where all the errors made on the count (false positives and false negatives) are cumulated in the metric.

A document published in 2018 by the German industry forum of transport companies VDV gives recommendations for the evaluation of automatic passenger counting systems [START_REF] Verkehrsunternehmen | Recommendations for the applications of apcss within public transport and regional rail transport[END_REF].

In particular, it sets the measure of overall error with compensation as the difference between P a the number of people counted automatically by the system and P m the number of passengers counted manually i.e. the ground truth (cf. equation 1). This measurement is applied distinctly to the upward and downward directions; the compensation is then done in the same direction -a false positive makes up for a false negative.

E g(lobal) = |P a -P m | P m (1)
We can formulate this metric in terms of true/false positives and false negatives by the equation 2, considering P a = F P + T P , P m = GT = F N + T P , and therefore :

E g = |(F P + T P ) -(F N + T P )| F N + T P = |F P -F N | F N + T P (2)
We propose to formalize the cumulative error E c without compensation by the equation 3, which accounts for the totality of errors w.r.t. the groundtruth.

E c = F P + F N F N + T P (3)
For the sake of interpretability, we express the performance of the counting system in terms of count rate without/with compensation T c = (1 -E c ) × 100 and T g = (1 -E g ) × 100. 4. Pre-port evaluations of the system. We evaluate SSD + DeepSORT on Jetson TX2, YOLOv5 + [START_REF] Labit-Bonis | Compact and discriminative multiobject tracking with siamese cnns[END_REF] on Titan Xp, as well as the optimization with TensorRT and OpenVINO of YOLOv5 alone on TX2 and on Intel Core i5.

Quantitative results

Table 3 contains the count rates without/with compensation for ups, downs, and for the overall count. In all cases, the final version performs better than the SSD + DeepSORT combination with gains of +5.9%, +1.3% and +2.8% (resp. +9.4%, +0.5% and +3.3%) on the count rates without compensation (resp. with compensation) in ascent, descent and overall.

Qualitative results

A comprehensive analysis of count errors on the SSD + DeepSORT method shows that among the count errors, about : (i) 58% are related to false detections resulting in the creation of false trajectories, (ii) 19% are caused by nondetections and therefore uncounted trajectories, (iii) 23% are due to bad associations from DeepSORT during tracking. The efficiency of the YOLOv5 detector as well as the better tracking performance brought by the siamese one allow to improve the counting performance for such types of errors.

Industrial requirements

The performance of our counting system is comparable to the best sensors on the market claiming a count rate of between 95 and 99% according to the VDV standard: we thus meet the specifications and present an industrially viable solution in terms of counting quality. Moreover, as we will show in the next section, our centralized architecture based on ordinary 2D sensors makes it possible to integrate the counting system while reducing the costs of the hardware targets.

Embedded prototype and preliminary results

In Table 4, we present the on-going evaluations of our system porting. In the first two rows, the complete counting system is evaluated with the detection, tracking and counting bricks, for an input image resolution of 360 × 480. The difference in FPS between SSD + DeepSORT and YOLOv5 + [START_REF] Labit-Bonis | Compact and discriminative multiobject tracking with siamese cnns[END_REF] can be explained by the fact that the Siamese tracker is applied on the patches at time instants t -1 and t, with a factor 2 to widen the search area, multiplying by 4 the amount of pixels to be processed. Even if the network used in [START_REF] Labit-Bonis | Compact and discriminative multiobject tracking with siamese cnns[END_REF] is lighter, it is executed a larger number of times and on larger images. However, in both cases the networks have not been transformed through TensorRT, so in addition to having more powerful platforms than the Jetson TX2, a margin of progress is possible thanks to the optimization phase during the embedding process.

As an example, we evaluate YOLOv5 alone on images whose size is divided by two, both on Jetson TX2 with TensorRT and on Intel Core i5 processor with OpenVINO (OpenVINO allows to optimize the networks for the Myriad V2 but more globally for any Intel processor, Core i5 included). The execution speed observed (respectively 40 FPS for 3 processed video streams and between 50 and 70 FPS for one stream) confirms that the porting of the new system is within our grasp while respecting the constraints of the specifications.

Conclusion

Automatic passenger counting in city buses is usually done thanks to independent processing units using 3D sensors placed above the doors. In this paper, we propose an embedded and centralized counting system using 2D cameras and achieving a count rate comparable to the best sensors on the market. To do so, we take advantage of recent deep learning methods applied to computer vision and integrate a recent, compact and discriminative siamese multiobject tracker used for both position regression and reidentification, in the overall counting framework, enabling us to reconstruct passengers trajectories and count them as they cross the doorstep. We evaluate this method on our large scale bus passengers dataset, which we annotated for detection, tracking and counting, and demonstrate performance meeting the requirements both in terms of counting results and speed.

Figure 2 .

 2 Figure 2. Tracker synoptic[START_REF] Labit-Bonis | Compact and discriminative multiobject tracking with siamese cnns[END_REF]. From two images cropped around targets position at t -1 and t, the tracker both regresses their position offset within the search area, and a reidentification score between the current target appearance and the predicted one, which can be interpreted as the prediction confidence.

Figure 3 .

 3 Figure 3. Illustration of the variability of the scenes in our database in terms of lighting, passenger appearance and environment clutter.

Table 2 .

 2 MOT evaluations of the different trackers on our dataset. (B) and (D) illustrate the interest of using the siamese tracker from[START_REF] Labit-Bonis | Compact and discriminative multiobject tracking with siamese cnns[END_REF], compared to the original configuration (A). (C) and (D) integrates YOLOv5 as a detector in replacement of SSD.

		Method	IDF1 Prcn	Rpl	GT MT ML	FP	FN	IDS	FM	MOTA
		(A) SSD + DeepSORT	77,7	88,6	89,1	45	68.9	8.9	1151	1090	10	69	77,5
	Seq-1	(B) SSD + [18] (C) YOLOv5 + DeepSORT	84,9 78.8	86,7 92.3	93,4 95.8	45 45	80.0 88.9	4.4 4.4	1433 805	659 422	5 15	22 15	79,1 87.6
		(D) YOLOv5 + [18]	79.0	90.3	95.6	45	88.9	4.4	1044	346	11	6	86.0
		(A) SSD + DeepSORT	30,6	92,1	55,6	48	25.0 22.9 3299 30860 233 1237	50,5
	Seq-5	(B) SSD + [18] (C) YOLOv5 + DeepSORT	31,6 61.3	89,3 92.6	65,1 93.0	48 48	47.9 20.8 5432 24279 222 70.8 10.4 5185 4897 123	297 414	56,9 85.3
		(D) YOLOv5 + [18]	65.9	91	95,1	48	85.4	8.3	6514	3394	112	143	85,6
		(A) SSD + DeepSORT	51,7	90,9 55,64	38	36.8 13.2	604	4812	27	175	49,8
	Seq-6	(B) SSD + [18] (C) YOLOv5 + DeepSORT	51,4 77.7	86,8 96.6	64,3 83.4	38 38	60.5 10.5 1061 81.6 7.9 317	3875 1806	25 17	31 53	54,3 80.3
		(D) YOLOv5 + [18]	74.0	94.8	85.3	38	86.8	7.9	507	1598	17	35	80.4
	catches detection artifacts better than DeepSORT's Kalman
	position prediction: trajectories are less fragmented (de-
	crease of -47, -940 and -144 on FMs), and better tracked
	(gain of +11.1%, +22.9% and +23.7% in MT).

Table 3 .

 3 Final comparison of the complete counting system between the SSD + DeepSORT combination and YOLOv5 +[START_REF] Labit-Bonis | Compact and discriminative multiobject tracking with siamese cnns[END_REF].

				Up : GT = 118			Down : GT = 239	Global
	Method	FP FN TP	Tc	Tg	FP FN TP	Tc	Tg	Tc	Tg
	SSD + DeepSORT	5	19	99	79.7% 88.1%	21	12 227 86.2% 96.2%	84.0% 93.6%
	YOLOv5 + [18]	10	7	111 85.6% 97.5% 11	19 220 87.5% 96.7% 86.8% 96.9%
		Detection	Tracking + Counting	Resolution	Training framework	Optim.	Platform	FPS
	SSD + DeepSORT					360 × 480		Tensorflow	-	Jetson TX2	∼20-23
	YOLOv5 + [18]					360 × 480		PyTorch	-	Titan Xp	∼31-37
	YOLOv5			-		192 × 224		PyTorch	TensorRT	Jetson TX2	∼40*
	YOLOv5			-		192 × 224		PyTorch	OpenVINO Intel Core i5 ∼50-70
										*For a 3 images batch.
	Table

A false negative (resp. positive) on the way up can be compensated for by a false negative (resp. positive) on the way down, or in the same direction of travel, a false positive catches up with a false negative.


Douglas A Santos

Lucas M Luza

Maria Kastriotou
email: maria.kastriotou@stfc.ac.uk

Carlo Cazzaniga
email: carlo.cazzaniga@stfc.ac.uk

Cesar A Zeferino
email: zeferino@univali.br

Douglas R Melo

Luigi Dilillo
email: dilillo@lirmm.fr

Characterization of a RISC-V System-on-Chip under Neutron Radiation

Keywords: RISC-V, System-on-Chip, Fault Tolerance, Neutrons, Radiation

Systems for harsh environments often use embedded processors for tasks that require reliability. However, harsh environments cause faulty behavior in electronics, which eventually lead to system failure. Therefore, embedded processors must use techniques to improve their reliability. In this context, this work presents the implementation and characterization of a RISC-V-based system-on-chip. We characterized our implementation by carrying out test campaigns at the ChipIr irradiation facility. This facility provides a beamline for testing electronics against neutrons, mimicking atmospheric-like environments. With this first test campaign, we identified the most critical parts of our system-on-chip and essential tips to improve the test effectiveness. In the second test campaign, we used an improved version of the system setup with higher reliability error observability features. The version embedding all the hardening techniques could correct or mitigate 98.1 % of the detected upsets under irradiation.

I. INTRODUCTION

Embedded systems are often used in harsh environments, in which ionizing particles interact with semiconductors causing faulty behavior. The atmospheric environment, for example, presents a cascade of secondary particles, including neutrons, protons, and electrons, due to the interaction of the atmospheric gases and galactic cosmic rays [START_REF] Cannon | Extreme space weather: impacts on engineered systems and infrastructure[END_REF]. The interaction of these particles with electronics can lead to transient, permanent, or intermittent faults [START_REF] Yang | Fault-tolerance techniques for spacecraft control computers[END_REF]. As a result, electronic systems may suffer malfunctions and catastrophic failures.

Among the effects that can affect the behavior of electronic devices, a major role is expressed by Single-Event Effects (SEEs). These effects arise from ionizing particles penetrating sensitive nodes of electronic systems [START_REF] Petersen | Single event effects in aerospace[END_REF]. SEEs may lead to many events, such as Single Event Transient (SET) and Single Event Upset (SEU). SETs are characterized by transient turbulence in the signals, which can lead to incorrect circuit action. SEUs are similar to SETs, but they affect the memory elements with bit-flips [START_REF] Yang | Fault-tolerance techniques for spacecraft control computers[END_REF]. According to [START_REF] Velazco | How to characterize the problem of SEU in processors representative errors observed on flight[END_REF], SEUs have a very high occurrence probability and must be considered when designing embedded systems.

Several techniques are used to harden systems against SEUs and one of them is the use of redundancy. According to [START_REF] Sorin | Fault tolerant computer architecture[END_REF], there are three classes of redundancy: spatial (also referred to as "physical"), temporal, and information. Spatial redundancy comprises the utilization of more than one instance of the same component. TMR (Triple Modular Redundancy) is an example of spatial redundancy, which consists of instantiating a module three times and using a majority voter to select the most frequent output to mitigate errors. Temporal redundancy is the utilization of the same instance more than once. Information redundancy is the addition of redundant information to the data, such as ECCs (Error-Correcting Codes), that can detect and correct errors that may affect system behavior.

Several processors were designed to target harsh environments, applying one or more techniques to harden the processors. The LEON4-FT [START_REF] Hjorth | GR740: Rad-hard quad-core LEON4FT system-on-chip[END_REF] processor applies ECC in RAM blocks to mitigate SEUs. MIPS Crypto [START_REF] Ustaoglu | Fault tolerant register file design for MIPS AES-crypto microprocessor[END_REF] uses TMR and matrix encoding techniques to protect its buffer units. MIPSFT [START_REF] Didehban | Reducing of soft error effects on a MIPS-based dual-core processor[END_REF] employs Hamming code to protect registers and TMR for providing fault tolerance to the message passing interface.

RISC-V is an emerging processor architecture that was specified by [START_REF] Waterman | The RISC-V Instruction Set Manual, Volume I: User-Level ISA Document Version 20191213[END_REF]. Its design is similar to MIPS, and it also focuses on a simple design to simplify the implementation. Even though several open-source processors use this architecture, there are not many implementations of this architecture for reliable applications. The authors in [START_REF] Wilson | Neutron radiation testing of fault tolerant RISC-V soft processor on Xilinx SRAM-based FPGAs[END_REF], [START_REF] Wilson | Neutron radiation testing of a TMR VexRiscv soft processor on SRAM-based FPGAs[END_REF] adapted existing versions of RISC-V processors using a tool to harden the design at the netlist level. The work of [START_REF] Marcello | Design of a fault tolerant instruction decode stage in RISC-V core against soft and hard errors[END_REF] explored the soft-error hardening of the decoding stage of the processor's pipeline. They applied Hsiao ECC in the register file and TMR in the pipeline registers, decoder, and controller.

One way to evaluate the efficiency of the fault tolerance and characterize processors is through neutron irradiation, which causes SEUs [START_REF] Normand | Correlation of inflight neutron dosimeter and SEU measurements with atmospheric neutron model[END_REF]. The authors of the works [START_REF] Wilson | Neutron radiation testing of fault tolerant RISC-V soft processor on Xilinx SRAM-based FPGAs[END_REF], [START_REF] Wilson | Neutron radiation testing of a TMR VexRiscv soft processor on SRAM-based FPGAs[END_REF] adapted open-source RISC-V SoCs (Systems-on-Chip) and characterized them against neutron radiation. Their hardening process consists of using the BL-TMR tool [START_REF] Pratt | Improving FPGA design robustness with partial TMR[END_REF] to triplicate the design at the netlist level. This tool can identify and triplicate critical nodes in the netlist, improving the system's reliability based on user-defined granularity parameters. Their main focus was on evaluating the operation of the processors when running on an SRAM-based FPGA (Field-Programmable Gate-Array) device. The work of [START_REF] Lindoso | A hybrid fault-tolerant LEON3 soft core processor implemented in low-end SRAM FPGA[END_REF] evaluated the implementation of a hybrid fault-tolerant LEON3 processor in an SRAM-based FPGA against neutron radiation. This processor employs EDAC (Error Detection and Correction) code and scrubbing in the RAM, ROM, and cache memories. They implemented single-event correction with duplication in the register file. For the FPGA, they applied configuration memory scrubbing, which ensures that the programming bitstream is correct. As a result of the processor assessment against the neutron radiation, they reported a 4.13 times reliability improvement when the hardened implementation is used. It is worth noticing that the authors compared the system with hardening only in the memory versus the system with all the fault-tolerant techniques.

In this context, we developed a low-cost RISC-V core with basic hardening features in a previous work [START_REF] Santos | A low-cost fault-tolerant RISC-V processor for space systems[END_REF]. This core was a single-cycle processor implementation that employed Hamming ECC to protect all internal registers and TMR to protect the Arithmetic and Logic Unit (ALU) and the control unit. We evaluated this simple processor's reliability by running simulations with SEU and SET fault injections. As a result, we obtained an average error propagation rate improvement of 50.4 %, considering both SEUs and SETs and the worst-case scenario.

Here, the proposed work presents the assessment of an improved version of the previous implementation. We improved that processor to provide the means to use it as a microcontroller and evaluated its reliability features through two test campaigns in the ChipIr irradiation facility. Furthermore, for the implementation, we employed a flash-based FPGA innerly resilient to SEUs in the configuration memory [START_REF] Dsilva | Neutron SEE testing of the 65nm SmartFusion2 flash-based FPGA[END_REF] in order to focus on the microcontroller reliability features and filter the errors in the FPGA.

The remainder of this article is organized as follows. Section II describes the experimental setup and the irradiation facility. Section III presents the preliminary characterization campaign. Section IV describes the design of the second improved RISC-V SoC implementation. Subsequently, Section V discuss the characterization results. Finally, Section VI gives conclusions and final remarks.

II. EXPERIMENTAL SETUP

The goal of this work is to analyze the behavior of the implemented SoC against neutron radiation. For this purpose, we carried out tests at the Rutherford Appleton Laboratories -UK, more specifically at the ChipIr beamline [START_REF] Cazzaniga | Progress of the scientific commissioning of a fast neutron beamline for chip irradiation[END_REF]. The ChipIr instrument mimics the atmospheric neutron spectrum, which causes SEE on electronic devices. This instrument irradiates the device under test (DUT) with a flux up to 10 9 times greater than the typical natural radiation environment [START_REF] Cazzaniga | First tests of a new facility for device-level, board-level and system-level neutron irradiation of microelectronics[END_REF]. This high flux enables accelerated testing of electronic devices. According to [START_REF] Chiesa | Measurement of the neutron flux at spallation sources using multi-foil activation[END_REF], the average flux provided by this beamline is 5.6×10 6 n/cm 2 /s for energies above 10 MeV.

We used the M2S010 FPGA device from the Smartfusion2 family by Microsemi. This family of devices is designed for reliable applications [START_REF]IGLOO2 and SmartFusion2 Datasheet Version DS0128[END_REF] and is highly immune to SEUs because the FPGA configuration is flash-based, which is resistant to this type of SEE [START_REF]SmartFusion2 SoC FPGA Version PB0115[END_REF]. The chosen FPGA, SoC M2S010-VFG400 FPGA, was embedded in the Trenz SMF2000 board.

While the FPGA configuration memory is flash-based, the Block RAMs (BRAMs) use an SRAM technology, and the Flip-Flops are D-type (DFF). We are using both BRAMS and DFFs in our design, which are susceptible to SEUs, as shown in the device characterization made in [START_REF] Dsilva | Neutron SEE testing of the 65nm SmartFusion2 flash-based FPGA[END_REF].

We carried out two test campaigns at ChipIr. The first test campaign [START_REF] Dilillo | Evaluation of a fault-tolerant RISC-V[END_REF] consisted of a preliminary characterization of the microcontroller to analyze its critical parts, which is described in Section III. Based on this preliminary characterization, we improved the implementation of hardening techniques in the microcontroller and characterized it in a second test campaign, in which we also substantially improved the test setup, as described in Section IV.

III. PRELIMINARY SOC CHARACTERIZATION

We characterized an SoC based on the core developed in the previous work [START_REF] Santos | A low-cost fault-tolerant RISC-V processor for space systems[END_REF]. The core implements the entire integer instruction-set (RV32I), not including system calls and fence instructions. The main contribution of the previous work was the processor's hardening against SEUs by applying Hamming code and TMR techniques at critical components of the internal microarchitecture. We applied Hamming code with single-error correction at the register file and Program Counter (PC); and TMR at the ALU and the control. We focused on a small footprint to implement this core. Thus, we implemented a single-cycle processor microarchitecture that lacks interruptions and Control and Status Registers (CSRs).

We extended the implementation of the core by adding a state machine, changing the core into a multi-cycle processor. The state machine enabled the processor to support SRAM memories since a grant signal now controls the load and store instructions' execution. It also enables an external SoC wrapper to multiplex the signal between the data memory and the AMBA AXI4-lite bus.

We implemented an SoC top module with two separate memories, the instruction and data, which the memory programmer initializes. The memory programmer is also responsible for resetting and starting the execution of the core. After the core starts, the memories are no longer accessible by the memory programmer but only by the core. The accesses from the core to write and read to/from the data memory may have two targets: the data memory or the AXI4-lite bus. A multiplexer is responsible for selecting the access target based on the provided address. During this process, the core is stopped and waits for the grant signal.

We executed the preliminary test in the ChipIr facility using two M2S010 Smartfusion2 boards simultaneously. Both boards were exposed to the same radiation fluence and side-by-side. Fluence is the total number of particles per unit area irradiated in the board, defined as neutrons per square centimeter.

For the processor's execution, we followed some steps in a computer to test the hardened and non-hardened implementations of the microcontroller. The steps consist in (i) programming the bitstream, which alternates between the implementations; (ii) initialization of instruction and data memories; (iii) start the processor and monitor the UART output; (iv) stops the processor when it identifies a timeout (thirty seconds of inactivity) or a hundred lines with errors.

The benchmark chosen for this preliminary test is composed of both the sum of vectors and Fast Fourier Transform (FFT) algorithms. The sum of vectors algorithm consists of generating and summing two 256-element vectors and storing the result in a third 256-element vector. After that, the result vector is printed to the UART. The FFT algorithm generates an input 256-element vector, computes the Fourier transform, and stores the result into the input vector, which is then printed out.

We evaluated the errors in the UART output by comparing it to a golden execution free of errors. We are considering failure as any different value in the output or a timeout in the execution. Regarding the executions, we evaluated only those with at least one perfect complete execution of both algorithms. This filtering guarantees that the failures are arising from the processor's execution and not the initialization of the memory. After applying this filter, the number of executions was reduced from 284 to 251. We classified the executions according to the failure type. For that, we compared the output to a golden execution (with no radiation) and classified the UART output or outcome, as (i) 1and 2-bit upset, when the output value has the respective hamming distance in comparison with the golden value; (ii) hang, which represents the executions that were executing perfectly but did not finish; and (iii) mismatch, which represents outputs with more than two bit upsets and errors in the printed string.

Table I presents the failure classification for the executions in each board and implementation variation. We notice that, for most failure types, we do not have clear different results between hardened and non-hardened implementations. Between Boards 0 and 1, the percentage results for hang, 1-bit upset, and mismatch failures were inconsistent. While on Board 0 these failures had fewer occurrences in the non-hardened implementation, Board 2 had fewer occurrences in the hardened implementation. Since the results look inconclusive at first analysis, we used the data from the device characterization made by the FPGA producer in [START_REF] Dsilva | Neutron SEE testing of the 65nm SmartFusion2 flash-based FPGA[END_REF] to help their interpretation. For this characterization, several Microsemi devices were tested against neutron radiation, in particular the Smartfusion2 family. Among the reported results, the authors obtained an estimated failure rate for the logical resources of the FPGA. Although they did not use the same device that we used, we expect a similar behavior for our FPGA. Therefore, we have compared our results with the characterization of the M2S050 device.

Based on the average fluence that leads to a failure, we estimated that we had, on average, two upsets in the memory and 0.05 in the internal registers. The entire test campaign had 14 upsets in internal registers and 504 upsets in the memories. Therefore, the sources of failure in our SoC should be mainly located in the instruction and data memories since the fluence of our experiment was not large enough to cause a statistically significant number of upsets in the internal registers.

At the end of the preliminary test campaign, we could also express several conclusions besides the characterization results. The lack of observation techniques implemented in this first architecture restrained a detailed analysis of the system's fault source and propagation. Besides that, considering the type of beam and fluence, the number of DUTs could be increased to achieve better statistics concerning the occurred faults, especially when considering the processor itself. Furthermore, the choice of the benchmark algorithms (sum of vectors and FFT) could also be improved to stimulate more widely and effectively all the system's structure. Finally, and most importantly, we concluded that greater attention must be brought to the system memory since it represents the weakest part of the SoC. In particular, the instruction memory is the most critical one since, besides the sensitivity of the data itself, this part of the memory is not refreshed with new data during operation like data memory, thus enabling error accumulation.

IV. PROPOSED RISC-V SYSTEM-ON-CHIP

For the improved RISC-V SoC implementation, we started with the implementation presented in Section III and took into account all the lessons learned in the preliminary characterization campaign, as detailed above. Since the weakest part of the SoC was identified to be the instruction memory, we placed the instructions in a flash-based memory, which is more resilient to SEUs than the BRAM (SRAM-based). We still used BRAM to implement the data memory, but we hardened it by applying Hamming code with single error correction and double error detection (SECDED). We also learned that we need observation points to analyze the results properly. Hence, we implemented error counters in the core architecture, which can count the number of 1-and 2-bit upsets and are accessible for debugging. In order to improve the test of the processor, we employed the Coremark benchmark, which uses more functionalities of the processor's architecture. Also, we improved the statistics of the results by increasing the number of DUTs to four. Fig. 1 presents the final version of the SoC.

We used the device's eNVM (Embedded Non-Volatile Memory) as our on-chip flash memory for the application data. It is accessed through the AMBA bus and the APB3 bridge. We had to add this bridge because the flash memory is not directly accessible to the FPGA but only through the device's Microcontroller Subsystem (MSS). The eNVM stores the entire application code, so at the beginning of the execution, the processor loads all the data sections from the flash into the data memory. In order to ensure that the loading was performed correctly, we also calculate the CRC-32 (Cyclic Redundancy Check-32) and check with the result value.

The implementation of the data memory SECDED consists of increasing the data width from 32 to 39 bits, which corresponds to the required ECC data. We implemented a Hamming encoder to compute the ECC and a Hamming decoder for the error-correcting and detecting capabilities.

Regarding the hardening of the core registers, we extended the implementation of its Hamming decoders to support the double error detection, besides the single error correction.

From the RISC-V privileged specification CSRs [START_REF] Waterman | The RISC-V Instruction Set Manual, Volume II: Privileged Architecture, Document Version 20190608-Priv-MSU-Ratified[END_REF], we implemented the 64-bit cycle counter, registers for user traps, and machine information. Besides the specification CSRs, we also implemented custom registers to configure the core's hardening capabilities and provide event upsets counting, and the SoC reset cause.

The hardening configuration CSR activates flags to the hamming decoders and the TMR voters to indicate if the errors should be corrected. This register improves the test campaign since the FPGA does not need reconfiguration to test different hardening implementations. The hardening configuration works by activating an error correction signal for the Hamming decoders and TMR voters. Therefore, at the processor initialization, the software configures this register according to the desired hardening capabilities.

Regarding observability, we implemented CSRs for error counting. Two counters were implemented for each hamming decoder plus one for each TMR voter. In total, we implemented ten error counters: for the Hamming decoders of the PC, data memory, and both read ports of the register file, and for the control and ALU TMR voters. The error counters increment each single-or double-bit flip detected from the hamming decoders and for each error mitigated by the TMR voters.

At the SoC level, we also added a Watchdog Timer (WDT). This WDT understands when the processor stops responding and resets the entire SoC. This peripheral is connected to the AMBA bus, and it is enabled by default. If the timer value reaches zero, the watchdog reset triggers for the entire SoC. Since we did not connect the error counters to the reset and implemented the reset cause CSR, every time we identify a watchdog reset, the processor RISC-V Core

A. Implementation in the FPGA

The application of fault tolerance at the components increases the number of logical resources required. Table II presents the results obtained from the Libero v2021.1 tool with the SoC running at 50 MHz and targeting the FPGA M2S010-VFG400. We noticed that the hardening application in the processor increased the usage of logic elements by 50.7 % in comparison to the implementation without hardening. This increase is due to the triplicating of the ALU and control modules and the SECDED application at internal registers. The logic elements report combines a 4-input Look-Up Table (LUT-4) and a DFF since each logic element comprises a LUT-4 and a flip-flop. When we apply SECDED in the data memory, we notice an increase in the usage of RAM18K blocks, which is due to the 7-bit increase in the data width. Besides the increase in memory blocks, we noticed an increase of 22.8 % in logic elements when we harden only the memory; this results from the implementation of the hamming encoder and decoder. Implementation Proc+Mem resulted in an increase of 75 % in logic elements usage compared to implementation None.

Besides the logic elements overhead, the SoC suffers a decrease in the maximum operating frequency when hardening techniques are applied. The estimated maximum operating frequency drops from 33.71 MHz to 25.61 MHz. The reason is mainly that Hamming encoders and decoders are in the system's critical path. However, since the critical path is never fully activated, the SoC can operate at a frequency higher than estimated. Our current SoC is running with a clock of 50 MHz.

B. Characterization experiments

We characterized the RISC-V SoC implementation by running the Coremark benchmark simultaneously in four Smartfusion2 boards; two of the boards are the same used in the preliminary test campaign. All four boards were positioned close to each other and exposed to the same radiation fluence.

The Coremark is a benchmark developed by the Embedded Microprocessor Benchmark Consortium (EMBC), and its goal is to test the performance of microcontrollers. According to the Coremark white paper [START_REF] Gal-On | Exploring coremark a benchmark maximizing simplicity and efficacy, Embedded Microprocessor Benchmark Consortium[END_REF], the benchmark is composed of the operations: (i) list processing, which performs reversing, searching, or sorting operations; (ii) matrices processing, executing a multiplication with a constant, a vector, or another matrix; and (iii) state machine processing, which tests an input string to detect if it is a number. The verification of the correct Coremark operation is done by using CRC-16 on the elements of the processed list.

For our test campaign, we defined the Coremark parameters to improve our analysis. We run the benchmark in a single thread since our algorithm runs as a bare-metal code and does not support context changing. We set the number of Coremark iterations to 200, which takes about 12 minutes to execute. Besides that, we also defined the debug flag of the Coremark, which enables debug printing of the Coremark operations.

At the beginning of each execution during the test campaign, the software sets the hardening CSR according to the previously defined tests. The tests are, in order: (i) hardening in the processor and the memory; (ii) hardening only in the memory; (iii) no hardening; and (iv ) hardening only in the processor. In each Coremark iteration, we set 10 seconds to the watchdog timer and print the number of errors from the counters. At the end of each execution, a WDT reset is forced by the software, and each execution starts from a reset state.

V. CHARACTERIZATION RESULTS

In the second test campaign, we tested the proposed RISC-V SoC implementation against neutron radiation. In total, the devices were irradiated with a cumulative fluence of 45.9×10 10 n/cm 2 , distributed in the 221 Coremark's executions and all hardening configurations.

In Table III, the benchmark executions are classified according to the UART output. The match classification corresponds to the executions that finished perfectly (i.e., all the errors were mitigated or did not affect the execution). The mismatch classification comprises executions that printed a different value than expected by the golden run. Moreover, the timeout classification corresponds to the executions that did not print lines with errors and suffered a watchdog reset before the execution finished. The columns correspond to the hardened component of the SoC. The columns are: (i) None, when no hardening were configured; (ii) Processor, in which hardening configured only for the processor; (iii) Memory, with hardening configured only for the data memory; and (iv ) Processor+Memory, when all hardening configurations were enabled.

We noticed that the hardening application in the processor improved the number of match executions by 5.31 times compared to the none configuration. This improvement increases to 33.48 times when the hardening in the memory is configured. The improvement obtained from hardening only the memory to hardening both the processor and the memory was small (1.34 %) because most of the errors come from upsets in the data memory. It is worth noting that the only configuration that had timeout failures was the one without hardening.

Regardless of the errors in the UART output, the Coremark operation may still run correctly. The errors in the UART could be due to several reasons, such as an error in the transmission, the AMBA bus, and the UART peripheral. Therefore, Table IV presents the classification of the executions based on the Coremark error counter. The correct classification means that the Coremark did not detect errors in the execution, while the error classification is the

Fig. 1 .

 1 Fig. 1. RISC-V System-on-Chip

This work was financed by CNPq -the Brazilian National Council for Scientific and Technological Development -Processes 315287/2018-7 and 436982/2018-8, the Region d'Occitanie -Contract 20007368/ALDOCT-000932, and the École Doctorale I2S de l'Universit é de Montpellier.

contrary. The timeout classification is for the executions that failed before reaching the end of the execution. We noticed an improvement of 5.4 % between the none and processor hardening configurations.

On the other hand, when we compare the None configuration to the memory hardening, it improves 34.4 %. The only execution with memory hardening that failed was due to a non-corrected upset in the registers, which affected the processor's control and led to a timeout failure. Considering the Coremark result, the configuration processor+memory never failed. Table V presents the detected number of upsets during the test campaign for the characterization of the RISC-V SoC and the percentage of them that were corrected. Since most of the upsets are in the memory, the processor-only hardening corrects solely 2.04 % of all the upsets, which correspond to the number of upsets in the processor's internal registers. On the other hand, when we apply hardening in the memory, we increase the percentage of corrected upsets to 95.28 %. When we hardened both the memory and the processor, we noticed an increase of 2.82% in the percentage of corrected errors. It is worth noting that the latest configuration cannot correct all the errors because, in our implementation, the double bit upsets are not correctable. Regarding the integrity checks, the FPGA bitstream configuration never suffered upsets. Furthermore, the CRC-32 check performed in the application's initialization never detected errors, regardless of the hardening configuration.

VI. CONCLUSION

In this work, we started from the baseline core from previous work and extended it to an SoC to support more complex applications. Subsequently, we performed a preliminary test campaign in order to characterize it in an atmospheric-like neutron environment. From this first test campaign, we observed that the lack of error observation techniques restrained a detailed analysis of the system's fault source and propagation. Besides that, we concluded that the number of DUTs should be increased to have more statistics. Another issue of this preliminary characterization was the lack of a proper benchmark algorithm to test the functionality of the SoC. Finally, the primary source of the errors was identified in the memory.

Afterward, we used these preliminary characterization results to develop an improved version of the SoC. In this second RISC-V SoC, we hardened the memories by placing the instructions in a flash memory, which has high inner resilience to SEUs and applying SECDED in the SRAM memory used for data storage. We also developed observation points by implementing error counters in the core architecture. These error counters were constantly read out to analyze the error propagation within the system. Besides that, we used the Coremark as a benchmark application of the SoC, which better utilizes SoC functionalities.

Finally, we characterized the improved RISC-V SoC implementation in a second test campaign. The results have shown that the SoC could correct most of the upsets caused by neutron radiation. Furthermore, the SoC hardening configuration in both the processor and the memory executed the Coremark benchmark without errors in all the executions.

In future works, we intend to improve the characterization of the SoC by carrying out test campaigns with particles present in space environments, such as heavy ions and protons. The source code of the processor is available at [START_REF] Risc-V | [END_REF].