Joachim Rosseel

Valérian Mannoni

Inbar Fijalkow
email: inbar.fijalkow@ensea.fr

Valentin Savin
email: valentin.savin@cea.fr

Decoding Short LDPC Codes via BP-RNN Diversity and Reliability-Based Post-Processing

Keywords: LDPC, short LDPC, neural network aided decoding, belief propagation, ordered statistics decoding postprocessing

published or not. The documents may come   L'archive ouverte pluridisciplinaire

I. INTRODUCTION

L AST years have witnessed an increased interest in re- search and practice of efficient error correcting codes for messages ranging from a few tens up to a few hundred bits, revived for instance by short-packet machine-to-machine communications, central to the emerging Internet of Things technology. While important progress has been made in understanding the limits of coding at short block lengths [START_REF] Polyanskiy | Channel coding rate in the finite blocklength regime[END_REF], the design of efficient short codes and decoding algorithms still raises many challenges [START_REF] Cos ¸kun | Efficient error-correcting codes in the short blocklength regime[END_REF].

Low-density parity-check (LDPC) codes [START_REF] Gallager | Low density parity check codes[END_REF] are a class of error correcting codes defined by sparse bipartite graphs [START_REF] Tanner | A recursive approach to low complexity codes[END_REF]. They are well-known for their excellent error correction performance at suitably large blocklengths, achieving near Shannon channel capacity performance under iterative belief propagation (BP) decoding [START_REF] Richardson | Design of capacityapproaching irregular low-density parity-check codes[END_REF]. For codes defined by cycle-free bipartite graphs, BP decoding outputs maximum a posteriori estimates of the coded bits [START_REF] Wiberg | Codes and decoding on general graphs[END_REF]. However, good codes are actually defined by graphs with cycles, in which case BP decoding is known to be sub-optimal.

First works on improving the iterative BP decoding performance actually focused on so-called high-density parity-check (HDPC) codes [START_REF] Kothiyal | Iterative reliability-based decoding of linear block codes with adaptive belief propagation[END_REF]- [START_REF] Hehn | Multiplebases belief-propagation decoding of high-density cyclic codes[END_REF]. These codes are defined by higher density bipartite graphs -e.g., Bose-Chaudhuri-Hocquenghem (BCH) codes, Reed-Muller codes, binary Golay codes, etc. -This work was supported by the French Agence Nationale de la Recherche (ANR), under grant number ANR-21-CE25-0006 (AI4CODE project).

for which the standard iterative BP decoding usually yields very poor error correction performance. In [START_REF] Kothiyal | Iterative reliability-based decoding of linear block codes with adaptive belief propagation[END_REF], an adaptive BP decoding approach was proposed, in which the decoding graph used by BP is updated at each iteration, according to the output of a reliability-based decoding algorithm, such as the ordered statistics decoding (OSD) [START_REF] Fossorier | Soft-decision decoding of linear block codes based on ordered statistics[END_REF]. The random recurrent decoding in [START_REF] Halford | Random redundant soft-in soft-out decoding of linear block codes[END_REF], [START_REF] Dimnik | Improved random redundant iterative HDPC decoding[END_REF] and the multiple-bases BP decoding in [START_REF] Hehn | Multiplebases belief-propagation decoding of high-density cyclic codes[END_REF] exploit a decoding diversity approach, in the form of different graph representations of the code, implying several BP decoders working either in serial or in parallel.

More recently, deep neural networks have received significant interest for improving the decoding performance of short codes [START_REF] Nachmani | Learning to decode linear codes using deep learning[END_REF]- [START_REF] Doan | Neural belief propagation decoding of CRC-polar concatenated codes[END_REF]. A weighted BP decoding has been introduced in [START_REF] Nachmani | Learning to decode linear codes using deep learning[END_REF], where the weights are optimized using a neural network (NN). The topology of the NN mimics the BP decoding process, with unwrapped decoding iterations. The approach can be used with either a feedforward (FF) or a recurrent NN (RNN), the corresponding decoders being termed as BP-FF or BP-RNN. It has been shown in [START_REF] Nachmani | Deep learning methods for improved decoding of linear codes[END_REF] that the BP-RNN is able to outperform the standard BP decoder for short BCH codes, belonging to the class of HDPC codes. Subsequently, several variants of NN-based BP decoding have been proposed in the literature. In [START_REF] Buchberger | Pruning and quantizing neural belief propagation decoders[END_REF], a pruning method of irrelevant check-nodes in a neural BP model has been proposed, aimed at jointly optimizing the code construction and the decoding. The design of new decoding rules for finitealphabet iterative decoders (FAIDs), based on a quantized NN model, was proposed in [START_REF] Xiao | Designing finite alphabet iterative decoders of LDPC codes via recurrent quantized neural networks[END_REF]. Moreover, a new training method of the quantized NN model was introduced in [START_REF] Xiao | FAID diversity via neural networks[END_REF], where training sets are constructed by sampling errors with trapping set support, to achieve decoding diversity for FAIDs on the binary symmetric channel. Resolving decoding failures due to trapping sets, by means of deep learning techniques, has also been recently investigated in [START_REF] Han | Deep-learning for breaking the trapping sets in low-density parity-check codes[END_REF].

In this paper, we consider the BP-RNN decoding of short LDPC codes. One critical difficulty faced by BP decoding in the finite blocklength regime is the presence of particular structures in the bipartite graph, which prevent the decoding algorithm from converging. Examples of such structures are trapping sets [START_REF] Richardson | Error floors of LDPC codes[END_REF] and absorbing sets [START_REF] Dolecek | Predicting error floors of structured LDPC codes: Deterministic bounds and estimates[END_REF], and they are closely related to the pseudo-codewords [START_REF] Frey | Skewness and pseudocodewords in iterative decoding[END_REF] and near-codewords [START_REF] Mackay | Weaknesses of Margulis and Ramanujan-Margulis low-density parity-check codes[END_REF] concepts. Here, we shall focus on the absorbing set concept, introduced in [START_REF] Dolecek | Predicting error floors of structured LDPC codes: Deterministic bounds and estimates[END_REF] as a combinatorial object associated with the bipartite graph, and defined independently of the particular message-passing decoding or channel noise model. For long LDPC codes, such structures are known to be responsible of the so-called error floor phenomenon, since their size may be relatively small with respect to the length of the code [START_REF] Dolecek | Predicting error floors of structured LDPC codes: Deterministic bounds and estimates[END_REF]. However, for short codes, their size may be comparable to the number of errors that the code must correct, therefore possibly inducing a significant degradation of the error correction performance in the waterfall region. To address this issue, we take a decoding diversity approach, implying several BP-RNN decoders working either in serial or in parallel, where each BP-RNN is trained to decode errors corresponding to absorbing sets of a specific type. We used a similar approach in our previous work [START_REF] Rosseel | Error structure aware parallel BP-RNN decoders for short LDPC codes[END_REF], where several BP-RNN decoders working in parallel were trained by exhaustively enumerating and classifying (according to the structure of the induced subgraph) all patterns of two and three errors. The work in [START_REF] Rosseel | Error structure aware parallel BP-RNN decoders for short LDPC codes[END_REF] was relevant to high coding rate LDPC codes, correcting a small number of errors. Here, we extend our previous work to lower rate codes, correcting a higher number of errors, by concentrating our attention to errors with absorbing set support. We further combine our approach with a low-order OSD post-processing step, providing an efficient way to bridge the gap to maximum likelihood (ML) decoding. The main contributions of the paper are summarized below.

Absorbing set classification and specialization of BP-RNN decoding: We first propose a graph-search based algorithm, combining backtracking and a deep-first search like procedure, to enumerate, in an efficient way, all the absorbing sets of a given size in the bipartite graph. We then perform a fine classification of the enumerated absorbing sets, according to the degree profile of the check-nodes in the induced subgraph, and train a specific BP-RNN decoder for each absorbing set class.

BP-RNNs selection and decoding architectures: To reduce the number of the BP-RNN decoders, we propose a selection procedure, where the selected decoders are the most complementary in terms of the errors they can decode. We then consider two decoding architectures, in which the selected BP-RNN decoders are executed in an either parallel or serial manner, and define the appropriate metrics to assess their computational complexity and decoding latency.

OSD post-processing: We further combine our approach with an OSD post-processing step, applied in case none of the selected BP-RNN decoders outputs a codeword. There are two motivations behind the use of such a post-processing step. First, for short LDPC codes, the OSD complexity is limited, compatible with practical applications, especially when the OSD order is small. Here, we restrict the order of the OSD post-processing step to either 0 or 1. Second, it may benefit from the diversity brought by the use of multiple BP-RNN decoders. Indeed, we show that the coding gain brought by the use of multiple BP-RNN decoders is actually amplified by the use of the OSD post-processing, resulting in a significant improvement of the error correction performance.

The paper is organized as follow. Section II introduces the notation and the BP-RNN decoding algorithm. Absorbing sets, as well as the tree-search based algorithm to enumerate them, their classification, and the training method used to specialize the BP-RNNs to absorbing set classes, are presented in Section III. The BP-RNNs selection, together with the parallel and serial and decoding architectures are presented in Section IV. Section V provides the details of the OSD postprocessing step. Finally, Section VI presents the numerical results, and Section VII concludes the paper.

II. PRELIMINARIES A. Neural BP decoding

We consider an LDPC code defined by a Tanner (bipartite) graph [START_REF] Tanner | A recursive approach to low complexity codes[END_REF] with N variable-nodes and M check-nodes, denoted respectively by n ∈ {1, . . . , N } and m ∈ {1, . . . , M }. We further denote by N (m) the set of variable-nodes connected to a check-node m, and by M(n) the set of check-nodes connected to a variable-node n.

BP decoding consists of an iterative exchange of messages along the edges of the Tanner graph, where each message provides an estimation of the incident variable-node [START_REF] Savin | LDPC decoders[END_REF]. BP-RNN and BP-FF decoding algorithms are weighted variants of the BP decoding. Exchanged messages are multiplied by weights which are learned through an either RNN or FF-NN approach, respectively. The underlying NN contains three types of neural layers, each one corresponding to a step of the BP algorithm. The check-pass layer and the data-pass layer carry out the computation of messages outgoing from checknodes and variable-nodes, respectively. Each one of them contains a number of neurons equal to the number of edges of the Tanner graph. In addition, the a posteriori layer consists of N neurons, computing the a posteriori Log Likelihood Ratio (LLR) values of the N variable-nodes. The three layers of the NN are connected such that a check pass layer, followed by a data pass layer and an a posteriori LLR layer model one iteration of the BP decoding. In particular, it should be noted that neurons in the check-pass and data-pass layers of the NN correspond to directed edges of the Tanner graph, m → n and n → m, respectively. NN edges between these layers connect two neurons sharing either a common variable-node (i.e., m → n and n → m ′ , with m ̸ = m ′ ) or a common check-node (i.e., n → m and m → n ′ , with n ̸ = n ′ ). For more details, we refer to [START_REF] Nachmani | Learning to decode linear codes using deep learning[END_REF].

The formulas below detail the calculation of messages within each layer. We denote by β m→n and α n→m the messages computed by the check-pass and data-pass layers, respectively, and by Ln the messages computed by the a posteriori LLR layer. The observed (channel) LLR values are denoted by L ch,n , and are used to initialize α n→m messages prior to the first iteration.

β m→n = 2 tanh -1   n ′ ∈N (m)\{n} tanh α n ′ →m 2   (1)
α n→m = L ch,n + m ′ ∈M(n)\{m} w m ′ →n→m β m ′ →n (2)
Ln = L ch,n + m∈M(n) wm→n β m→n (3)
It can be observed that weights (w, w) are applied only on the NN edges incoming to the data-pass (2) and a posteriori LLR (3) layers. Each weight corresponds to one specific edge of the NN. In (2) the weights are denoted by w m ′ →n→m , where the subscript indicates both the corresponding neuron n → m in the data-pass layer, and the incoming NN edge from neuron m ′ → n in the check-pass layer. In (3), the weights are denoted by wm→n , where the subscript indicates the corresponding neuron n in the a posteriori LLR layer, and the incoming NN edge from neuron m → n in the check-pass layer. For the BP-RNN, the weights only depend on the corresponding edges of the NN, while for the BP-FF they also depend on the iteration number (note that, to simplify notation, we have not indicated the iteration number on the above formulas).

It is worth noticing that despite a lower number of trainable weights, the BP-RNN achieves similar performance to the BP-FF [START_REF] Nachmani | Deep learning methods for improved decoding of linear codes[END_REF]. Therefore, we will only consider the BP-RNN model, although our work can be readily adapted to the BP-FF model. Moreover, to further reduce the number of trainable weights, we use the approach suggested in [START_REF] Nachmani | Deep learning methods for improved decoding of linear codes[END_REF], with the data-pass layer computation modified as,

α n→m = L ch,n + w n→m m ′ ∈M(n)\m β m ′ →n , (4)
where the applied weight w n→m only depends on the datapass neuron n → m. This simplification reduces the training complexity and makes it possible to reuse conventional BP decoding architectures for an efficient hardware implementation.

B. Loss function

The goal of BP-RNN decoding is to map the received signal to a valid codeword. This process can be interpreted as a classification task, with one class corresponding to one codeword of the code. However, this is practically unrealistic due to the high number of classes [START_REF] Lian | Learned beliefpropagation decoding with simple scaling and SNR adaptation[END_REF]. A generally accepted method to avoid this problem is to replace the initial classification task by a simpler binary classification task for each bit. To do so, the use of the average binary cross-entropy loss function was suggested in [START_REF] Nachmani | Learning to decode linear codes using deep learning[END_REF], [START_REF] Nachmani | Deep learning methods for improved decoding of linear codes[END_REF]. When both the channel and the decoder are symmetric -e.g., the additive white Gaussian noise (AWGN) channel and BP-RNN decoder considered in this work -training (as well as testing) may be performed by assuming the transmitted codeword is the all-zero one. In this case, the average binary cross-entropy loss function simply writes as (see [START_REF] Nachmani | Deep learning methods for improved decoding of linear codes[END_REF] for details),

Loss( L) = - 1 N N n=1 log(σ( Ln )), (5)
where the a posteriori LLR value Ln is taken at the last decoding iteration, and σ(z) = (1+exp(-z)) -1 is the sigmoid function, converting the LLR value into the probability that the decoded bit is equal to zero. The loss function ( 5) is then minimized during the NN training, thus minimizing the average bit error probability.

Multi-loss functions have also been considered in [START_REF] Nachmani | Learning to decode linear codes using deep learning[END_REF], [START_REF] Nachmani | Deep learning methods for improved decoding of linear codes[END_REF] (see also [START_REF] Lian | Learned beliefpropagation decoding with simple scaling and SNR adaptation[END_REF]), which take into account the a posteriori LLR values after each decoding iteration. This allows amplifying the contribution of earlier iterations to the computed gradient. In [START_REF] Nachmani | Deep learning methods for improved decoding of linear codes[END_REF], multi-loss optimization has been shown to produce effective gains for high-density BCH codes, with both BP-RNN and BP-FF models. However, when sparser, cycle reduced parity check matrices where chosen for the same codes, the gain was observed to significantly reduce, or even completely vanish. The same observation was made for checknode regular matrices. In this work we consider short LDPC codes, whose parity-check matrices are both cycle-reduced and check-node regular. In our experiments, we did not observe any gain brought by multi-loss optimization compared to [START_REF] Richardson | Design of capacityapproaching irregular low-density parity-check codes[END_REF]. We therefore consider the loss function in [START_REF] Richardson | Design of capacityapproaching irregular low-density parity-check codes[END_REF] to train our BP-RNN decoders.

C. Number of decoding iterations

The standard approach for NN-based BP is to both train and test the decoder for a same maximum number of decoding iterations, usually small, such as 5 or 10 iterations (e.g., [START_REF] Nachmani | Learning to decode linear codes using deep learning[END_REF], [START_REF] Nachmani | Deep learning methods for improved decoding of linear codes[END_REF], [START_REF] Lian | Learned beliefpropagation decoding with simple scaling and SNR adaptation[END_REF]). For the BP-FF model, increasing the number of decoding iterations for training also increases the number of trainable weights, which may become computationally intractable. For the BP-RNN model, this is no longer an issue. Yet, assuming that we want to test the BP-RNN decoder for an increased number of decoding iterations, a natural question is whether it is optimal to use the same number of decoding iterations for training. At first sight, this might be the case. However, when the BP decoder does not converge, it may exhibit erratic behavior, especially during the last decoding iterations. Increasing the number of decoding iterations for training may amplify this behavior, and impair the optimization result. We will provide numerical evidence for this in Section VI-B. For the moment, this motivates us to introduce two parameters, denoted I train and I test , corresponding to the maximum number of decoding iterations during the training and the testing phases, respectively.

III. ABSORBING SET CLASSIFICATION AND SPECIALIZATION OF BP-RNN DECODING A. Absorbing sets

We consider the definition of absorbing sets from [START_REF] Dolecek | Predicting error floors of structured LDPC codes: Deterministic bounds and estimates[END_REF], [START_REF] Dolecek | Analysis of absorbing sets and fully absorbing sets of array-based LDPC codes[END_REF], given below for the reader's convenience.

Let A ⊂ {1, . . . , N } be a set of variable-nodes. We consider the set of check-nodes connected to A at least once, which we partition in two disjoint subsets O(A) and E(A), denoting the sets of check-nodes connected to A either an odd or an even number of times, respectively. The set A is said to be an absorbing set, if any variable-node in A has strictly more neighbors in E(A) than in O(A).

According to the above definition, if the variable-nodes in error form as absorbing set, then each of them is connected to a higher number of satisfied check-nodes (not detecting the error) than unsatisfied check-nodes (detecting the error). Such errors are likely to mislead the BP decoder, yielding a decoding failure with high probability.

We say that an absorbing set A is of type ν-(ω, ε), where ν := |A|, ω := |O(D)| and ε := |E(D)|, and |S| denotes the number of elements in the set S (see also Fig. 1). Thus, the subgraph induced by A comprises ν variable-nodes and ω + ε check-nodes, of which ω are of odd degree and ε are of even degree. Note that absorbing sets of the same type do

B. Graph-search based algorithm

A brute-force search algorithm to enumerate all the trapping/absorbing sets of a given size ν would have to explore all the N ν candidates, which may become computationally intractable even for relatively small values of N and ν. Several exhaustive/non-exhaustive enumeration algorithms have been proposed in the literature, to enumerate elementary trapping sets (i.e., whose induced subgraphs contain only degree-1 and degree-2 check nodes) [START_REF] Karimi | On characterization of elementary trapping sets of variable-regular LDPC codes[END_REF]- [START_REF] Falsafain | Exhaustive enumeration of elementary trapping sets of an arbitrary Tanner graph[END_REF], trapping or absorbing sets for specific classes of LDPC codes [START_REF] Dolecek | Analysis of absorbing sets and fully absorbing sets of array-based LDPC codes[END_REF], [START_REF] Abu-Surra | Trapping set enumerators for specific LDPC codes[END_REF] or only small or dominant such structures [START_REF] Wang | Artificial neural network[END_REF], [START_REF] Karimi | Efficient algorithm for finding dominant trapping sets of LDPC codes[END_REF], and fully absorbing sets (i.e., where the absorbing condition is satisfied by all the variable-nodes in the graph) [START_REF] Kyung | Exhaustive search for small fully absorbing sets and the corresponding low error-floor decoder[END_REF], [START_REF] Zhang | Efficient algorithms to find all small errorprone substructures in LDPC codes[END_REF]. Several of these algorithms rely on a linear programming based branch-andbound approach, e.g., [START_REF] Falsafain | Exhaustive enumeration of elementary trapping sets of an arbitrary Tanner graph[END_REF], [START_REF] Kyung | Exhaustive search for small fully absorbing sets and the corresponding low error-floor decoder[END_REF], [START_REF] Zhang | Efficient algorithms to find all small errorprone substructures in LDPC codes[END_REF].

Here, we propose a graph search based algorithm, which, to the best of our knowledge, is the first specifically developed for the exhaustive enumeration of absorbing sets, without any restriction on the structure of the Tanner graph of the absorbing sets to be enumerated. The proposed algorithm is essentially a backtracking algorithm that incrementally builds absorbing set candidates, and abandons a candidate as soon as it determines that it cannot possibly be completed to an absorbing set. Candidates are built incrementally by traversing the bipartite graph in a depth-first search (DFS) manner, that is, starting from a variable-node chosen as root, and exploring as deeply as possible before backtracking. The main difference with the standard depth-first search is that our algorithm does not visit only one, but a subset of variable-nodes at each depth level, which are chosen to increment the candidate solution.

Let n ∈ {1, . . . , N } be a fixed variable-node, which we will refer to as root node. Our algorithm enumerates all the absorbing sets A ⊂ {1, . . . , N } containing n, of size |A| = ν.

To avoid enumerating a same absorbing set multiple times (i.e., starting with different root nodes), we will further require n to be the smallest element of A, that is, A ⊂ {n, . . . , N }.

We first expand the bipartite graph starting from the root node n, and incrementally adding check and variable nodes at an increasing distance1 from n, until the most distant node in the bipartite graph is reached. This expansion produces a rooted bipartite graph, which we will denote by H. For ℓ ≥ 0, we denote by V ℓ the set of variable-nodes at distance 2ℓ from n (thus, V 0 = {n}), and by C ℓ the set of check-nodes at distance 2ℓ + 1 from n. Let A be any set of variable-nodes, containing n, and let C be the set of check-nodes connected to A at least once. For ℓ ≥ 0, we define A ℓ := A∩V ℓ and C ℓ := C ∩C ℓ . We say that A ℓ satisfies the absorbing set condition, and we write AS check(A ℓ ) = true, if any variable-node in A ℓ has strictly more neighbors in E(A) than in O(A). Now, the neighbors of the variable-nodes in A ℓ are check-nodes that belong to either C ℓ-1 or C ℓ , and whose degrees, in the subgraph induced by A, only depend on A ℓ-1 (for ℓ > 0), A ℓ , and A ℓ+1 (for ℓ less than the maximum depth of A). Hence, one may check the absorbing set condition for A in an incremental manner, by checking it for each subset A ℓ , which however requires the knowledge of the next level subset A ℓ+1 (except for the last level).

The proposed algorithm incrementally builds an absorbing set A, by choosing subsets A ℓ , for ℓ ≥ 0, and checking the absorbing set condition to determine whether the current choice could possibly be completed to a valid absorbing set. For ℓ = 0, we have A 0 = {n}. Consider some choice of subsets A 0 , . . . , A ℓ , with ℓ ≥ 0, such that νℓ := ℓ t=0 |A t | ≤ ν. We define the set of possible completions C(A 0 , . . . , A ℓ ), as follows:

• if νℓ = ν, then C(A 0 , . . . , A ℓ ) = ∅ (absorbing set size ν already reached, thus, no completion needed).

• if νℓ < ν, then
(1) determine the subset N ℓ+1 ⊂ V ℓ+1 containing the descendants of the variable-nodes in A ℓ .

(2) set N (≥n) ℓ+1 := N ℓ+1 ∩ {n, . . . , N }, and

C(A 0 , . . . , A ℓ ) := A ℓ+1 ⊆ N (≥n) ℓ+1 |A ℓ+1 | ≤ ν -νℓ . (6)
Note that C(A 0 , . . . , A ℓ ) is a set of subsets A ℓ+1 (which may be empty, if either νℓ = ν or N (≥n) ℓ+1 = ∅), and which we may iterate through in any convenient order, e.g., according to increasing size of subsets A ℓ+1 .

The procedure 1 AS DFS provides a recursive implementation of the proposed algorithm. Given a variable-node n, the procedure is simply called from the main program with inputs AS DFS(H, ℓ = 0, A 0 = {n}).

To illustrate the capability of the proposed algorithm, we consider the following two codes, both of rate 1/2, which will be subsequently used throughout this paper. Code-1 is a regular LDPC code, of length 64 bits, with variable-nodes of degree 3 and check-nodes of degree 6, constructed by using the progressive edge growth (PEG) algorithm [START_REF] Hu | Regular and irregular progressive edge-growth Tanner graphs[END_REF]. It has girth g = 6, with multiplicity 164 (i.e., number of cycles of length g). Code-2 is the LDPC code from [START_REF]Short block length LDPC codes for TC synchronization and channel codding[END_REF], known as the CCSDS LDPC code. It is of length 128 bits, with half of variablenodes of degree 3 and the other half of degree 5, and check-nodes of degree 8. It has girth g = 6, with multiplicity 2336.

Procedure 1 AS DFS(H, ℓ, A 0 , . . . , A ℓ ) if ℓ > 0 and AS check(A ℓ-1 ) = false then return // backtrack end if νℓ ← |A 0 | + • • • + |A ℓ | if νℓ = ν then if AS check(A ℓ ) = true then Add A := A 0 ∪ • • • ∪ A ℓ to
In Table I, we provide the total number of different extended-type (ET) values and the total number of absorbing sets, for both Code-1 and Code-2, and absorbing set size values ν ≤ 8. We note that for Code-2 and ν = 8, a brute-force search algorithm would require exploring a number of 128 8 ≈ 2 40 candidates. Our algorithm enumerated all the absorbing sets of size ν = 8 in 38 minutes (Intel Xeon @2.20GHz processor).

C. Training specialized BP-RNN decoders

Having at our disposal an efficient algorithm to enumerate absorbing sets of a given size, we perform a fine classification of the found absorbing sets, by grouping into a same class absorbing sets with the same extended type. The goal is to train a specific BR-RNN decoder for each class. The approach bears similarity to, and is motivated by [START_REF] Xiao | FAID diversity via neural networks[END_REF], [START_REF] Declercq | Finite alphabet iterative decoders -Part II: Towards guaranteed error correction of LDPC codes via iterative decoder diversity[END_REF], where decoding rules for FAIDs have been either designed or learned to correct specific trapping sets for the binary symmetric channel.

We consider a binary-input AWGN channel, with BPSK alphabet (±1) inputs, and fixed noise variance σ 2 . Assuming the all-zero codeword is transmitted (see Section II-B), corre- sponding to the +1 input of the channel, the received word is given by

y n = 1 + z n , z n ∼ N (0, σ 2 ), n = 1, . . . , N, (7)
where N (0, σ 2 ) denotes a normal distribution, with mean 0 and variance σ 2 . The signal-to-noise ratio (SNR) is defined as SNR = -10 log 10 (σ 2 ) dB. Given a received word y := (y 1 , . . . , y N ), we define its error set as E(y) := {n | y n ≤ 0} ⊂ {1, . . . , N }. Our goal is to construct a specific training set, containing only words y, such that E(y) is an absorbing set of a given extended type ν-(ω, ε, P c ). As a neural network specializes to trends embedded in the training set [START_REF] Wang | Artificial neural network[END_REF], we expect the trained BP-RNN to specialize in decoding words y, such that E(y) is, or possibly contains, an absorbing set of extended type ν-(ω, ε, P c ).

One way to generate such a training set is to randomly generate words y according to [START_REF] Kothiyal | Iterative reliability-based decoding of linear block codes with adaptive belief propagation[END_REF], and to select those for which E(y) is an absorbing set of extended type ν-(ω, ε, P c ). This would be rather tedious and time consuming. A more efficient way is to use our knowledge of absorbing sets (determined by the algorithm from Section III-B), and generate Gaussian noise by means of a truncated Gaussian distribution, to produce errors only on desired locations. Precisely, we proceed as follows. We first chose an absorbing set A at random, from those having the desired extended type ν-(ω, ε, P c ). Then we generate a random word y = (y n = 1 + z n ) n=1,...,N , with

z n ∼ N (0, σ 2 , -∞, -1), if n ∈ A N (0, σ 2 , -1, ∞), otherwise (8)
where N (0, To illustrate the specialization of the trained decoder, Fig. 2 shows the weight values for three BP-RNN decoders, trained for three different error classes, with corresponding extended types shown in the legend. The (unspecialized) BP-RNN decoder trained as in [START_REF] Nachmani | Deep learning methods for improved decoding of linear codes[END_REF] is also shown. To avoid clutter, weight values are sorted in increasing order, and an ordered set of weight values is referred to as weight profile2 . We consider the Code-1 and show the weight profiles for the both data pass and a posteriori layers. The data pass layer is implemented as in (4), with a same weight applied on all the incoming edges to each neuron (see Section VI for a complete list of training parameters). Hence, the number of weights is the same for both the data pass and the a posteriori layers, equal to the number of edges of the Tanner graph (192 for Code-1). The weight profiles in Fig. 2 clearly indicate that weight optimization responds differently to training sets corresponding to different error classes. We exploit in the next section the diversity induced by the BP-RNN specialization.

IV. BP-RNNS SELECTION AND DECODING ARCHITECTURES

A. BP-RNN diversity selection

Using the procedure from Section III-C, we train one specialized BP-RNN decoder for each error class, i.e., extended type3 ν-(ω, ε, P c ), with absorbing error set size ν ≤ ν max (the choice of ν max is discussed in Section VI). Let J denote the total number of specialized BP-RNN decoders.

Here, we propose a selection procedure to reduce the number of BP-RNN decoders, as well as to avoid similar training effects (which may arise, for instance, due to absorbing sets of a given size containing absorbing sets of smaller sizes). To do so, we assess the complementarity of the trained decoders, in terms of the errors they can decode.

We first generate a common test set, T , containing random words y defined as in [START_REF] Kothiyal | Iterative reliability-based decoding of linear block codes with adaptive belief propagation[END_REF], which is then used to assess the individual error correction performance of each of the trained decoders. We denote by F j ⊂ T the susbset of words on which the BP-RNN decoder D j failed, where j = 1, . . . , J. Then, we recursively construct an ordered list of decoder indexes, denoted J, as follows. We start by initializing J as the empty list, J = ∅. To add a new index, j new to J, we use the following rule,

j new = argmin j∈{1,...,J}\J |F J ∩ F j | , (9)
where F J := T , if J = ∅, and F J := ∩ i∈J F i , otherwise. The above rule is applied recursively J times, until J contains all the decoder indexes j = 1, . . . , J, in a sorted order. In case the argmin in ( 9) is not unique, an arbitrary one is chosen. Note that when J = ∅, the rule (9) rewrites as j new = argmin j∈{1,...,J} |F j |. Hence, the first decoder (index) added to the list is the one minimizing the word error rate. Subsequently, when J ̸ = ∅, the new decoder added to the list is the most complementary with those already in J, in the sens that it minimizes the number of words on which all the decoders indexed by J ∪ {j new } fail.

For Z ≤ J, let J(1:Z) ⊂ J be the sublist defined by the first Z indexes in J. We define

D Z := {D j | j ∈ J(1:Z)} (10)
Note that D Z is an ordered list of decoders 4 , which we will refer to as BP-RNN diversity of size Z (using a similar terminology to the one in [START_REF] Xiao | FAID diversity via neural networks[END_REF], [START_REF] Declercq | Finite alphabet iterative decoders -Part II: Towards guaranteed error correction of LDPC codes via iterative decoder diversity[END_REF]). The Z BP-RNN decoders in D Z may then be used with either a parallel or a serial decoding architecture, as discussed in the next section. The value of Z may be dictated by complexity reasons, or, as illustrated in Section VI-C, chosen to ensure small (negligible) degradation of the error correction performance, with respect to the case when all J BP-RNN decoders are used.

B. BP-RNN diversity decoding architectures

We consider a BP-RNN diversity D Z , comprising Z BP-RNN decoders. For simplicity, we index the BP-RNN decoders in D Z from 1 to Z, thus D Z = {D 1 , . . . , D Z }. Fig. 3 shows the proposed parallel and serial architectures, using the Z BP-RNN decoders in D Z .

In the parallel architecture, each decoder outputs an estimate ĉj = (c j,1 , . . . , c j,N ) ∈ {0, 1} N of the transmitted codeword c, according to the sign of the corresponding LLR values at the last decoding iteration. The output of the parallel architecture is determined as the maximum likelihood (ML) codeword, among the codewords outputted by the constituent BP-RNN decoders (if any, see below). For the binary-input AWGN channel (with ±1 inputs), the ML criterion simply writes as

where S := {ĉ j | syndrome(ĉ j ) = 0} denotes the set of ĉj 's verifying the syndrome. Decoding is then successful if ĉ is equal to the transmitted codeword. If none of the BP-RNN decoders outputs a codeword, the decoding fails. In such a case, the ML criterion -or a similar bitwise maximum a posteriori criterion -may still be applied to select one of the ĉj outputs, if desired, e.g., in order to minimize the bit error rate of the decoder (however, we will only be concerned with word error rate results in this paper).

In the serial architecture, the constituent BP-RNN decoders are run sequentially according to the order given by the sorting procedure from Section IV-A. Decoding stops as soon as a BP-RNN decoder D j outputs a codeword ĉj (syndrome(ĉ j ) = 0), which becomes the output ĉ of the serial architecture. Decoding is then successful if ĉ is equal to the transmitted codeword. If none of the BP-RNN decoders outputs a codeword, decoding fails. For simplicity, in Fig. 3b, we take ĉZ as the output of the serial architecture in this case.

The parallel architecture yields a reduced maximum decoding latency, compared to the serial one. This comes however at the cost of an increased complexity, from the both computational and hardware perspectives, since Z BP-RNN decoders must be instantiated in hardware. For the serial architecture, only one decoder may be instantiated in hardware, which may then be reused to perform sequentially the Z BP-RNNs (while updating the corresponding set of weights). The averagecase computational complexity, as well as the average-case decoding latency, for the both parallel and serial architectures are discussed in the next section.

C. Average-case complexity and decoding latency metrics

We consider a BP-RNN diversity D Z , where all the constituent decoders are set with a maximum number of decoding iterations I test (see Section II-C). For a BP-RNN decoder D j , we denote by I Dj ,y ≤ I test the number of iterations performed by D j to decode y, and define I Dj := E y (I Dj ,y ), where E denotes the expected value operator. In practice, I Dj is estimated by averaging over the decoded words y. We further define the average number of decoding iterations of a decoding (parallel or serial) architecture using D Z , as

I(D Z ) = Dj ∈D Z I Dj . ( 12
)
Note that for the serial architecture, I Dj ,y may be equal to zero, in case the decoding process stops before reaching D j . Thus, I Dj may be close to 0, if D j is rarely used (unlike the parallel architecture, which uses all decoders). Since the computational complexity scales linearly with the number of decoding iterations performed by the decoder, we use the average number of decoding iterations I D Z as a measure of the average-case computational complexity of the BP-RNN diversity architecture.

For the serial architecture, I Dj may also be seen as a measure of the average-case decoding latency, since decoding iterations are performed sequentially within each constituent decoder, while the constituent decoders are also run sequentially. However, this is no longer the case for the parallel architecture. Hence, we define the decoding latency of the parallel architecture, for decoding a given word y, as

L D Z ,y = max Dj ∈D Z I Dj ,y , (13)
corresponding to the maximum number of iterations performed by the constituent decoders to decode y. Finally, the averagecase decoding latency of the parallel architecture is defined as

L(D Z ) := E y (L D Z ,y ), (14)
and is estimated by averaging over the decoded words y.

V. OSD POST-PROCESSING FOR BP-RNN DECODERS

OSD was first proposed in [START_REF] Fossorier | Soft-decision decoding of linear block codes based on ordered statistics[END_REF], as a decoding method capable to approach the ML decoding performance, for moderatelength linear block codes, with polynomial complexity. It can be used as a stand-alone decoding algorithm, exploiting the soft-output of the channel (L ch,n ), or as a post-processing step, exploiting the output of a soft-decision decoder ( Ln , see Section II-A, for the notation).

In OSD, variable-nodes are first sorted according to their reliability (that is the absolute value of the corresponding soft decision). The parity-check matrix of the code is then brought to a systematic form 5

, H = [A | I],

where A is a matrix of size M × (N -M ) and I is the identity matrix of size M × M , and so that the K := N -M columns of A correspond to the most possible6 reliable variable-nodes. By a slight abuse of language, we simply refer to variablenodes corresponding to the columns of A as the most reliable ones, and to the remaining variable-nodes as the least-reliable ones. In OSD-0, hard-decision is made on the most reliable variable-nodes, and the least reliable ones are determined by solving the linear system given by H. Hence, decoding is successful if and only if the most reliable variable-nodes are error-free. To address the case where these variable-nodes contain errors, OSD-w considers all the possible choices of at most w errors among them. For each choice, the initial hard-decision of the corresponding variable-nodes is flipped, and the least reliable variable-nodes are determined again by solving the linear system given by H. This procedure produces a list of w i=0 K i codewords, from which the most likely one is selected, according to an ML rule, such as [START_REF] Fossorier | Soft-decision decoding of linear block codes based on ordered statistics[END_REF]. OSD-w may closely approach the ML decoding performance, assuming the w value (referred to as OSD order) is suitably large.

To bridge the error correction performance gap between suboptimal BP decoding and ML decoding, [START_REF] Fossorier | Iterative reliability-based decoding of low-density parity check codes[END_REF] suggested combining BP decoding with a low-order OSD (w ≤ 1), where an OSD step is performed at the end of each iteration of the BP decoding. Here, we propose the use of OSD as a post-processing step, applied only in case that none of the constituent BP-RNN decoders (of the BP-RNN diversity D Z ) outputs a codeword. In such a case, we process one OSD using the soft-decision (a posteriori LLRs) delivered by each of the constituent BP-RNN decoders. This produces a list of Z codewords (one for the OSD post-processing of each BP-RNN decoder), and the ML rule ( 11) is used to determine the most-likely one, which becomes the outputted codeword ĉ. Note that the above description applies to the both parallel and serial architectures, since OSD post-processing is only performed when all the constituent BP-RNN decoders failed to find a codeword. To reduce the complexity of the postprocessing step, we also limit the order of the OSD to w ≤ 1.

There are two main motivations behind the use of the OSD post-processing step. First, the complexity of the low-order OSD is dominated by the Gaussian elimination step, needed to bring the parity check-matrix to a systematic form. However, for LDPC codes, the sparsity of the parity-check matrix can be advantageously exploited to significantly reduce the complexity of this step. See for instance the method proposed in [START_REF] Lamacchia | Solving large sparse linear systems over finite fields[END_REF] for solving sparse linear systems, which has been adapted in [START_REF] Richardson | Efficient encoding of low-density paritycheck codes[END_REF] to derive an efficient encoding technique for LDPC codes, and in [START_REF] Burshtein | Efficient maximum-likelihood decoding of LDPC codes over the binary erasure channel[END_REF] to derive an efficient ML decoding algorithm for LDPC codes over erasure channels (the situation is similar for OSD, where the least reliable variable-nodes can be seen as being erased). Second, we use OSD to postprocess the soft-output of BP-RNN decoders. Since the loss function ( 5) used to train these decoders penalizes negative LLR values, corresponding to variable-nodes in error 7 , we expect such negative LLR values to have a reduced amplitude, thus reducing the probability of error on the most reliable variable-nodes. This is illustrated in Fig. 4, where we plot the cumulative distribution function (CDF) of the a-posterori LLR values for the BP decoder, and for the BP-RNN decoder trained on the error class 7- (5, 8, (5, 8)). Moreover, we also expect OSD post-processing to benefit from the diversity brought by the use of multiple BP-RNN decoders, increasing the probability that at least one of these decoders has at most w errors among the most-reliable variable-nodes.

VI. NUMERICAL RESULTS

A. Training settings

We consider the two LDPC codes of rate 1/2 and length either 64 (Code-1) or 128 (Code-2) bits, detailed in Section III-B. We train one specialized BP-RNN decoder for each absorbing set error class, with error set size ν ≤ ν max . 7 Recall the all-zero codeword assumption from Section II-B. • For Code-1, we choose ν max = 8, which gives a total number of J = 52 error classes (see Table I 8 ). • For Code-2, we choose ν max = 7, which gives a total number of J = 120 error classes (see Table I). For Code-2, the choice of ν max = 7 is due to complexity reasons (to limit the number of trained decoders). However, we note that for an SNR = 4 dB, in the waterfall region of Code-2, the average number of errors is N p e = 7.2, where N = 128 is the code-length, p e = Q(1/σ) = 0.0565 is the error probability of the binary-input AWGN channel, σ is the standard deviation of the Gaussian noise, and Q denotes the Q-function. For Code-1, for the same SNR = 4 dB, the size of a random error set is less than or equal to the chosen ν max = 8, with probability slightly greater than 0.99.

All the numerical results presented in this section use the BP-RNN model from Section II-A, with the data-pass layer defined in [START_REF] Tanner | A recursive approach to low complexity codes[END_REF]. Each specialized BP-RNN is trained independently, using the training set construction method presented in Section III-C. In addition, we also train an unspecialized BP-RNN decoder, according to the procedure described in [START_REF] Nachmani | Deep learning methods for improved decoding of linear codes[END_REF], to provide a benchmark for the presented numerical results. We use the same SNR for training and testing, thus, all BP-RNNs are trained for each SNR value ranging from 1 dB to 6 dB, with a step of either 0.5 dB or 1 dB. The choice of the maximum number of decoding iterations during the training and the testing phases, I train and I test (see Section II-C), will be discussed in the next section. Finally, we mention that we used the Keras library for training, with training parameters shown in Table II.

B. Maximum number of decoding iterations for training and testing

To illustrate the discussion from Section II-C, we fix the maximum number of decoding iterations for testing our BP-RNN decoders to I test = 25 iterations, and investigate here the impact of the maximum number of decoding iterations used at the training phase, I train .

We consider the Code-1, and train all the BP-RNN decoders for I train ∈ {5, 10, 20, 25}. Fig. 5 shows the frame error rate (FER) results, using I test = 25 iterations, for (a) the BP-RNN decoder specialized on the error class 7-(5, 8, (5, 8)), indicated as D 1 in the legend9 , (b) the unspecialized BP-RNN decoder [START_REF] Nachmani | Deep learning methods for improved decoding of linear codes[END_REF], and (c) the serial architecture using the 52 specialized BP-RNN decoders. We observe no noticeable difference on the FER performance, except in Fig. 5a, where the FER performance for I train = 5 is slightly degraded with respect to I train ∈ {10, 20, 25}. In the following, we choose I train = 10, which allows for faster training. We can note that similar observations hold for Code-2 (not shown here).

C. BP-RNN diversity selection

We apply now the BP-RNN diversity selection procedure from Section IV-A, to both Code-1 and Code-2. We proceed as follows. First, we fix SNR = 5 dB, and we train J BP-RNN decoders specialized on the J error classes (J = 52 for Code-1, J = 120 for Code-2). We order these decoders according to the procedure described in Section IV-A, where we use a common test set T containing 10 8 noisy codewords (SNR = 5 dB) to assess their individual error correction performance. Then we select a diversity D Z of BP-RNN decoders according to [START_REF] Hehn | Multiplebases belief-propagation decoding of high-density cyclic codes[END_REF], corresponding to Z different error classes. Subsequently, we only consider BP-RNN decoders specialized on the selected error classes, but we train them again for each SNR value used for testing 10 .

For Code-1, Fig. 6 shows the SNR required for achieving a target FER = 10 -4 , as a function of the number of selected BP-RNN decoders Z (only results for Z = 1, . . . , 20 are 10 We have also performed simulations using the BP-RNN decoders trained for SNR = 5 dB only, and testing them on different SNR values. Our simulation results were very similar to those obtained by training again the BP-RNN decoders for the actual SNR value used for testing. We chose to show simulation results for the case where the training and testing SNR values are equal, simply for consistency reasons. shown, since we observe no further improvement of the FER for higher Z values). We choose Z = 10, corresponding to an SNR loss of less than 0.1 dB, with respect to the case when all the J BP-RNN decoders are used. We use the same procedure to select a number of BP-RNN decoders for Code-2, which yields the same value Z = 10, for which the SNR loss is again less than 0.1 dB. Finally, in Table III, we show the extended types for the Z = 10 error classes, corresponding to the selected decoders.

D. FER results, complexity and decoding latency evaluations

We consider the BP-RNN diversity D 10 composed of the Z = 10 BR-RNN decoders selected in the previous section, and evaluate the FER performance, as well as the average-case complexity and decoding latency, for both parallel and serial architectures from Section IV-B. Fig. 7 shows the FER results for (a) Code-1, and (b) Code-2. For comparison purposes, we also show the FER performance of the BP decoder and the BP-RNN decoder from [START_REF] Nachmani | Deep learning methods for improved decoding of linear codes[END_REF], with either I test = 25, or I test = 250. The latter I test value corresponds to the cumulative maximum number of iterations performed by the BP-RNN decoders in the diversity D 10 . First, we observe that the parallel and serial architectures exhibit virtually the same FER performance, the corresponding curves being practically superimposed one on another. Compared to the conventional BP decoding, the BP-RNN diversity D 10 produces an SNR gain of approximately 0.4 dB with respect to BP(I test = 25). This comparison is relevant to applications with strict latency requirements, since both the BP(I test = 25) and the parallel BP-RNN diversity have the same worst-case decoding latency. By way of comparison, for Code-2, a similar gain over the conventional BP has been recently reported in [START_REF] Geiselhart | Automorphism ensemble decoding of quasi-cyclic LDPC codes by breaking graph symmetries[END_REF]Fig. 6], by using an automorphism ensemble decoding (AED) approach, with 16 BP decoders working in parallel 11 . If the worst-case latency constraint is relaxed, it can be observed from Fig. 7 that BP(I test = 250) achieves similar FER performance to the BP-RNN diversity D 10 . Similar considerations hold for the comparison between the (unspecialized) BP-RNN [START_REF] Nachmani | Deep learning methods for improved decoding of linear codes[END_REF] and the BP-RNN diversity D 10 , with one noticeable exception for Code-1, for which it can be observe that increasing the number of iterations from I test = 25 to I test = 250 does not improve the FER performance of the BP-RNN [START_REF] Nachmani | Deep learning methods for improved decoding of linear codes[END_REF] decoder (or only slightly in the low FER region). Finally, we note that using only the first decoder (D 1 ) of our specialized BP-RNN decoders yields similar FER performance to the unspecialized BP-RNN [START_REF] Nachmani | Deep learning methods for improved decoding of linear codes[END_REF]. While this is not shown in the figure (to avoid clutter), it can be observed for Code-1 by comparing the FER results in Figs. 5a and5b. Fig. 7c shows the average-case computational complexity and the average-case decoding latency results (Code-2 only), evaluated using ( 12) and ( 14) respectively. We compare the BP(I test = 250) and the BP-RNN diversity D 10 with either the serial or the parallel architecture, as they achieve similar FER performance. For the BP decoder and the serial D 10 , the average number of decoding iterations ( Ī) is a measure of both average-case computational complexity and averagecase decoding latency. For the parallel D 10 , Ī only measures the average-case computational complexity, while the averagecase decoding latency is measured by L. While the parallel D 10 has the lower worst-case latency, it can be observed that it also exhibits a reduced average-case latency in the first part of the waterfall (up to 4 dB). The serial D 10 may be seen as an alternative to BP(I test = 250), providing not only similar decoding performance, but also similar worst-and averagecase computational complexity and decoding latency. Yet, both the parallel and the serial D 10 retain the advantage of decoding diversity, which can be conveniently exploited by the proposed OSD post-processing step, as illustrated in the next section. 11 We did not include the AED-16 curve from [START_REF] Geiselhart | Automorphism ensemble decoding of quasi-cyclic LDPC codes by breaking graph symmetries[END_REF] in Fig. 7b, to avoid clutter. The gain reported in [START_REF] Geiselhart | Automorphism ensemble decoding of quasi-cyclic LDPC codes by breaking graph symmetries[END_REF] was observed using 32 decoding iterations.

E. FER results using OSD post-processing

We consider low-order (w = 0, 1) OSD post-processing applied to the BP(I test = 250), the unspecialized BP-RNN(I test = 250) decoder [START_REF] Nachmani | Deep learning methods for improved decoding of linear codes[END_REF], and the BP-RNN diversity D 10 (I test = 25). Since the parallel and the serial D 10 exhibit similar FER performance, and OSD post-processing is only applied in case all the BP-RNN decoders composing D 10 fail to find a codeword, it follows that the OSD post-processing step yields similar performance when applied to either one of the parallel or serial architecture. We simply refer to the corresponding decoder as D 10 -OSD, without mention of the diversity architecture.

Simulation results are presented in Fig. 8 for (a) Code-1, and (b) Code-2. For Code-1, first we note that the BP-OSD-1 provides better performance than the unspecialized BP-RNN-OSD-1. Using only the first decoder of our BP-RNN diversity (D 1 -OSD-1) outperforms the BP-OSD-1 by about 0.31 dB at FER = 10 -4 . Using all the BP-RNN diversity (D 10 -OSD-1) provides an extra gain of 0.12 dB, i.e., a total gain of about 0.43 dB with respect to BP-OSD-1. Furthermore, we observe that D 10 -OSD-1 virtually achieves the ML decoding performance, where the latter is estimated according to [START_REF] Fossorier | Soft-decision decoding of linear block codes based on ordered statistics[END_REF] (we also note that the OSD-3 decoder provides an accurate approximation of the ML decoding performance). Finally, a gain of 0.52 dB can be observed for D 10 -OSD-0 with respect to BP-OSD-0.

For Code-2, we note that the unspecialized BP-RNN-OSD-1 provides slightly better performance than the BP-OSD-1. D 10 -OSD-w outperforms BP-OSD-w, by 0.32 dB, for w = 0, and 0.72 dB, for w = 1, at FER = 10 -4 . Using only the first decoder of our BP-RNN diversity, we observe that D 1 -OSD-1 outperforms BP-OSD-1 by about 0.11 dB. For comparison purposes, we have also included in Fig. 8b the FER performance of the Pruning Based Neural BP (PB-NBP) decoder from [START_REF] Buchberger | Pruning and quantizing neural belief propagation decoders[END_REF]. Several PB-NBP variants are presented in [START_REF] Buchberger | Pruning and quantizing neural belief propagation decoders[END_REF], we consider here the PB-NBP decoder D 1 (see Fig. 6 in loc. cit.). It can be observed that D 10 -OSD-1 outperforms the PB-NBP decoder by 0.84 dB, our decoder achieving a FER performance at only 0.63 dB from the ML decoding.

To further analyze the diversity obtained by D 10 -OSD-w, we compare it with a decoding diversity combining BP and OSD, where the latter is applied periodically throughout the BP decoding iterations [START_REF] Fossorier | Iterative reliability-based decoding of low-density parity check codes[END_REF]. Precisely, we consider BP(I test = 250), and record the a-posteriori LLR values after every 25 decoding iterations. In case a codeword is not found after 250 decoding iterations (BP fails), we apply one OSD-w on each recorded set of a-posteriori values. Hence, the OSD-w is carried out ten times in case of a decoding failure, as with D 10 -OSD-w. The ML rule [START_REF] Fossorier | Soft-decision decoding of linear block codes based on ordered statistics[END_REF] is used afterwards to determine the final codeword.

We restrict our attention to Code-2 and, in order to further narrow the gap to ML decoding performance, we consider OSD of order w = 1 and w = 2. The corresponding FER results are shown in Fig. 9. We observe that the proposed D 10 -OSD-w performs better than BP(I test = 250) with OSDw every 25 iterations, especially for w = 2, where it achieves a FER performance within 0.2 dB from the ML decoding. Moreover, it should be noticed that using the parallel decoding architecture from Section IV.B, the worst case latency of the D 10 diversity corresponds to 25 decoding iterations, while the worst case latency of BP(I test = 250) is equal to 250 decoding iterations. Finally, we note that D 10 -OSD-2 exhibits a gain of 1 dB with respect to BP(I test = 250)-OSD-2.

VII. CONCLUSION

We addressed the problem of enhancing the BP-RNN performance at short code length, by exploring two complementary approaches: (1) decoding diversity, derived by specializing BP-RNN decoders to specific classes of errors, and (2) reliability-based post-processing. We showed that the proposed BP-RNN diversity coupled with a parallel decoding architecture allows increasing the error correction capability, without increasing the worst-case latency. Moreover, we showed that the proposed OSD post-processing step advantageously leverage the bit-error rate optimization induced by the use of the cross-entropy loss function, as well as the diversity brought by the use of multiple BP-RNN decoders. The proposed approach, combining decoding diversity and low-order OSD post-processing, provides an efficient way to bridge the gap to ML decoding. It also opens new perspectives for the emerging domain of NN-based decoders. Indeed, we believe that new approaches may be considered for the optimization of NNbased decoders, not to deliver the best possible bit or frame error rate performance, but merely an output that best suits the reliability-based post-processing step.

AVAILABILITY OF DATA

Parity check-matrices of Code-1 and Code-2, as well as the complete list of absorbing sets of size ν ≤ 8, are openly available at https://ai4code.projects.labsticc.fr/software/.

APPENDIX EXTENDED COMPLEXITY DISCUSSION

In this appendix, we extend the complexity discussion of Section VI-D. First, we compare BP-RNN and BP in terms of memory requirements and number of operations. Since our BP-RNN decoders use the data-pass layer defined in (4), they can be implemented using conventional BP decoding architectures. In this case, the extra memory cost for a single BP-RNN decoder with respect to the BP decoder comes from the weights storage only, and the BP-RNN decoder also requires an extra multiplication for each weight in (3) and (4). Since the check-node updates are the most computationally intensive operations in BP decoding, counting their number is also relevant to evaluate the complexity of a BPbased decoder [START_REF] Smith | Design of irregular LDPC codes with optimized performance-complexity tradeoff[END_REF]. In [START_REF] Buchberger | Pruning and quantizing neural belief propagation decoders[END_REF], the worst case complexity of the proposed PB-NBP decoder is evaluated by counting the number of check-node updates when all decoding iterations are performed. In our case, the number of check-node updates is simply given by the number of edges of the Tanner graph, multiplied by the number of decoding iterations. We can then evaluate the worst-case decoding complexity (corresponding to the maximum number of decoding iterations), the best-case complexity (corresponding to one decoding iteration only), and the average-case complexity (corresponding to the average number of decoding iterations). The corresponding results for the BP, the PB-NBP D 1 [START_REF] Buchberger | Pruning and quantizing neural belief propagation decoders[END_REF], and the serial D 10 decoders are given in Table IV. In the worst-case scenario, we observe that the serial D 10 has the same decoding complexity as BP(I test = 250), which is 4.85 times higher than that of PB-NBP D 1 . In the best-case scenario, the PB-NBP D 1 decoder is penalized by the fact that it uses a high number of check-nodes during the first iteration (some of which may be punctured during subsequent iterations). Precisely, its first iteration comprises 640 check nodes of degree at least 8 (see [START_REF] Buchberger | Pruning and quantizing neural belief propagation decoders[END_REF]Fig. 4b]), giving a number of check-node updates greater than or equal to 5120. This amounts to 10 times the best-case decoding complexity (or equivalently, the complexity of 10 decoding iterations) of the serial D 10 or the BP decoder. In Fig. 7c, we notice that the average number of decoding iterations Ī is equal to 10 for an SNR = 3.37 dB for both BP and serial D 10 . As a result, we conclude the serial D 10 or BP is less complex than PB-NBP D 1 in the waterfall region. To further illustrate this, we observe in Table IV that the serial D 10 decoder has an average decoding complexity at SNR = 4.5 dB that is 5 times lower than the best-case decoding complexity of PB-NBP D 1 . Finally, we note that PB-NBP D 1 has 2.75 times more weights than D 10 , due to its neural network architecture, requiring therefore a higher weight storage cost.

Fig. 1 .

 1 Fig. 1. Examples of absorbing sets of type 4-(2, 5)

Fig. 2 .

 2 Fig. 2. Weight profiles of various BP-RNN decoders (Code-1, SNR = 5 dB)

  σ 2 , a, b) denotes the truncated normal distribution with mean 0 and variance σ 2 , taking values in the interval (a, b). Clearly, we have E(y) = A. The training set is obtained by repeating the above procedure (including the random choice of A) multiple times. This way, the training set is representative of the error class (identified by the extended type of the absorbing error set), and thus the trained BP-RNN decoder becomes specialized in decoding the errors of the class.

Fig. 3 .

 3 Fig. 3. BP-RNN diversity decoding architectures

Fig. 4 .

 4 Fig. 4. CDF of the a posteriori LLR values when decoding fails (Code-1, SNR = 4 dB, maximum number of decoding iterations Itest = 25)

Fig. 5 .Fig. 6 .

 56 Fig. 5. Impact of training parameter I train on the FER performance, for BP-RNN decoders using Itest = 25 (Code-1)

2 Fig. 7 .

 27 Fig. 7. FER results, complexity and decoding latency evaluations (without OSD post-processing)

Fig. 8 .

 8 Fig. 8. FER results using OSD post-processing

Fig. 9 .

 9 Fig. 9. FER results using OSD post-processing of order w = 1 and w = 2 for Code-2

  , . . . , A ℓ ) ← set of possible completions for all A ℓ+1 in C(A 0 , . . . , A ℓ ) do AS DFS(H, ℓ + 1, A 0 , . . . , A ℓ , A ℓ+1 ) // recursive call end for

	the list of absorbing sets
	end if
	return // backtrack
	end if
	C(A 0

TABLE IV DECODING

 IV COMPLEXITY (IN TERMS OF THE NUMBER OF CHECK-NODE UPDATES) FOR CODE-2 Average-case decoding complexity is evaluated at SNR = 4.5 dB.

	Decoding architecture	Decoding complexity (number of check-node updates [45]) Worst-case Best-case Average-case ( * )	Number of weights
	BP(Itest = 25)	12800	512	983	0
	BP(Itest = 250)	128000	512	1044	0
	PB-NBP D 1 [16]	25920	5120	N/A	28416
	Serial D 10	128000	512	1044	10240
	( * )

The distance between two nodes is the length (i.e., number of edges) of a shortest path connecting them.

Note that different weight profiles indicate different sets of weight values, while similar weight profiles indicate similar sets of weight values. However, in the latter case, similar weights may apply to different edges.

We take off extended types for which ω = 0, as they correspond to the support of non-zero codewords. Such errors cannot be detected or corrected.

As a matter of fact, the three BP-RNN decoders in Fig.2, specialized on absorbing set error classes, correspond to D Z=3 .

For simplicity, we assume here that the parity check-matrix is of rank M .

Taking into account that column swaps may be needed, in case the M least reliable columns of H are not linear independent.

Note that the total number of error classes (ET-Number) in Table I is 55, three of which correspond to the support of non-zero codewords of size either ν = 6 or ν = 8, for which training is not performed.

See also TableIIIand the corresponding discussion from Section VI-C.


Joachim Rosseel
email: joachim.rosseel@cea.fr

Valérian Mannoni
email: valerian.mannoni@cea.fr

Valentin Savin
email: valentin.savin@cea.fr

Inbar Fijalkow
email: inbar.fijalkow@ensea.fr

Error Structure Aware Parallel BP-RNN Decoders for Short LDPC Codes

come   L'archive ouverte pluridisciplinaire

I. INTRODUCTION

Short-packet machine-to-machine communications, central to the emerging Internet of Things (IoT) technology, have revitalized interest in research and practice of efficient error correcting codes, for messages ranging from a few tens up to a few hundred bits. While important progress has been made over the last years in understanding the limits of coding at short block lengths [START_REF] Polyanskiy | Channel coding rate in the finite blocklength regime[END_REF], the design of efficient short codes and decoding algorithms still raises many challenges [START_REF] Cos ¸kun | Efficient error-correcting codes in the short blocklength regime[END_REF].

Low Density Parity Check (LDPC) codes [START_REF] Gallager | Low density parity check codes[END_REF] are a class of error correcting codes defined by sparse bipartite graphs [START_REF] Tanner | A recursive approach to low complexity codes[END_REF]. They are well-known for their excellent error correction performance at long block lengths, achieving near Shannon channel capacity performance under Belief Propagation (BP) decoding, in the asymptotic limit of the code length [START_REF] Richardson | Design of capacityapproaching irregular low-density parity-check codes[END_REF]. For codes defined by cycle-free bipartite graphs, BP decoding outputs the maximum a posteriori estimates of the coded bits [START_REF] Wiberg | Codes and decoding on general graphs[END_REF]. Although bipartite graphs associated with practical codes contain cycles, BP decoding may still take effective advantage of sparse, long enough graphs, devoid of short cycles. However, for short codes, short cycles may not be avoidable, thus significantly degrading the BP performance. This is even more pronounced for High Density Parity Check (HDPC) codes, defined by higher density bipartite graphs [START_REF] Dimnik | Improved random redundant iterative HDPC decoding[END_REF].

To reduce the impact of short cycles, a weighted BP decoding has been introduced in [START_REF] Nachmani | Deep learning methods for improved decoding of linear codes[END_REF], where the weights are optimized using a Neural Network (NN). The topology of the NN mimics the BP decoding process, with unwrapped decoding iterations. The approach may use either a feedforward (FF) This work was partially supported by the ECSEL Joint Undertaking (JU) programme, under grant number N°826276 (CPS4EU project).

or a Recurrent NN (RNN). The corresponding decoders are termed as BP-FF and BP-RNN. It has been shown in [START_REF] Nachmani | Deep learning methods for improved decoding of linear codes[END_REF] that the BP-RNN is able to outperform the usual BP decoder for short Bose-Chaudhuri-Hocquenghem (BCH) codes, belonging to the class of HDPC codes. Subsequently, several variants of NN-based BP decoding have been proposed in the literature. [START_REF] Xiao | Designing finite alphabet iterative decoders of ldpc codes via recurrent quantized neural networks[END_REF] proposed the design of new decoding rules for finitealphabet iterative decoders, based on a quantized NN model. [START_REF] Buchberger | Pruning neural belief propagation decoders[END_REF] developed a pruning method of irrelevant check nodes in a neural BP model, aimed at jointly optimizing the code construction and the decoding. In [START_REF] Doan | Neural belief propagation decoding of crc-polar concatenated codes[END_REF], a neural BP decoding approach was proposed for cyclic redundancy check (CRC)assisted polar codes.

In this paper, we focus on BP-RNN decoding of short block length LDPC codes. To improve the decoding performance, our approach aims at specializing BP-RNN decoders to difficult error events. To do so, we first propose a classification of the error events, according to the structure of the induced subgraph. The classification is driven by the impact of the induced sub-graph on the BP decoding performance. Then, a parallel construction, comprising several BP-RNN decoders running in parallel, is described, where each BP-RNN is specialized to a specific error class. Finally, we discuss the training of the parallel BP-RNN decoders, and provide a method to reduce their number, without jeopardizing the decoding performance, by introducing a similarity rate metric.

The paper is organized as follow. Section II introduces the notations and recalls the BP-RNN decoding algorithm. Section III defines the classification of the error events, the parallel construction of BP-RNN decoders, the training of the BP-RNN decoders, and the similarity metric used to reduce their number. Finally, Section IV presents the numerical results, and Section V concludes the paper.

II. NEURAL BP DECODING

We consider an LDPC code defined by a Tanner (bipartite) graph with N variable-nodes and M check-nodes, denoted respectively by n ∈ {1, . . . , N } and m ∈ {1, . . . , M }. We further denote by N (m) the set of variable-nodes connected to a check-node m, and by M(n) the set of check-nodes connected to a variable-node n.

BP decoding consists of an iterative exchange of messages along the edges of the Tanner graph, where each message 978-1-6654-0943-8/21/$31.00 ©2021 IEEE provides an estimation of the incident variable-node. BP-RNN and BP-FF decoding algorithms are weighted variants of the BP decoding, where exchanged messages are multiplied by weights learned through an either RNN or FF-NN approach. The underlying NN contains three types of neural layers, each one corresponding to a step of the BP algorithm. The checkpass layer and the data-pass layer carry out the computation of messages outgoing from check-nodes and variable-nodes, respectively. Each one of them contains a number of neurons equal to the number of edges of the Tanner graph. In addition, the a posteriori Log Likelihood Ratio (LLR) layer consists of N neurons, computing the a posteriori LLR values of the N variable-nodes. The three layers of the NN are connected such that a check pass layer, followed by a data pass layer and an a posteriori LLR layer model one iteration of the BP decoding. In particular, it is worth stressing out the differences between the edges of the Tanner graph (corresponding to neurons in the check-pass and data-pass layers), and the edges of the NN.

The formulas below detail the calculation of messages within each layer. We denote by β and α n→m the messages computed by the check-pass and data-pass layers (where (m, n) is an edge of the Tanner graph), and by Ln the messages computed by the a posteriori LLR layer. The observed (channel) LLR values are denoted by L ch,n , and are used to initialize α n→m messages prior to the first iteration.

β m→n = 2 tanh -1   n ∈N (m)\n tanh α n →m 2   (1)
α n→m = L ch,n + m ∈M(n)\m w m →n→m β m →n (2)
Ln = L ch,n + m∈M(n) wm→n β m→n (3)
It can be observed that weights are applied only on the NN edges incoming to the data-pass (2) and a posteriori LLR (3) layers. Each weight corresponds to one specific edge of the NN. In (2) the weights are denoted by w m →n→m , where the subscript indicates both the corresponding neuron n → m in the data-pass layer, and the incoming NN edge from neuron m → n in the check-pass layer. In (3) the weights are denoted by wm→n , where the subscript indicates the corresponding neuron n in the a posteriori LLR layer, and the incoming NN edge from neuron m → n in the check-pass layer. For the BP-RNN, the weights only depend on the corresponding edges of the NN, while for the BP-FF they also depend on the iteration number (note that, to simplify notation, we have not indicated the iteration number on the above formulas). It is worth noticing that despite the reduced number of trained weights, the BP-RNN achieves similar performance to the BP-FF [START_REF] Nachmani | Deep learning methods for improved decoding of linear codes[END_REF]. An alternative approach suggested in [START_REF] Nachmani | Deep learning methods for improved decoding of linear codes[END_REF] to further reduce the number of weights is based on the following datapass layer,

α n→m = L ch,n + w n→m m ∈M(n)\m β m →n , (4)
where the applied weight only depends on the data-pass neuron. This simplification reduces the training complexity and makes it possible to reuse conventional BP decoding architectures for efficient hardware implementation. The BP-RNN using (4) will be referred to as BP-RNN Hardware Friendly Implementation (BP-RNN-HFI).

To train the BP-RNN, we use the following Bit Error Rate (BER) loss function, assuming without loss of generality that the zero codeword is transmitted:

Loss( L) = -1 N N -1 n=0 log(σ( Ln )) (5)
where σ(x) = (1 + exp(-x)) -1 is the sigmoid function, converting the LLRs into probability values. The loss function is minimized during the NN training, thus improving the BER of the trained decoder.

III. SPECIALIZING BP-RNN DECODERS ACCORDING TO AN ERROR EVENTS CLASSIFICATION A. Absorbing-type classification of error events

Let V be a set of variable-nodes, and C be the set of check-nodes connected to at least one variable-node in V . We denote by O(V ) ⊂ C the set of check-nodes connected an odd number of times to V (that is, they have odd degree in the sub-graph induced by V ). Thus, E(V ) := C\O(V ) is the set of check-nodes connected an even number of times to V . The set V is said to be an absorbing set [START_REF] Dolecek | Predicting error floors of structured ldpc codes: Deterministic bounds and estimates[END_REF], if each variable-node in V has fewer neighbors in O(V ) than in E(V ). Fig. 1(a) shows an example of absorbing set, where each variable-node of V is connected to one check-node in O(V ) and two check-nodes in E(V ). While absorbing sets are combinatorial substructures of the Tanner graph, defined independently of the particular decoding algorithm, they are known to be particularly harmful to BP or other forms of message-passing decoding. Indeed, assuming that the set of errors V is an absorbing set, then each variable-node in V has less neighbor check-nodes indicating an error (unsatisfied), than indicating no error (satisfied). Consequently, V represents a difficult error event, yielding a decoding failure with high probability.

We are interested in classifying error events with a given number of errors ν. Let V a set of variable-nodes, with card(V ) = ν. We define the absorbing type of V as the pair (ω, ε), where ω := card(O(V )) and ε := card(E(V )). We shall sometimes denote the absorbing type as ν-(ω, ε), to also account for the cardinality of V . Note that the absorbing type indicates the total number of (unsatisfied, satisfied) checknodes, in case the variable-nodes in V are in error. However, variable-node sets of same absorbing type may induce different (precisely, non-isomorphic) sub-graphs. Such an example is illustrated in Fig. 1, for two variable-node sets of absorbing type 3-(3, 3), the first of which is an absorbing set (a), but not the second (b). For the set V in (b), it can be seen that variable-node n is connected to two unsatisfied checknodes and one satisfied (most favorable case among the three variable-nodes), n to only satisfied check-nodes (worst case),

B. Proposed parallel BP-RNN decoders

To improve the decoding performance of LDPC codes at short coding length, we propose to specialize (i.e., train) a BP-RNN decoder for each error class ν-(ω, ε, s), according to the classification from the previous subsection. The number of different error classes, and thus of BP-RNN decoders, depend on the particular Tanner graph defining the LDPC code, and the value of ν. For a given short LDPC codes, we determine all the possible error classes, by considering all the variablenode subsets V of cardinality ν. In this work, we consider ν = 2, 3, thus the proposed approach is particularly relevant to high coding rate LDPC codes, correcting a small number of errors.

Once a BN-RNN decoder has been trained for each error class (the training procedure will be detailed in next subsection), we consider a parallel decoding architecture, where all the trained decoders are run in parallel (note that different architectural choices are possible and not discussed in this paper). We include the conventional BP decoder in the parallel structure. If none of the parallel decoders outputs a codeword (which is verified by computing the syndrome), decoding fails. Otherwise, among the decoded codewords, we select the one that has been outputted the most often (in case different decoders output different codewords). Decoding is successful if the selected codeword is equal to the transmitted one.

C. Training of the parallel BP-RNN decoders

We propose in this section a construction of the training set, used to train the BP-RNN decoder for a particular error class. We assume that coded bits are mapped to ±1 modulated symbols, which undergo real additive white Gaussian noise (AWGN). Since both the noise model and the BP-RNN decoder are symmetric [START_REF] Nachmani | Deep learning methods for improved decoding of linear codes[END_REF], we may assume the all-zero codeword is transmitted, corresponding to an all +1 modulated signal. Hence, under the AWGN model, received symbols are given by y n = 1+z n , n = 1, . . . , N , where z n denotes a real-valued normal distributed random variable, with mean 0, and variance σ 2 .

To generate a random error event in a given error class ν-(ω, ε, s), we first consider an underlying variable-node error set V , randomly chosen from those corresponding to the given error class, and then generate received symbols y n , by

y n = 1 + z n , ∀n = 1, . . . , N (6)
where

z n ∼ N (0, σ 2 , -∞, -1), if n ∈ V N (0, σ 2 , -1, ∞), otherwise (7)
where N (0, σ 2 , a, b) denotes the truncated normal distribution with mean 0 and variance σ 2 , taking values in the interval (a, b). The training set is obtained by repeating the above procedure multiple times, for each variable-node set V in the given error class. In this way, the training set is representative of the error class, and thus the trained BP-RNN decoder becomes specialized to error events in the class.

D. Complementary selection of trained BP-RNNs

In practical applications, it is desirable to reduce the number of decoders running in parallel. To this end, in this subsection we propose a complementarity metric between the trained decoders. Intuitively, two decoders are complementary if the probability to fail on the same error event is low. This maximizes the gain when the two decoders are run in parallel.

We define the similarity rate between two decoders D p and D q , p = q, as the probability that both decoders fail, when either one of them fails. Precisely, we define S(D p , D q ) := Pr (D p and D q fail | either D p or D q fails) (8)

In practice, this metric can be numerically estimated by Monte-Carlo simulation, using

S(D p , D q ) ≈ N p,q N p + N q -N p,q , (9)
where N p (resp. N q ) is the number of times the decoder D p (resp. D q ) failed, and N p,q is the number times they both failed. Consequently, S(D p , D q ) measures the effectiveness of specialized training in producing decoders able to correct different error events. Put differently, it characterizes the level of complementary between the two decoders. An overall similarity coefficient is then calculated for each decoder D p , by averaging the similarity rate between D p and the other decoders.

S(D

p ) = 1 D -1 q =p S(D p , D q ) (10)
where D denotes the total number of parallel BP-RNN decoders. Subsequently, given a similarity threshold value S th , we keep only the decoders with overall similarity coefficient S(D p ) less than S th . Hence, only the most complementary decoders are maintained in the final parallel structure. We note that the choice of the S th value may yield different trade-offs between complexity and decoding performance.

IV. NUMERICAL RESULTS

A. Simulation Settings

Two LDPC codes with regular variable-node degree have been considered in our simulations. Code parameters are provided in Table I, where N denotes the code length, K the number of information bits, R c := K/N the coding rate, d v the variable nodes degree, d c the check nodes degree, and n 4-cycles the number of length-4 cycles. The Tanner graphs of the two codes have been constructed by using the Progressive Edge Growth (PEG) algorithm [START_REF] Hu | Regular and irregular progressive edge-growth Tanner graphs[END_REF], a greedy algorithm making the best-effort to reduce the number of short cycles in the constructed graph. Yet, for small code length, short cycles, including cycles of length-4, cannot be completely avoided.

The number of decoding iterations was set to ten, for all the decoders. Since we consider codes with high coding rate, the error events classification was conducted for error sets V of size ν = 2, 31 . Applying the error classification procedure described in Section III-A, we found three 2-(ω, ε, s) classes, for both Code-1 and Code-2, and ten (resp. eleven) 3-(ω, ε, s) error classes for Code-1 (resp. Code-2). Thus, a total of thirteen (resp. fourteen) BP-RNNs were used in parallel, alongside the BP decoder. Each BP-RNN was trained independently with the training set construction technique described in Section III-C.

The same procedure was repeated for the BP-RNN-FHI decoder, for both Code-1 and Code-2, in order to assess the weight reduction in (4). In addition, we also trained a single BP-RNN decoder according to the procedure described in [START_REF] Nachmani | Deep learning methods for improved decoding of linear codes[END_REF], to provide a benchmark for our parallel BP-RNNs approach.

To train the BP-RNN decoders, we used the Keras library, with the hyper parameters shown in Table II. All BP-RNNs were trained for each signal-to-noise ratio (SNR) value ranging from 1 dB to 8 dB, with a step of 1 dB, thus providing eight optimized weight sets for each decoder. While this increases Finally, we used the similarity metric introduced in Section III-D, in order to reduce the number of parallel decoders. The similarity coefficient has been numerically estimated by using ( 9) and ( 10), based on the first simulation results. We fixed a similarity threshold value S th = 0.6 (resp. S th = 0.56) for Code-1 (resp. Code-2), at a FER of 10 -4 , which led to the selection of only eight (resp. nine) trained BP-RNN decoder alongside the conventional BP. Then, new independent simulations were run for both resulting parallel structures, to assess their FER performance.

B. FER performance

We compare the different decoding strategies discussed in the previous section, in terms of FER. Reported SNR gains are evaluated at a FER of 10 -4 .

Simulation results for Code-1 are shown in Fig. 2. The BP-RNN trained as in [START_REF] Nachmani | Deep learning methods for improved decoding of linear codes[END_REF] yields an SNR gain of 0.18 dB, with respect to conventional BP. Using the proposed structure, with specialized BP-RNNs, the SNR gain is increased to 0.42 dB. The parallel BP-RNN-FHIs exhibit only negligible performance degradation compared to the parallel BP-RNNs, due to the weight reduction constraint in (4). Finally, it can be observed that the parallel BP-RNNs construction with complementary selection shows virtually the same FER performance as the original complete construction of parallel BP-RNNs. Therefore, choosing S th = 0.6 leads to a selection of eight decoders which effectively complement each others.

Simulation results for Code-2 are shown in Fig. 3. Despite the increase in the length of the code, it should be noted that Code-2 exhibits an increased number of length-4 cycles, due to its higher coding rate. However the parallel BP-RNN decoder yields a similar SNR gain, of 0.42 dB, with respect to the conventional BP. The parallel BP-RNN-FHIs tend to be a little less efficient. Furthermore, Fig. 3 also corroborates the relevance of the complementary selection, the corresponding parallel BP-RNNs construction yielding again almost the same performance as the original complete construction of parallel BP-RNNs.

Finally, Fig. 4 provides a comparison in terms of the number of decoding failures, for various values of ν (size of the error event), for an SNR of 8 dB. It demonstrates that for both Code-1 and Code-2, the parallel BP-RNNs are clearly able to correct numerous error events which are not decoded by the BP, especially for the two and three error events (ν = 2, 3). Therefore, we conclude that the specialization of the training for the 2-(ω, ε, s) and the 3-(ω, ε, s) error classes effectively induces an understanding of the BP-RNNs of how to decode several type of size two and three error events. Furthermore, some error events of size four and five are also successfully decoded thanks to the previous specialization.

V. CONCLUSION AND PERSPECTIVES

In this paper, we addressed the problem of enhancing the BP-RNN performance at short coding length. To this end, we studied and classified error events according to the impact of the induced sub-graphs on the BP decoding performance. Then, we proposed a new decoding strategy consisting of parallel specialized BP-RNN decoders where each BP-RNN was trained for a specific error class. In addition, we introduced a complementary selection method of the trained BP-RNN decoders, proven to be efficient in keeping the most relevant trained decoders in the final parallel structure.

This work is a first step towards a framework of specialized neural BP decoders, and we believe that further work may reveal alternative specialization strategies. The final aim would be to approach maximum likelihood decoding performance at short to moderate code-length, for which we will probably need to rely on a bunch of practical decoders, rather than a unique one.

Fig. 1 .

 1 Fig. 1. Example of two sets V , with card(O(V )) = card(E(V )) = 3. If variable-nodes in V are in error, check-nodes marked by an U are unsatisfied, while those marked by an S are satisfied.

Fig. 2 .

 2 Fig. 2. FER results for Code-1.

Fig. 3 .

 3 Fig. 3. FER results for Code-2.

Fig. 4 .

 4 Fig. 4. Number of failed decoding according to the size of an error event.

  Error Structure Aware Parallel BP-RNN Decoders for Short LDPC Codes Joachim Rosseel * † , Valérian Mannoni * , Valentin Savin * , Inbar Fijalkow †

For comparison, a BCH code with either (N, K) = (63, 45) or (N, K) = (127, 106) has minimum distance d = 7, thus may correct 3 errors