Sebastián Basterrech
email: sebastian.basterrech@vsb.cz

Gerardo Rubino
email: gerardo.rubino@inria.fr

Evolving Reservoir Weights in the Frequency Domain

Keywords: Computing methodologies → Neural networks, Bio-inspired approaches, Genetic algorithms, Neuroevolution, Recurrent Neural Networks, Echo State Network, Genetic Algorithms, Reservoir Computing

Reservoir Computing models are a class of recurrent neural networks that have enjoyed recent attention, in particular, their main family, Echo State Networks (ESNs). These models have a large number of hidden-hidden weights (in the so-called reservoir) forming a recurrent topology. The reservoir is randomly connected with fixed weights during learning: only readout parameters (from reservoir to output neurons) are trained; the reservoir weights are frozen after initialized. Since the reservoir structure is fixed during learning, only its initialization process has an impact on the model's performance.

In this work, we introduce an evolutionary method for adjusting the reservoir non-null weights. Moreover, the evolutionary process runs on the frequency space corresponding to a Fourier transformation of the weights. We combine an evolutionary search in the Fourier space with supervised learning for the readout weights. The resulting algorithm, called EvoESN (Evolutionary ESN), obtains promising results modeling two well-known problems of the chaotic time-series area.

INTRODUCTION

A Recurrent Neural Network (RNN) is a dynamical system with the ability of conveying information across time. The recurrences ensure that historical information from the input data can be stored in internal hidden states. In spite of the good properties and encouraging experimental success, RNNs are complex and difficult to Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s). GECCO '21 Companion, July 10-14, 2021, Lille, France © 2021 Copyright held by the owner/author(s). ACM ISBN 978-1-4503-8351-6/21/07. https://doi.org/10.1145/3449726.3459457 properly train, especially when the network is large and there are long-term data dependencies [? ].

During the last 20 years, a type of RNNs named Reservoir Computing (RC) has captured considerable attention, due to its outstanding results for modeling chaotic systems and solving real-world problems [? ]. A pioneer RC model is the Echo State Network (ESN) [? ]. The model uses a recurrent hidden-to-hidden structure (the reservoir) to memorize patterns of the input history. The reservoir is driven by the input signal, and projects the input into a high-dimensional space. The reservoir weights are realizations of i.i.d. random variables. To guarantee specific stability conditions, the random initialised weights are scaled in order to keep control over the spectral radius of 𝑊 r matrix [? ? ]. The training algorithm adjusts only the rest of the model parameters, which composes a fast adapting readout layer. The dynamics of the model is described in terms of a discrete-time system with an input signal 𝑢 (𝑡) ∈ R 𝐾 and a hidden state 𝑥 (𝑡) ∈ R 𝑁 . The recurrent state 𝑥 (𝑡) and the model's output 𝑦 (𝑡) are computed according to the following expressions [? ]: 𝑥 (𝑡) = 𝑓 𝑊 in 𝑢 (𝑡) +𝑊 r 𝑥 (𝑡 -1) +𝑊 fb 𝑦 (𝑡) +𝜀 (𝑡) , and

𝑦 (𝑡) = 𝑔(𝑊 out [𝑢 (𝑡); 𝑥 (𝑡)]
), where 𝑡 is the discrete time index, 𝑊 in is a matrix with input-to-reservoir weights, 𝑊 r has the reservoir internal weights, 𝑊 out is the reservoir-to-output weight matrix, 𝑊 fb is the matrix with feedback connections and 𝜀 (𝑡) is a noise vector. The [•; •] operation denotes vector concatenation. Functions 𝑓 (•) and 𝑔(•) are predefined coordinate-wise functions. Matrix 𝑊 fb and vector 𝜀 (𝑡) are optional (they are set to zero in the case of standard ESN).

Since the large recurrent structure is fixed during the training, the RC family has also received some critics [? ]. In this work, we present an evolutionary computational model for creating efficient reservoirs called EVOlutionary Echo State Network (EvoESN). We combine ideas regarding indirect encoding of the weight connections [? ], EVOLINO [? ], and the efficiency of ESNs [? ]. In the proposed model, the reservoir has a fixed pattern of connectivity (as in standard ESNs), and we apply the evolutionary process to find good weight connections. The evaluation of the proposed computational model was made over two well-known benchmark problems: Mackey Glass system and Lorenz system [? ? ? ].

EVOLUTIONARY ECHO STATE NETWORK

Let us denote by 𝑀 the number of non-null weights in 𝑊 r . Since reservoirs are designed to be sparse for efficiency reasons, then 𝑀 ≪ 𝑁 2 [? ]. We arbitrary design the pattern of connectivity of the reservoir, i.e. we choose 𝑀 positions in 𝑊 r with non-zeros values. Let's denote by 𝑝 the positions of those non-null coefficients, a vector having size 𝑀. By 𝑝 𝑘 = (𝑖, 𝑗) we mean that the 𝑘th position in 𝑝 refers to the weight in row 𝑖 and column 𝑗 of 𝑊 r . We also use a vector 𝜈 having dimension 𝑀, containing those weights, that is, 𝜈 𝑘 = 𝑊 r 𝑝 𝑘 = 𝑊 r 𝑖,𝑗 . During the procedures described below, 𝑝 (the positions) remains fixed and 𝜈 (the values) evolves. In order to obtain a good reservoir, we move vector 𝜈 to the frequency domain using the DCT transform. Then, we change it running a Genetic Algorithm (GA) and we go back to weights through the inverse DCT. We evaluate the introduced changes by training the ESN with the just obtained reservoir. This process is repeated until some convergence condition is satisfied.

Let us denote 𝛼 = (𝛼 1 , . . . , 𝛼 𝐶 ) the vector with the first 𝐶 DCT coefficients, 𝐶 ≤ 𝑀, that is randomly initialized. Denote now by 𝜙 (•) the inverse DCT transformation. In case 𝐶 < 𝑀, we extend 𝛼 with zeros (padding) until dimension 𝑀 is reached, in order to make 𝜙 (𝛼) a vector with dimension 𝑀. Once we have computed 𝛼 (the chromosome) and 𝜈 = 𝜙 (𝛼) (the phenotype), the ESN is trained using a classic technique [? ]. The whole procedure is repeated until some appropriate convergence condition is satisfied. The algorithm can then be summarized as follows: (i) Choose 𝑀, 𝐶 and initialize 𝛼.

EXPERIMENTAL RESULTS

We evaluate the performance of the proposed model on well-known benchmark problems. Due to constrains in space, we report only some results obtained over Mackey-Glass and Lorenz systems. In both benchmark problems, we adopted the experimental setting well-described in [? ], i.e. we applied the same network architectures and learning configurations. The inverse DCT was made with the orthogonal norm using the fftpack Python package. The evolutionary search was conducted employing GAs, and it was implemented using the DEAP library [? ]. Table ?? shows the obtained results for Lorenz task. We show the NRMSE 84 error which is a standard measure for MGS and also used in Lorenz task [? ? ]. The error has same order of magnitude than that obtained by Jaeger et al. using a feedback connection and a refined learning method [? ]. We present the results for EvoESN when the number 𝐶 of coefficients in the frequency representation has values 50, 100 and 150. In addition, we present the achieved error when the model predicts a 600 time horizon (600H). Figure ?? illustrates the power of the proposed approach. The red curves (errors obtained by EvoESN with 500 coefficients) correspond to the evolution of the errors against the generations, for different independent experiments on the MGS problem. In horizontal lines we show Evolino [? ] and ESN with Feedback connections and refined learning method (ESN-FB II) [? ]. After a relatively few number of generations, it is possible to reach the best registered performance for solving the MG task. We expect to do further studies using statistical analysis and a full evaluation of the GA parameters, in order to explore the differences between ESN with feedback connections and EvoESN on the MSG problem.

CONCLUSIONS

This work introduces new insights in the Reservoir Computing (RC) paradigm. We present a new computational model entitled EvoESN. We added to the standard RC approach an evolutionary procedure for finding a good set of reservoir weights. The proposed method projects the reservoir weights into the frequency domain using the DCT. Then, the reservoir weights are found in the Fourier space using an evolutionary algorithm. Promising results have been achieved modeling two popular chaotic systems.

  (ii) Extend 𝛼 with zeros until dimension 𝑀 is reached. (iii) Compute 𝜈 = 𝜙 (𝛼). (iv) Store the values in 𝜈 to their positions 𝑝 in the reservoir. (v) Use some standard training schema for adjusting the ESN readout weights. (vi) Compute a fitness function using the trained ESN and testing data. (vii) If the convergence condition is not satisfied, apply the evolution operations (run the GA), compute a new vector 𝛼 and repeat from (ii).

Figure 1 :

 1 Figure 1: NRMSE 84 (log 10 scale) of the MGS prediction.

Table 1 :

 1 Results for Lorenz system prediction.

	Model	Coeff. Error	Metric
	EvoESN 50	-3.4666 𝑙𝑜𝑔 10 NRMSE 84
	EvoESN 100	-3.6014 𝑙𝑜𝑔 10 NRMSE 84
	EvoESN 150	-4.1379 𝑙𝑜𝑔 10 NRMSE 84
	EvoESN 50	-0.2704 𝑙𝑜𝑔 10 absolute error (600H)
	EvoESN 150	-0.1407 𝑙𝑜𝑔 10 absolute error (600H)

ACKNOWLEDGEMENTS

This work was supported by the GACR-Czech Science Foundation project no. 21-33574K "Lifelong Machine Learning on Data Streams".