Thomas Hecht

Mathieu Lefort

Alexander Gepperth

Using self-organizing maps for regression: the importance of the output function

Self-organizing map (SOM) is a powerful paradigm that is extensively applied for clustering and visualization purpose. It is also used for regression learning, especially in robotics, thanks to its ability to provide a topological projection of high dimensional non linear data. In this case, data extracted from the SOM are usually restricted to the best matching unit (BMU), which is the usual way to use SOM for classification, where class labels are attached to individual neurons. In this article, we investigate the influence of considering more information from the SOM than just the BMU when performing regression. For this purpose, we quantitatively study several output functions for the SOM, when using these data as input of a linear regression, and find that the use of additional activities to the BMU can strongly improve regression performance. Thus, we propose an unified and generic framework that embraces a large spectrum of models from the traditional way to use SOM, with the best matching unit as output, to models related to the radial basis function network paradigm, when using local receptive field as output.

Introduction

Self-organizing map (SOM), especially the well known Kohonen model [START_REF] Kohonen | Self-organized formation of topologically correct feature maps[END_REF], is a powerful paradigm that is traditionally used for clustering, visualization and data analysis in various domains [START_REF] Flexer | On the use of self-organizing maps for clustering and visualization[END_REF][START_REF] Kohonen | Essentials of the self-organizing map[END_REF], where each unit is usually associated with some label. It has also been used as an associative memory of motor and visual data to learn direct and inverse models for robotic control. These models exploit the SOM ability to cope with non linear redundant data in an adaptive way and to be well suited for planning [START_REF] De | Self-organizing feature maps for modeling and control of robotic manipulators[END_REF][START_REF] Hülsen | Self-organising locally interpolating maps in control engineering[END_REF]. The best matching unit (BMU) alone is often used for computing the motor command to send in response to a visual goal. In order to improve precision, some authors propose to extend the Kohonen model by considering the motor commands associated to units close to the BMU with various interpolation methods (see [START_REF] De | Self-organizing feature maps for modeling and control of robotic manipulators[END_REF][START_REF] Hülsen | Self-organising locally interpolating maps in control engineering[END_REF] for a review). Similar methods are also applied to time series prediction [START_REF] De | Time series prediction with the self-organizing map: A review[END_REF].

In addition to its use as an associative memory, the SOM paradigm is also used in various feedforward and recurrent architectures to represent one data flow before combining this representation with, for example, data from other modalities [START_REF] Mathieu Lefort | Somma: Cortically inspired paradigms for multimodal processing[END_REF] or from previous time steps [START_REF] Voegtlin | Recursive self-organizing maps[END_REF]. This separated and combined multimodal processing allows the system to have online and adaptability properties which are highly desirable, especially in robotics [START_REF] Gepperth | Efficient online bootstrapping of sensory representations[END_REF][START_REF] Lefort | Propre: Projection and prediction for multimodal correlations learning. an application to pedestrians visual data discrimination[END_REF]. In most cases, the BMU position is used as the relevant SOM output (see [START_REF] Schillaci | Online learning of visuo-motor coordination in a humanoid robot. a biologically inspired model[END_REF][START_REF] Jantvik | Papliński. A self-organized artificial neural network architecture for sensory integration with applications to letter-phoneme integration[END_REF] for example), even if a limited number of models proposes alternative functions [START_REF] Mathieu Lefort | Somma: Cortically inspired paradigms for multimodal processing[END_REF][START_REF] Voegtlin | Recursive self-organizing maps[END_REF][START_REF] Farkaš | Systematicity in sentence processing with a recursive self-organizing neural network[END_REF][START_REF] Johnsson | Associative self-organizing map[END_REF][START_REF] Martin | Modeling multisensory enhancement with self-organizing maps[END_REF].

Targeting the use of SOM-based models for online and adaptable regression learning, especially for multimodal and robotic purposes, this article quantitatively investigates the influence of the SOM output function on the system's performance. For that purpose, we use an unified framework combining a SOM which processes the inputs and whose output, after passing through the output function, feeds a linear regression step. In the next section, we introduce our architecture and the tested output functions and in section 3, we present the performance achieved with the different functions on a robotic control task.

Architecture

To study the influence of the output activity of the SOM when using it in an integrated model, we consider the architecture depicted in figure 1 (see next sections for equations and details of each component). This architecture is designed for online supervised learning of an input/output relationship. We already used this processing flow in a more global unsupervised system, with another data flow representation as the target signal, for the purpose of online and adaptable multimodal correlation learning [START_REF] Lefort | Propre: Projection and prediction for multimodal correlations learning. an application to pedestrians visual data discrimination[END_REF]. This article targets to be a step towards the improvement of such models. Fig. 1: Studied architecture that couples a SOM, representing the input with a generative learning procedure, with a learning regression step, learning the relationship between this SOM output representation and a given output target.

Self-organizing map

In this article we use the Kohonen self-organizing map model. It is composed of n × n units organized on a two-dimensional grid lattice. Unit prototypes are updated with the learning rule proposed by Kohonen, with decreasing learning rate and Gaussian neighborhood [START_REF] Kohonen | Self-organized formation of topologically correct feature maps[END_REF]. The map activation a(t) = {a ij (t)} (i,j) is equal for each unit located at position (i, j) in the map to the euclidean distance between its prototype w S ij (t) and the current input x(t):

a ij (t) = ||w S ij (t) -x(t)|| 2.

Output function

The SOM output activity o(t) = {o ij (t)} (i,j) is the activity seen from the other modules of an integrated architecture, here the linear regression. It is defined by applying an output function to the map activation, i.e. for a unit at (i, j):

o ij (t) = φ(a(t), θ ij )
with θ ij the set of (optional) parameters of the function φ. Note that the function φ is the same for all the units, even if its parameters can vary from unit to unit (although we do not make use of this possibility in this article).

We study various possible φ functions and present here three of them, inspired by the literature, one for each of three main types of possible output functions: global (section 2.2.1), local (section 2.2.2) and mixed local/global (section 2.2.3).

Global: Best matching unit (BMU)

The SOM output activity corresponds to the BMU, with coordinates (i * , j * ), which is the usual way to define SOM representation (see [START_REF] Schillaci | Online learning of visuo-motor coordination in a humanoid robot. a biologically inspired model[END_REF][START_REF] Jantvik | Papliński. A self-organized artificial neural network architecture for sensory integration with applications to letter-phoneme integration[END_REF] e.g.):

o ij (t) = φ(a(t)) = 1 if (i, j) = (i * , j * ) 0 otherwise

Local: Gaussian similarity

The output activity of each unit depends only on the activation of this unit and is equal to a Gaussian similarity, as proposed in [START_REF] Voegtlin | Recursive self-organizing maps[END_REF], with σ its variance:

o ij (t) = φ(a(t), σ ) = e -a ij (t) 2 2σ 2
From a biological point of view, this output activity corresponds to overlapping receptive fields in the input space which are similar to what we can observe in visual areas. From a machine learning perspective, the complete proposed architecture corresponds to an online version of the radial basis function network (RBFN) paradigm [START_REF] Buhmann | Radial basis functions[END_REF] with Gaussian basis functions whose centers are placed with the SOM algorithm. RBFN is an universal approximator as proved in [START_REF] Park | Universal approximation using radial-basisfunction networks[END_REF] but to our knowledge, the influence of the algorithm used for placing basis function centers on the model performance was not yet reviewed.

Mixed local/global: Softmax

The output activity takes into account local and global activation. It is equal to a softmax function on a similarity measure (such as in [START_REF] Johnsson | Associative self-organizing map[END_REF]), here chosen Gaussian, with σ the Gaussian variance and p an integer power:

o ij (t) = φ(a(t), σ , p) = e -a ij (t) 2 2σ 2 p max (i ,j ) e - a i j (t) 2 2σ 2 p
Decreasing the softmax power parameter p leads to consider more units in the SOM output and provides a gradual change from a best matching unit representation (p → ∞) to a Gaussian similarity one (p = 1), except for the normalization, closing the gap between the two previously described output functions.

Linear regression

Each output value of the linear regression module, that computes the prediction p(t) = {p k (t)} k made by the whole system, is equal to:

p k (t) = (i,j) w L kij (t)o ij (t)
with w L kij (t) the weight between the unit at position (i, j) in the SOM and the k-th value of the prediction.

For an online architecture, the weights can be updated with an online version of linear regression. In practice, in this article, this linear regression is computed offline because this suppresses the stochasticity of the online version, thus providing a fair comparison between the tested output functions without having to perform a large amount of trials.

Experiments

Protocol

We test the influence of the output function in our architecture on the direct model learning of a simulated robotic arm (see figure 2). 100000 motor commands are generated uniformly in the motor space and current joint angles and corresponding spatial egocentric position of the effector were recorded every 0.5 seconds during the movements leading to 232462 data samples. The model is trained on a randomly chosen subset containing 90% of the data, the other 10% are used for testing its generalization ability.

We first train the 20 × 20 SOM, that we fix after checking that it is unfold, and then perform the offline linear regression on all learning examples with the different output functions. The variance of the Gaussian similarity is fixed to 200 for every output function tested. This value offers a good trade-off between the average system's performance and the spread of activity in the SOM.

Human size -Same joint limits as human arm Joints (in degree) are in Egocentric hand position (in m) is in Fig. 2: Seven degrees of freedom compliant arm (a2 from Meka Robotics).

Results

The system error is computed as the average euclidean distance between the predicted and the real spatial position of the effector over the 25830 testing examples. The average performance over 10 different SOMs, learned with random initialization, depending on the output function used are presented in figure 3. We can clearly observe that the choice of the SOM output function has a high influence on the system's regression performance. Using all activities in the map (Gaussian similarity) leads to a 7 times lower prediction error than considering only the BMU. The softmax provides a large spectrum of performances interpolating between the two other functions when gradually changing the parameter p. This seems reasonable as softmax is close to the two other functions for extremal values of p (see section 2.2.3). However, it can be a worthwhile function for using SOM in an integrated architecture by providing various trade-offs between the system's performance and compression of output information.

BMU

Conclusion and perspectives

In this article, we study how the SOM output activity, when using it in a feedforward architecture, influences the system's regression performance. We tested three SOM output functions, defined on the set of distance to prototypes of all units, in an unified architecture: the best matching unit (which is the traditional way to use SOM inherited from its application to classification problems), Gaussian similarity (leading to a system related to the radial basis function network paradigm) and softmax on Gaussian similarity (providing a spectrum of functions between the two first). Results obtained on the learning of a simulated robotic arm direct model clearly show that considering activities additional to the best matching unit significantly increases the system's regression performance, reducing the error by up to 7 times. These results indicate that the choice of the SOM output activity (which is independent from the prototype learning rule) is definitively an important point when using SOM in a feedforward architecture, which seems unfortunately underestimated in the literature.

This article is a first step towards the use of SOM-based architectures for regression learning, closing the gap between the traditional use of SOM for classification and algorithms from the machine learning field. Of course, our architecture can be used with other functions proposed in the artificial neural network field (see [START_REF] Duch | Survey of neural transfer functions[END_REF] for a review) and a deeper study will be necessary to determine the advantages and drawbacks of each function. Moreover, this work opens interesting perspectives to study how SOM-based algorithms can bring topological and adaptability properties to the machine learning field while keeping reasonable regression performances.

Fig. 3 :

 3 Fig. 3: Average error (in mm) depending on the SOM output function used.

* Thomas Hecht gratefully acknowledges funding support by the "Direction Générale de l'Armement" (DGA) and École Polytechnique.


Alexander Gepperth

Mathieu Lefort

Thomas Hecht

Ursula Körner

Resource-efficient incremental learning in very high dimensions

come

Introduction

Incremental learning remains a challenging issue in machine learning. While it is almost self-evident to biologists that learning should be incremental, the technical realization presents baffling difficulties. First of all, incremental learning is inherently sub-optimal when it comes to optimizing an objective (or loss) function. As one can never assume to have seen all training samples at any single point during training, optimization can only take into account the examples seen up to the present moment. Furthermore, the statistics of input-output relations are usually not homogeneous for any finite dataset, so incremental learning must essentially assume non-stationary input statistics at some time scale, which raises the question of how to fuse already learned aspects of a task, without destroying them, with new ones. The latter issue is a real problem for connectionist models Fig. 1: Schema of the three-layer PROPRE architecture composed of input, induced and output representations. Initially there is a forward transmission step, propagating the information to the top-level of the hierarchy where it is decided whether a correct result was obtained. In case it was not, SOM weights are updated in the feedback step, thus leading to a representation of difficult samples in N .

of learning [START_REF] Goodfellow | An empirical investigation of catastrophic forgeting in gradient-based neural networks[END_REF] and has been termed "catastrophic forgetting", and it is clear that any feasible incremental learning algorithm needs to avoid this issue.

Biological foundations

As biological incremental learning has reached a high degree of perfection, we explicitly investigated the biological literature for hints as how to this might be achieved. Basing ourselves on observations from the basic sensory cortices, we noted that sensory representations seem to be prototypebased, where prototype-sensitive neurons are topologically arranged by similarity [START_REF] Tanaka | Inferotemporal cortex and object vision[END_REF][START_REF] David A Leopold | Norm-based face encoding by single neurons in the monkey inferotemporal cortex[END_REF][START_REF] David A Ross | Not just the norm: Exemplarbased models also predict face aftereffects[END_REF][START_REF] Erickson | Clustering of perirhinal neurons with similar properties following visual experience in adult monkeys[END_REF]. Learning seems to act on these representations in a task-specific way, where more prototypes are allocated to sensory regions where finer discrimination is necessary [START_REF] Daniel B Polley | Perceptual learning directs auditory cortical map reorganization through top-down influences[END_REF], a mechanism which presumably gated learning through acetylcholine release in case of task failures [START_REF] Norman | The nucleus basalis and memory codes: Auditory cortical plasticity and the induction of specific, associative behavioral memory[END_REF][START_REF] Michael | The role of acetylcholine in learning and memory[END_REF]. In particular, learning seems to respect and even generate topological layout of prototypes by changing only a small subset of neural selectivities [START_REF] Rolls | The effect of learning on the face selective responses of neurons in the cortex in the superior temporal sulcus of the monkey[END_REF], namely those neurons who previously best matched the stimuli to be learned [START_REF] Erickson | Clustering of perirhinal neurons with similar properties following visual experience in adult monkeys[END_REF].

Model properties

We propose a three-layer neural model for incremental learning that contains a topologically organized representation of prototypes in its hidden layer (termed "induced representation"), trained by the self-organized map (SOM) algorithm [START_REF] Kohonen | Self-organized formation of topologically correct feature maps[END_REF]. Due to the properties of SOMs, learning is always strictly local in the sense that only the prototypes that are similar to the bestmatching one are adapted, thus avoiding catastrophic forgetting. SOM learning is activated by adverse task performance, which conversely means that learning stops once the task is acquired, thus maintaining long-term stability. Classification is performed by simple linear regression from the hidden layer towards a population-coded target vector after first applying a non-linear transfer function to all hidden layer activities.

Related work

Incremental learning algorithms are especially interesting for robotics applications [START_REF] Sigaud | On-line regression algorithms for learning mechanical models of robots: a survey[END_REF], and in fact several very interesting proposals have already been made in this context [START_REF] Sigaud | On-line regression algorithms for learning mechanical models of robots: a survey[END_REF]. An especially popular algorithm in robotics is LWPR [START_REF] Vijayakumar | Incremental online learning in high dimensions[END_REF], which partitions the input spaces into receptive fields (RFs), volumes that are defined by a centroid and a covariance matrix, to which separate linear models are applied. Many other incremental algorithms, reviewed in [START_REF] Sigaud | On-line regression algorithms for learning mechanical models of robots: a survey[END_REF] partition the input space in a similar way and thus will presumably run into memory problems when input dimensionality is high, as LWPR does.

Contribution of this article

In this article, we will propose a model for incremental learning that can cope with scenarios where KM ∼ 10000 (K, M denoting input.output dimensionality) and beyond, and evaluate its performance on the well-known MNIST benchmark [START_REF] Lecun | Gradient-based learning applied to document recognition[END_REF]. We perform a comparison to LWPR that explicitly evaluates the incremental aspect of learning by training the algorithms on a subset of classes and subsequently adding the remaining classes.

Methods

The PROPRE architecture

PROPRE is an architecture composed of different algorithmic modules, rather than an algorithm in itself. One PROPRE iteration consists of the following steps, as described in [START_REF] Gepperth | Efficient online bootstrapping of representations[END_REF], where only the computation of the predictability measure λ is changed to represent the current binary classification error: input: new data is fed into the input representation I and provided to the SOM, and a new target representation T is provided projection: activity is formed in the induced representation N (see Fig. 1) by projection of I onto the SOM prototypes prediction: based on activity in N , a linear regression step is performed to produce representation P which predicts class membership evaluation: a mismatch measure is computed between P and T update: linear regression weights are updated. SOM weights are updated only if mismatch was detected. In mathematical terms, the whole model is governed by the following equations, where we denote neural activity at position y = (a, b) in a 2D representation X by z X ( y, t) and weight matrices for SOM and LR, represented by their line vectors attached to target position y = (a, b) by w SOM y :

z N ( y, t) = w SOM y (t) • z I (t) (1)
z P ( y, t) = w LR y (t) • z I (t) (2)
λ(t) = 0 if argmax y z P ( y, t) = argmax y z T ( y, t), 1 else (3)
w LR y (t + 1) = w LR y (t) + 2ǫ LR z I (t) z P (t) -z T (t) (4)
w SOM y (t + 1) = norm w SOM y (t) + λ(t)ǫ SOM g σ ( y -y * )(z I -w SOM y ) (5) (6)
where g σ ( x) is a zero-mean Gaussian function with standard deviation σ and y * denotes the position of the best-matching unit (the one with the highest activity) in N . In accordance with standard SOM training practices, the SOM learning rate and radius, ǫ SOM and σ, start at ǫ 0 , σ 0 and are exponentially decreased in order to attain their long-term values ǫ ∞ , σ ∞ at t = T conv .

LWPR

We use the LWPR algorithm as described in [START_REF] Vijayakumar | Incremental online learning in high dimensions[END_REF] using a publicly available implementation [START_REF] Vijayakumar | A library for locally weighted projection regression[END_REF].

The MNIST handwritten digit database

For all experiments, we use he publicly available MNIST classification benchmark as described in [START_REF] Lecun | Gradient-based learning applied to document recognition[END_REF]. It contains 10 classes, corresponding to the 10 handwritten digits from "0" to "9", see also Fig. 1. Each sample has a dimensionality of K = 28 × 28 = 784. We split the data into two sets: D 0-4 containing the digits from "0" to "4", and D 5-9 containing the remaining digits. Each set is again split, at a proportion of 5:1, into a training and a test set to measure generalization performance, giving in total four data sets: D train 0-4 ( 25.000 samples), D test 0-4

( 5.000 samples), and analogously D train 5-9 , D test 5-9 . For training and evaluating performance on all digits, we also create the sets D train 0-9 , D test 0-9 in an analogous fashion.

Experiments

Experimental setup We conduct an identical set of experiments both for PRO-PRE and for LWPR, which is designed to measure the capability to perform incremental learning. To this effect, we present D train 0-4 for T 1 = 15000 iterations, subsequently D train 5-9 for T 2 = 15000 iterations, and lastly D train 0-9 for T 3 = 2000 iterations, which is fair to both algorithms as it allows them to converge (longer training times did not change results). At all times, we can measure generalization performance on any of the sets D test 0-9 , D test 0-4 and D test 5-9 . In order to establish a baseline performance, to be compared to offline, batch type algorithms, we also train and evaluate both algorithms on D 0-9 . We use the following parameters for PROPRE: n = 30, ǫ SOM = 0.2, ǫ LR = 0.09 n 2 , ǫ 0 = 0.8, σ 0 = 0.6n, T 1 = 4000, ǫ ∞ = 0.2 and σ ∞ = 1. Both SOM and LR weight matrices were initialized to random uniform values between -0.001 and 0.001. For PROPRE, input vectors where always normalized to have an L2 norm of 1.0 before presenting them. LWPR is parametrized as follows, using notation from [START_REF] Vijayakumar | A library for locally weighted projection regression[END_REF] for details: init alpha=0, init D =3.2 (in order to limit receptive field number to ¡ 8), diag only=0, update D=1, using default parameters everywhere else. No normalization of inputs was performed for LWPR.

Results Results are given in Tab. 1 and show that baseline performance of both algorithms is roughly comparable, with a slight edge to PROPRE. Both algorithms convincingly avoid catastrophic forgetting since performance on D test 0-9 after successive training on D train 0-4 and D train 5-9 is >> 10% which we would expect in case the digits 0-4 had been forgotten. In both cases performance improves

PROPRE LWPR ❳ ❳ ❳ ❳ ❳ ❳ ❳ ❳ ❳ ❳ train set test set D test 0-9 D test 0-4 D test 5-9 D test 0-9 D test 0-4 D test 5-9 D train 0-9 90% x x 83% x x D train 0-4 43% 95% x 44% 93% x D train 5-9 80 x x 71 x x D train 0-9 85 x x 80 x x%
Table 1: Performance evaluation of PROPRE and LWPR on MNIST data. First row: performance on D test 0-9 when both algorithms are trained on all classes simultaneously, e.g., on D train 0-9 . Rows 3,4,5: performance when successively training on D train 0-4 (row 3), D train 5-9 (row 4) and, for a small interval on D train 0-9 (row 5).

when performing a short retraining on D train 0-9 . For PROPRE, this is because linear regression models for D train 0-4 are no longer fully valid after training on D train 5-9 , and retraining can help fix this. Similarly, for LWPR, some classes share receptive fields and therefore the associated linear models are no longer fully valid after training on D train 0-4 and need to be readjusted.

Discussion

Generally, LWPR had to be parametrized very carefully in order not to exceed the computer memory limits, allowing at most 8 receptive fields per output dimension. This is because, for K input dimensions, a single receptive field in LWPR requires roughly 5K 2 floating point values, mainly in order to store the covariance matrix and keep track of data statistics. As LWPR allocates RFs independently for each of M output dimensions, the overall memory requirements are O(5M K 2 ) which gets problematic for large K as it is the case for MNIST(K = 28 2 ). Limiting RF creation surely limited LWPR's ability to exert its full potential on this benchmark; on the other hand, it is a fair comparison because both algorithms were executed on the same computer (Ubuntu Linux, 2 GB RAM, CoreI7 processor) and had to make do with the same resources. PROPRE suffered from no memory limitations since, for a fixed size n × n of the induced representation N , PROPRE requires Kn 2 + n 2 M = n 2 (K + M ) floating point values for storing the weight matrices. If we compare the baseline performance of both algorithms to the performances of other algorithms on MNIST, we find that PROPRE performs better than some but significantly worse than the best algorithms, whereas LWPR (with these specific parameters) performs worse than even linear models. For the case of PROPRE, this is the price to pay for online and incremental learning capacity, as already amply discussed in [START_REF] Vijayakumar | Incremental online learning in high dimensions[END_REF]. For LWPR, it can be concluded that in the present form it is unsuited for this kind of input/output dimensionalities. The fact that LWPR is intended to perform regression and not classification does not influence memory requirements, and in addition classification is a special case of regression even if the reverse does not hold.

We have presented an algorithm for resource-efficient incremental learning that draws its efficiency from principles of biological information processing. We showed that it compares favorably with the quasi-standard algorithm for incremental learning, LWPR, when tested on a standard machine learning benchmark while requiring only a fraction of computational, and above all, memory resources. What is more, the bulk of computation is contained in the SOM projection implemented by a matrix multiplication, which is an operation that can is easily parallelized. Therefore, not only memory consumption hut also runtime speed can be very low when using PROPRE. Next steps will include tests on robotic applications (e.g., online learning of forward and inverse models) as well as a hierarchical versions the architecture with multiple stacked induced representations N for improving computational power.

* Thomas Hecht gratefully acknowledges financial support by the French Armaments Procurement Agency (DGA) and Ecole Polytechnique.


Randa Kassab
email: randa.kassab@inria.fr

Frédéric Alexandre
email: frederic.alexandre@inria.fr

A Modular Network Architecture Resolving Memory Interference through Inhibition

Keywords: Associative memory, interference, inhibition, biological systems

In real learning paradigms like pavlovian conditioning, several modes of learning are associated, including generalization from cues and integration of specific cases in context. Associative memories have been shown to be interesting neuronal models to learn quickly specific cases but they are hardly used in realistic applications because of their limited storage capacities resulting in interferences when too many examples are considered. Inspired by biological considerations, we propose a modular model of associative memory including mechanisms to manipulate properly multimodal inputs and to detect and manage interferences. This paper reports experiments that demonstrate the good behavior of the model in a wide series of simulations and discusses its impact both in machine learning and in biological modeling.

Introduction

In the domain of machine learning, models of neural networks are classified along their architecture and their mode of learning [START_REF] Hertz | Introduction to the theory of neural computation[END_REF], specifically corresponding to supervised and unsupervised modes. In contrast, in the domain of cognitive science, a natural learning paradigm considered in a realistic behavioral and ecological environment often associates several neuronal architectures and learning modes. This is for example the case with pavlovian conditioning that has been shown to require learning a variety of invariants and to modify the neuronal circuitry in several brain regions including the amygdala, hippocampus and cortex [START_REF] Maren | Building and Burying Fear Memories in the Brain[END_REF]. Consequently, in addition to developing efficient models of neural networks designed for their specific characteristics, there is also a need for a more systemic view of learning, considered at the global cognitive level.

Such an approach was already proposed twenty years ago in [START_REF] Mcclelland | Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory[END_REF] arguing that the brain exploits complementary learning systems, with a slow and procedural learning in the cortex, able to extract structures and regularities in the data and to generalize, compared with a quick learning in the hippocampus able to retain the specifics of one's life experiences. This paper, with a very strong impact in both cognitive and machine learning communities, proposes that these systems might be respectively implemented with classical neural models of pattern matching like the multilayer perceptron for the slow learning and models of associative memory for the quick learning.

As an illustration, these models can be contrasted with the property of generalization. Generalization is often reported as a desirable property of artificial neural networks. This phenomenon occurs if, when a network is presented with an example it has never seen before, it is able to interpolate a satisfactory response from the combination of close previously learned examples. Such a response can be judged satisfactory not only because from a limited learning phase the network behaves well in a wider domain but also because in some sense learning went beyond specific cases and was able to extract some general structures or regularities in the example space. In some cases, however, this property might be considered a flaw. This is the case for example when there is no useful topography in the example space or when the goal is to learn some arbitrary association. Consider for example learning to associate a phone number with a name: there is nothing to learn from the euclidean distance between two such numbers and you can in no way discover an association if it was not instructed to you before. This contrasts the cases of learning a general rule from a set of examples, as it is for example studied with layered architectures like the multilayer perceptron, versus learning by heart specific cases like in associative memories.

Neural models of associative memories have been proposed with recurrent networks like the Hopfield model [START_REF] Hopfield | Neural networks and physical systems with emergent collective computational abilities[END_REF] and the Willshaw model [START_REF] Willshaw | Non-holographic associative memory[END_REF]. Based on classical connectionist characteristics (like units with non linear activation functions and hebbian learning), the recurrent architecture of these networks indicates that learning is mainly focused on the inner characteristics of an example to be memorized and not on the elaboration of abstract representations in intermediate layers. Nevertheless, some problems can appear if too close examples are learned. In such a case, the network might elaborate an answer from the combination of several learned examples; what would be called generalization in other circumstances is called here interference.

As a consequence, models of associative memories are generally used as content addressable memories, where few prototypes are stored as stable states of the network and noisy or incomplete patterns are presented as inputs and reconstructed to the closest stored example. Beyond this use as an autoassociative memory (where initial input and final result have the same dimension), the adaptation to heteroassociative memory is straightforward: just virtually split the recurrent network in two sets of neurons A and B. The recurrent connectivity includes connections within A and within B (seen as two autoassociative memories) and between A and B (heteroassociative memory between the two sets of different dimension A and B). As configurations of A+B are learned as prototypes, proposing an incomplete pattern A (B neurons being set to 0) will result in the reconstruction of A+B, yielding the answer B. The main acknowl-edged weakness of these models is about their limited capacity of storage and the associated risk of catastrophic interference when this capacity is exceeded or when too close prototypes are stored [START_REF] Graham | Capacity and information efficiency of the associative net[END_REF][START_REF] Knoblauch | Memory capacities for synaptic and structural plasticity[END_REF]. The best solution to this problem is to require a sparse coding, which intrinsically also limits the maximum number of stored prototypes. An associated strategy is to orthogonalize the inputs and project their encoding in higher dimensions, which results in larger weight matrices to manipulate [START_REF] Mcnaughton | Hebb-marr networks and the neurobiological representation of action in space[END_REF]. In both cases, this might prevent associative memories from being applied to large scale realistic problems and can accordingly explain why they didn't have the same expansion in the machine learning community than layered networks. It is consequently highly desirable to develop scalable models of associative memory.

In previous work, we have proposed a modular network model of associative memory [START_REF] Kassab | Integration of exteroceptive and interoceptive information within the hippocampus: a computational study[END_REF] grounded on biological data [START_REF] Andersen | The Hippocampus Book. Oxford Neuroscience Series[END_REF][START_REF] Samura | Sequence disambiguation and pattern completion by cooperation between autoassociative and heteroassociative memories of functionally divided hippocampal CA3[END_REF]. These data report heterogeneities in the hippocampal structure that might support the coexistence of autoassociative and heteroassociative networks in this region. Specifically, the hippocampus is a neuronal structure known to be involved in episodic memory [START_REF] Tulving | Episodic and semantic memory. Organization of Memory[END_REF], corresponding to the storage of specific episodes including their context and their emotional or motivational significance. For example, the hippocampus is involved in contextual learning of pavlovian conditioning [START_REF] Carrere | A pavlovian model of the amygdala and its influence within the medial temporal lobe[END_REF], linking neutral stimuli and their context to biologically significant events (reward and punishment). Though primarily oriented toward biological modeling, we have also explained in [START_REF] Kassab | Integration of exteroceptive and interoceptive information within the hippocampus: a computational study[END_REF] the interest of such a segregation from an information processing point of view (cf. the concluding section for a summary). In addition, we have also postulated an additional mechanism for the association of autoassociative memories, that might result in a more robust system, particularly more resistant to interference. The goal of this paper is to evaluate more precisely the performances of this mechanism from an information processing point of view.

In the next section, we will present this model together with its formalism based on the associative memory initially proposed by Willshaw [START_REF] Willshaw | Non-holographic associative memory[END_REF]. Then we will report the experiments that were conducted to evaluate its resistance to interference and the associated results. We will conclude by explaining the interest of such a mechanism both in neuroscience and in information processing domains.

Multiple Associative-Memory Model

The model is made up of two autoassociative networks that are heteroassociatively linked through a layer of intermediate cells (Fig. 1). The goal is to associate two multi-element patterns in such a way that when at least some elements of the first pattern are presented both patterns can be recalled as a whole. In the hippocampus, these two patterns are considered to represent two important dimensions of episodic memories: 1) The perceptual dimension arises from the integration of different kinds of signals coming from the perception of the outer world: exteroception. 2) The emotional dimension reflects the perception of internal cues of different valences related to pain and pleasure: interoception. Then, the two autoassociative networks considered in the model receive and store independently two types of input patterns, a (e) and a (i) . The layer of intermediate cells is organized into a small number of ordered groups of valence cells that receive valence-related information from the same interoceptive pathways as the interoceptive autoassociative network. The cells in the first group can be directly activated by interoceptive inputs to the model and can therefore be thought of as the primary valence cells. Interoceptive inputs on the cells in the other groups, which are termed associated cells, are conditional, that is, they can not evoke postsynaptic activity within associated cells unless a concomitant signal, m k , related to the activity pattern of a precedent group is applied.

The valence cells belonging to the same group of intermediate cells are not interconnected. By contrast, inhibitory connections, I ij , exist between cells belonging to different groups. The inhibitory connections are not plastic. They are prewired such that an inhibitory connection from cell i to cell j exists (I ij = 1) if the two cells belong respectively to different groups, k and l, and l precedes k (l < k). Thus, each group of associated cells, once activated, silences excitable cells in its preceding groups including the primary group of valence cells. This means that at most valence cells in one group can be active at a time.

The formation of extero-interoceptive associations is done at the level of heteroassociative links, w

(e-v) ij
, between the exteroceptive autoassociative network and the groups of intermediate valence cells. These latter provide direct excitatory input to the interoceptive autoassociative network through non-plastic connections, w

(v-i) ij
. These connections are prewired only between valence cells that are sensitive to the same kind of valence.

The classical binary version of the Willshaw network [START_REF] Willshaw | Non-holographic associative memory[END_REF] is chosen as the basis for the implementation of both auto-and heteroassociative memory functions in the model. The neurons are simple McCulloch-Pitts binary threshold units and learning begins with all the synaptic weights set to zero. Synaptic plasticity is achieved according to a clipped version of Hebbian learning: a single coincidence of presynaptic and postsynaptic activity changes the synaptic weight w ij from 0 to 1, while further co-activations do not induce further changes. The recall process is done by presenting a cue pattern x and counting the dendritic sum for each cell j (s j = n i=1 w ij xi ) in one-time step. The output cells that have a dendritic sum equal to or higher than the number of active inputs are activated. The quality of a recalled pattern can be assessed according to its Hamming distance (HD) from the originally stored pattern (i.e. the number of elements that differ between the two patterns. For example, if x=(0 1 1 1 0) and y=(1 1 0 1 0) then HD(x,y)=2).

Similarly to cholinergic models of the hippocampus [START_REF] Hasselmo | Encoding and retrieval of episodic memories: Role of cholinergic and gabaergic modulation in the hippocampus[END_REF][START_REF] Meeter | Mode shifting between storage and recall based on novelty detection in oscillating hippocampal circuits[END_REF], our model operates in transition between two modes, storage and recall, depending on a hyperparameter ACh. This mechanism is inspired from biological data describing mode switching under the dynamic regulation of the levels of acetylcholine (ACh) released from septal cholinergic projections to the hippocampus. During recall, a retrieval cue, a (e) , is applied to the exteroceptive autoassociative network. The pattern of activity obtained at the output, â(e) , drives retrieval in the heteroassociative network. An intermediate valence cell, l, can fire only if the dendritic sum of its excitatory inputs exceeds the threshold value and if it does not receive inhibitory inputs from other valence cells that have already fired. The activity of the intermediate valence cells, ã(i) , triggers recall in the interoceptive autoassociative network yielding the valence prediction by the model, â(i) .

Just after delivery of the interoceptive information, two novelty-detection processes take place to compare the retrieved patterns to the actual patterns from extero-and interoception. The novelty condition occurs when the Hamming distance between two patterns exceeds pre-specified thresholds (HD (e) > e or HD (i) > v). Novelty induces ACh dynamics that favor learning of new inputs, otherwise the model settles in recall mode.

During learning, excitatory intrinsic synaptic transmission along the recurrent connections is removed and activity in the model is purely driven by afferent extero-and interoceptive inputs, a (e) and a (i) . In the model, two kinds of interference can occur due to a saturation, or overload of learning. The first kind of interference occurs within the autoassociative memories when too many or too close inputs are stored. It is called pattern overload and can be much mitigated using sparse patterns and low memory load conditions. The second kind of interference is called valence overload and is more likely to occur when elements making up the stored patterns become simultaneously associated to different valences. Consider for example learning AB+, AC-and BD-, where A, B, C and D are exteroceptive patterns and + and -are interoceptive valences. Since A and B are simultaneously associated to + and -valences, the recall of AB would probably generate an interference (both responses produced). The model deals with valence-overload interference by monitoring activity of intermediate valence cells, y (v) . If any activity is observed among intermediate valence cells ( i y (v) i > 0) in response to exteroceptive inputs a matching process takes place to determine whether this activity matches interoceptive valence-specific inputs. A mismatch (HD (v) > v) signals a potential interference to a successive group of associated valence cells that become able to respond to valence-related inputs and rapidly silence valence cells that were active in preceding groups.

Experiments

The validity of the proposed model is examined through a series of numerical experiments (cf. [START_REF] Kassab | Integration of exteroceptive and interoceptive information within the hippocampus: a computational study[END_REF] for the description of other numerical experiments with this model). The simulated model is configured with 150 cells in the exteroceptive autoassociative network and 3 cells in the interoceptive autoassociative network. The intermediate valence cells are organized into 5 groups of 3 cells each.

Inputs are provided to the model as two independent patterns of activity. The exteroceptive inputs are generated as random 150-element binary patterns with 6 elements being active (set to 1). The interoceptive inputs are modeled by 3 binary cells to differentiate positive, negative and neutral valence states. One of these cells switches to its active state according to whether a pleasant (100), unpleasant (010), or neutral (001) stimulus is present.

The performance is evaluated by comparing the output patterns recalled by the model against the original representation of the input patterns that were presented to the model as new information to be stored. Specifically, two kinds of recall errors are considered when evaluating simulation results. Pattern completion errors which reflect the Hamming distance between the learned and retrieved activation for exteroceptive patterns, and valence prediction errors which reflect the Hamming distance between the correct and predicted valence. In both cases, errors are scored when Hamming distance is greater than zero.

Two types of simulations are set out to test the model for its ability to rapidly link exteroceptive patterns and their emotional valences while avoiding valence overload interference. The first set of simulations examines the effect of the number of stored patterns on the accuracy of valence prediction. The model is tested under full-cue and partial-cue recall conditions. The number of stored patterns is kept low enough that under full-cue conditions almost no pattern overload occurs at the level of autoassociative memories. This is important to ensure that any prediction errors might be detected arise directly from valence overload at the level of heteroassociative links between exteroceptive and interoceptive patterns. The second set of simulations focuses on how to quantify the ability of associated valence units to orthogonalize conflicting associations arising from a change in previously learned valence values.

In all of the simulations, the performance of the proposed model, also called the full model, is compared with that of a reduced model with the groups of associated cells removed. A third model with a single autoassociative memory in which both exteroceptive and interoceptive information are merged into a single pattern is also considered to further delineate benefits of the proposed architecture under partial-cue conditions. All results are averaged over 10 simulation runs and are displayed throughout the figures as mean ± standard error of the mean. The novelty-detection thresholds, e and v, are set to zero for all the simulations. The first set of simulations is run by varying the number of training patterns and observing how valence prediction is affected with and without the groups of associated cells included in the model (Fig. 2). Training patterns are presented randomly into blocks of N trials with N varying from 10 to 100 in steps of 10. At the different values of N, the full and reduced models were able to recognize exteroceptive patterns with pattern completion errors less than 0.3%. However, there was a noticeable difference between the two models in terms of valence prediction.

Results

Storage capacity

As illustrated in Fig. 2A, following the first presentation of training patterns, both models perform perfectly up to N=20, after which point valence prediction errors begin to occur more frequently with increasing size of the blocks of training trials. But as expected, adding the associated cells decreases valence prediction errors at each value of N. For instance, at N=100, the percentage of prediction errors is about 32% for the reduced model but falls to about 20% for the full model. This reduction results from the identification of about 7% of the stored associations as interfering associations (Fig. 2B). Interference effect is accordingly reduced through the recruitment of one group of associated cells (Fig. 2C). During the second presentation of training patterns, the full model detects all the interfering associations that remain and orthogonalizes them using the same group of associated cells (Fig. 2C). Therefore, the performance of valence prediction differs significantly between the two models after the second presentation of training patterns: the reduced model continues to commit the same prediction errors while the proposed model performs with no errors at all.

Pattern completion

In the above simulations, it is pertinent to emphasize that no differences were observed between the autoassociative model and the heteroassociative model with the groups of associated cells removed. This is of no surprise because in both cases valence prediction is initiated by a complete set of exteroceptive cues. Under partial-cue conditions, the proposed model as well as its reduced version are expected to take advantage of the fact that pattern completion of exteroceptive cues is performed prior to valence prediction. To test this premise, the three models are trained in the same manner as in the previous simulations except that recall is triggered by partial versions of the original trained patterns. Specifically, the block size is set to 100 training patterns and the model is cued with partial versions with either 1, 2 or 3 of the 6 active inputs turned off.

As shown in Fig. 3A and B, all the models perform similarly and reasonably well in terms of pattern completion of exteroceptive cues. The accuracy of valence prediction of the autoassociative model is much worse than that of the heteroassociative models and monotonously drops as the number of deleted elements increases (Fig. 3C). On the contrary, the heteroassociative models are much less sensitive to the percentage of deleted elements. The accuracy of valence prediction with the 1/6 partial-cue condition is the same as that obtained with the full-cue condition (Fig. 3D andE). This is because exteroceptive patterns are almost perfectly reconstructed as shown in Fig. 3B. The removal of two or three of the six active cues causes a proportional decrease in the accuracy of pattern completion of exteroceptive patterns. Consequently, the improvement in valence prediction by the proposed model is less pronounced but still highly significant as compared to the reduced model. For all the percentages of removal simulated, the model makes use of one group of associated cells to tackle valence-overload interference (Fig. 3G).

Discrimination

Here we investigate the functional significance of the groups of associated cells using numerical simulations with reversal learning tasks. The task in the first set of simulations involves two phases. In the first phase the model is presented repeatedly with 50 training patterns [e.g. A+, B-, C (neutral), etc.] over 4 blocks of trials and the percentage of prediction errors made at the beginning of each trial is measured and displayed in Fig. 4A. This is a simple discrimination learning problem similar to those tested in the previous simulations. Thus as was observed before, valence-overload interference occurs at the early stages of learning and exhibits the recruitment of one group of associated cells to tackle it. When the groups of associated cells are removed the reduced model shows impaired performance that persists over the repeated trials. In the second phase, emotional valences of the training patterns are randomly changed to other value with a probability of 50% [e.g. A-, B (neutral), C (neutral), etc.]. As shown in Fig. 4A the proposed model quickly learns to reverse its behavior as all the emotionally changed patterns are detected and learned on the first training trials after reversal. On the other hand, the reduced model fails to acquire the new associations since the old ones have not been unlearned.

Reversal learning

Further analysis of the model behavior is based on a cue-context reversal learning task similar to that established by [START_REF] Levy-Gigi | Impaired context reversal learning, but not cue reversal learning, in patients with amnestic mild cognitive impairment[END_REF] to investigate reversal learning in patients with mild amnesic cognitive impairment. To simulate this task, three groups of 4 exteroceptive patterns each are formed such that one of the 6 active elements is used to encode the presence of a sensory cue and the others to encode contextual cues. No overlap is allowed between cells encoding for different cues or contexts (cf. Table 1).

Table 1. The experimental design of the task of [START_REF] Levy-Gigi | Impaired context reversal learning, but not cue reversal learning, in patients with amnestic mild cognitive impairment[END_REF]. Note. A-H refer to eight cue shapes, 1-8, eight contexts, + and -indicate respectively positive and negative valences.

Training patterns Task

Group 1 Group 2 Group 3 Phase 1 Phase 2

(original) (cue reversal) (context reversal) (acquisition) (retention & reversal) A1+ E1- A5- Group1 Group1 B2+ F2- B6- Group2 C3- G3+ C7+ Group3 D4- H4+ D8+
In the first phase of acquisition, the model is repeatedly presented with the training patterns in the first group and valence prediction is evaluated over four blocks of training trials. Fig. 5 shows that both full and reduced models make correct valence prediction after a single exposure to the training patterns. Then, the reversal phase is immediately followed by exposing the models to new training patterns from the second and third groups, in addition to the old ones. The training patterns are also presented repeatedly four times in random order. The results show that, in the first block of trials, valence prediction errors are made for both new and old patterns. This reflects the fact that heteroassociative connections are irrelevantly strengthened between the original patterns and valences of new patterns. When interference is detected, one group of valence-associated cells is recruited and prediction errors fall to zero rapidly on the third block of trials after reversal. In contrast, the number of prediction errors the reduced model makes is still the same as the blocks progress for the same reason stated above.

Discussion

The primary goal of this paper was to propose a new framework to consider associative memories, particularly to make them efficient even in adverse conditions. Indeed, associative memories have powerful properties for learning by heart specific patterns and recalling them from partial information. They can learn quickly and recall patterns as they were initially presented, without modification nor generalization. It has been shown [START_REF] Mcclelland | Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory[END_REF] that such properties are highly desirable for specific classes of cognitive functions and a class of models able to emulate them should consequently be considered carefully. Nevertheless, associative memories are little exploited in classical machine learning because they suffer from limited storage capacities, particularly when patterns to be stored are close, resulting in interferences and catastrophic forgetting [START_REF] Hertz | Introduction to the theory of neural computation[END_REF].

It has also been shown [START_REF] Hertz | Introduction to the theory of neural computation[END_REF] that associative memories can be simply used in auto-association for pattern retrieval but also in heteroassociation between two different classes of inputs. In the model presented here, we exploit this heteroassociative view to propose a modular network. From an information processing point of view, we explain in that paper that a heteroassociation between two data spaces of different size leads to more robust retrieval than a simple autoassociation with a flat vector concatenating both kinds of information because the evaluation of the Hamming distance between stored and actual patterns would consider in this latter case that one error in any dimension yields the same penalty, which is obviously not the case. This is confirmed in the paper, considering comparison of performances between similar autoassociative and heteroassociative models.

A modular view of associative memories is also exploited to implement another powerful property of our model, for managing interferences, using another set of units called associated cells. When an association is learned between a high dimensional data space and a smaller space representing labels (valences in the present case), one central problem is about the association of close patterns with different labels or of different combinations of patterns with different labels. This classical problem has been termed configural learning [START_REF] Catalin | Attention, configuration, and hippocampal function[END_REF]. With a fully automatic algorithm to insert associated cells between the heteroassociative modules, we have proposed in the present model a mechanism able to detect interference at the heteroassociative level and to trigger new learning accordingly. The experiments reported here, particularly comparing performances of reduced and full heteroassociative models, show that our model is very efficient at performing such a learning. In addition, this learning process is very quick, which preserves another important specificity of episodic learning. In this paper, we also propose to relate the very good properties of modular heteroassociative memories to two different frameworks. In the framework of brain modeling, the model has been primarily built as a biologically informed model of the hippocampus [START_REF] Kassab | Integration of exteroceptive and interoceptive information within the hippocampus: a computational study[END_REF]. In addition to proposing some evidences for the implementation of a modular network in this cerebral structure, we also propose that heteroassociation could take place between exteroception and interoception, corresponding to different kinds of hippocampl inputs. In further studies, this could be extended to other classes of hippocampal inputs, particularly related to the frontal cortex.

The focus was set here on interoception and exteroception because this study was related to other studies in the team [START_REF] Carrere | A pavlovian model of the amygdala and its influence within the medial temporal lobe[END_REF] related to pavlovian conditioning. This learning paradigm is very interesting because it is an excellent basis for a systemic view of learning in the brain, with adaptive processing involving (at least) the amygdala, the hippocampus and the cortex [START_REF] Maren | Building and Burying Fear Memories in the Brain[END_REF]. Extending the duality between procedural learning in the cortex and specific cases learning in the hippocampus [START_REF] Mcclelland | Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory[END_REF], we explain in [START_REF] Carrere | A pavlovian model of the amygdala and its influence within the medial temporal lobe[END_REF] that the amygdala is designed to learn pavlovian associations from cues extracted by both structures with their own way of learning and also report, in accordance to other authors [START_REF] Moustafa | A model of amygdala-hippocampalprefrontal interaction in fear conditioning and extinction in animals[END_REF], a synergy between the three modes of learning, where an event in one learning module (an error of prediction, the occurrence or the storage of a specific case) can trigger or modify learning in another learning module. Considering the importance of such a distributed learning principle, better understanding its details deserves additional work.

Bio-inspiration was also a strong motivation for this work because, in addition to classical evaluation of performances, one of the experiments we made was also designed to reproduce behavioral and cognitive data in the medical domain for amnesic impairments [START_REF] Levy-Gigi | Impaired context reversal learning, but not cue reversal learning, in patients with amnestic mild cognitive impairment[END_REF]. Related medical data strongly suggest the central role of the hippocampus in this memory process, giving additional interest to the complementary learning system hypothesis [START_REF] Randall | Complementary Learning Systems[END_REF]. The cognitive framework initiated in [START_REF] Mcclelland | Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory[END_REF] postulates how procedural learning in the cortex, slowly learning and able of generalization, might be instructed by specific cases learned quickly in the hippocampus avoiding interferences. Adding emotional aspects with the dissociation between interoceptive and exteroceptive cues, extends this framework of mnemonic synergy in the brain, proposed for medical purposes.

In the framework of machine learning, we have also presented this work as a new model of associative memory and its main results have been described here mainly in the framework of information processing. This is also the reason why we use simple binary units in the hippocampal model, even if more complex functioning rules might be expected in the framework of a biologically inspired model: Even if more complex units might be considered in future works, particularly to fit with more precise biological data, the main goal of the present work was to settle the main computational principles of our modular model. Beyond the case for pavlovian conditioning with interoceptive and exteroceptive cues, we believe that it is not rare in the information processing domain to cope with such associations between data of different dimensions, as it is the case for example with labeled data (high-dimensional data associated with a symbolic label). In this case, we claim that combining autoassociation and heteroassociation as proposed here results in more robustness in the retrieval phase. More generally and beyond associative memories, heteroassociation using intermediate cells to reduce ambiguities is a general class of approaches in machine learning and the criteria proposed here to avoid interference and keep associations simple could be extended to other models of machine learning in further works. This could illustrate other cases where a primary biological inspiration yields efficient learning principles.

Fig. 1 .

 1 Fig. 1. The architecture of the hippocampal model. Black lines denote the basic circuit of the model while blue lines denote changes in circuitry mediated by one group of associated cells (blue) following the detection of valence-overload interference (red arrow). Autoassociative and heteroassociative connectivities between hippocampal cells are denoted respectively by bidirectional dashed lines and simple dashed lines without arrows. Inhibitory connections between valence cells are denoted by lines ended with circles. Stable non-plastic connections, both excitatory and inhibitory, are denoted by solid lines.

Fig. 2 .

 2 Fig. 2. Influence of the number of stored patterns on the accuracy of valence prediction. (A) Percentage of prediction errors of the model without associated cells (W/O) and with associated cells after one block (W (P1)) and two blocks (W (P2)) of training trials. (B) Rates of interference detection during the first (P1) and second (P2) training trials. (C) Number of groups of associated cells needed to resolve interference detected during training trials T1 and T2.

Fig. 3 .

 3 Fig. 3. Performance of the proposed model after training on 100 input patterns. The model is tested using partial cues in which 1, 2, or 3 out of 6 active elements in the original inputs are turned off. (A) Pattern completion performance, defined as the percentage of retrieved patterns that differ at least by one element from the originally stored patterns. (B) Pattern completion performance, defined in terms of Hamming distance between the stored and retrieved patterns. (C, D) Valence prediction performance of the proposed model with (w) and without (w/o) associated cells after one and two blocks of training trials. (E) Maximal number of groups of associated cells needed to resolve interference detected under all simulation conditions (one and two blocks of training trials P1 and P2, and for 1/6, 2/6 and 3/6 partial-cue conditions).

Fig. 4 .

 4 Fig. 4. Discrimination reversal learning. (A) Percentage of prediction errors of the model with (w) and without (w/o) associated cells. (B) Rates of interference detection over each block of trials. (C) Number of groups of associated cells needed to resolve interference across the different blocks of trials.

Fig. 5 .

 5 Fig. 5. Cue-context reversal learning. (A) Percentage of prediction errors of the model with (w) and without (w/o) associated cells. (B) Rates of interference detection over each block of trials. (C) Number of groups of associated cells needed to resolve interference across the different blocks of trials.


Carlos Flores

Vicente Milanés

Joshué Pérez

David González

Fawzi Nashashibi

Optimal Energy Consumption Algorithm based on Speed Reference Generation for Urban Electric Vehicles

Power consumption and battery life are two of the key aspect when it comes to improve electric transportation systems autonomy. This paper describes the design, development and implementation of a speed profile generation based on the calculation of the optimal energy consumption for electric Cybercar vehicles for each of the stretches that are covering. The proposed system considers a commuter daily route that is already known. It divides the pre-defined route into segments according to the road slope and stretch length, generating the proper speed reference. The developed system was tested on an experimental electric platform at Inria's facilities, showing a significant improvement in terms of energy consumption for a pre-defined route.

I. INTRODUCTION

Electric vehicles (EVs) are getting more and more attention because their contribution toward eco-friendly cities. Non-emission vehicles will definitely help to improve citizen's daily life, reducing dramatically both noise and pollution [START_REF] Cerovsky | Hybrid electric cars, combustion engine driven cars and their impact on environment[END_REF]. For this reason, some governments-i.e. Canada [START_REF] Graham | Greenhouse gas emissions from light duty vehicles under a variety of driving conditions[END_REF]-have carried out studies to figure out the emissions according to the kind of vehicle. Results showed that light duty gas vehicles are the biggest producer of CO 2 and the second greatest producer of N 2 O and methane, which makes them the main contributors towards gas emissions-because of their higher market penetration. In United States [START_REF] Rakha | Eco-vehicle speed control at signalized intersections using i2v communication[END_REF], the transportation sector consumes three-quarters of the total burned petroleum, which makes it the second largest carbon emitter in the country. Because of this, electric vehicles are an adequate solution to reduce greenhouse gas (GHG) emissions and pollution produced by road transport systems.

On the other hand, EVs present serious limitations for their market deployment. Specifically, there are two unsolved challenges: 1) battery charge [START_REF] Si | Development research about the power battery management system of pure electric vehicle[END_REF]: how long it takes to fully load when running off at driving; and 2) battery life [START_REF] Marano | Lithium-ion batteries life estimation for plug-in hybrid electric vehicles[END_REF]: how much energy will last when driving. Recent years have shown a lot of development on battery technologiesi.e. the mixed structures using supercapacitors [START_REF] Tankari | DC-bus voltage control for multi-sources systems -battery and supercapacitors[END_REF] or fuel cells [START_REF] Burke | Batteries and ultracapacitors for electric, hybrid, and fuel cell vehicles[END_REF]. However, EVs autonomy remains considerably lower than gas-powered vehicles. This paper deals with this second challenge, proposing an intelligent modular algorithm that provides better performance in order to improve EVs autonomy.

Specifically related with the solution to the EVs autonomy problem, there are two main ways for improving battery Authors are with the Robotics and Intelligent Transportation Systems (RITS) Team, Inria Paris-Rocquencourt, Domaine de Voluceau, 78153

Le Chesnay, France {carlos-eduardo.flores-pino, vicente.milanes, joshue.perez_rastelli, david.gonzalez-bautista, fawzi.nashashibi} @inria.fr life. One of them is optimizing the technology available in the car such as better batteries or more efficient motor and mechanical parts. This would bring better results when trying to extend batteries' life, but modifying the vehicle physical characteristics would mean increase the cost and technology, which is sometimes a very complex task. The other way for improving EVs autonomy is modifying the driving performance, considering all the circumstances around the vehicle to take advantage of them and performing a more efficient driving.

For the development of a reliable algorithm that takes into account the driving environment for extending battery life, several parameters have to be considered: distance, traffic state, traversing intersections and merging zones [START_REF] Li | A real time traffic light control scheme for reducing vehicles CO2 emissions[END_REF]. Having this in mind, optimal route generation algorithms are a suitable candidate to solve this problem. These methods offer a route, avoiding heavy traffic on urban roads and traffic lights in intersections [START_REF] Rojas | Fuzzy logic techniques for cybercars: a control and decision approach[END_REF], which would cause more unnecessary braking maneuvers. Nonetheless, none of those considerations really provides a reliable solution when trying to save battery life.

An optimal route planning would lead to a less energy consumption [START_REF] Chrenko | Autonomy estimation for EV based on road planning software[END_REF], where a real time state of the vehicle and the road layout are considered in order to make a speed profile plan. Once this optimal route is obtained from an algorithm-i.e. [START_REF] Sachenbacher | Efficient energy-optimal routing for electric vehicles[END_REF] and [START_REF] Larsson | Commuter route optimized energy management of hybrid electric vehicles[END_REF]-and the environment conditions are established, it is necessary to optimize the energy consumption for this pre-defined route. Drivers are usually covering the same trajectory from home to work everyday. However, they are usually following the traffic flow, including consecutive braking and accelerating maneuvers that significantly affect the battery life of EVs. The proposed method considers this regular situations in order to cover a pre-defined distance in the desired time but increasing the battery life as much as possible.

In the state-of-art of eco-driving, the optimal route algorithm is one of the main tools to improve vehicle's autonomy. [START_REF] Jurik | Energy optimal real-time navigation system: Application to a hybrid electrical vehicle[END_REF] presents an approach that aims on finding the best route in terms of energy. It considers an origin, destination and the initial battery state of charge. With these parameters, the optimal route is computed by finding the path segments with the minimal energy cost. Even though this method offers a solution to extend the vehicle autonomy, it does not approach the problem from the point of view that this work assumes. This article presents an intelligent algorithm to generate a velocity profile that would allow a commuter go from one point to another taking a certain time, achieving the less energy consumption possible for electric vehicles.

The rest of the paper is structured as follows. The problem formulation that motivated this research is described in section II. Section III presents the experimental platform and its adaptation for carrying out the proposed work. The speed profile algorithm is proposed in section IV. Experimental results-including a comparison with a human driver-are shown on section V and, finally, the conclusions derived from this work and possible future works are discussed.

II. PROBLEM FORMULATION

This work assumes as starting point a commuter trip that everyday covers the same route from one point to another, spending the same time. We are also assuming that the vehicle is equipped with the proper sensor technology for allowing it to recognize his environment and where it is localized. It means that it will be able to autonomously drive-i.e., longitudinal control-which means less energy consumption through a more efficient way of driving. For doing so, some initial considerations were taking into account, as follow:

1) The route that will be covered by the commuter has been selected as the optimal trajectory, using an intelligent algorithm [START_REF] Jurik | Energy optimal real-time navigation system[END_REF]. This route has been selected taking into account the distance to cover, the amount of intersections to pass through or how heavy the traffic is by the time that the route computation is requested, among other parameters.

2) The vehicle is traveling through a fluid traffic flow and, as a consequence, the precedent vehicle impact has not been taken into consideration on this first approach. More vehicle scenarios will be considered for future improvements. 3) In order to focus on the longitudinal control and speed profile generation, the possible obstacles that would be faced by the vehicle-i.e. pedestrians, cyclist or others cars-are not taken into account when the algorithm is executed. This approach is focused on a more efficient driving that comes after having a route selected. Real time control and interaction with other elements are proposed as future works. With this premises, this work tries to find an intelligent and modular algorithm for improving the way that a selected route is covered in terms of efficiency. The unique requirement is that the trip duration has to be the same. Then, the goal is to avoid unnecessary power peaks as well as wasting more power than required on a certain situation.

III. EXPERIMENTAL PLATFORM

This section describes the experimental platform used for the validation on the real tests of the proposed algorithm. This platform is a four-wheeled robotized vehicle equipped with a perception system [START_REF] Xia | Cyberc3: A prototype cybernetic transportation system for urban applications[END_REF], using a SLAM (Simultaneous Localization and Mapping) technique for vehicle location. It allows the vehicle to compare the information perceived by the LIDAR (Light Detection and Ranging) with a preloaded map containing the route, deducing a match degree between them. Thanks to this, the vehicle is able to know which are both its position and its heading angle. The platform is also equipped with two lasers for obstacle detection. Two encoders are installed over the rear wheels and connected to the main computer through a Controller Area Network (CAN) interface. Figure 1 shows a picture of the electric Cybercar used. After the perception stage where the vehicle is located, the decision stage is responsible of the intelligence of the vehicle. In previous works, automated driving capabilities were developed to follow the predefined trajectory without any human intervention (see [START_REF] Rastelli | Dynamic trajectory generation using continuous-curvature algorithms for door to door assistance vehicles[END_REF] and [START_REF] Gonzalez | Control architecture for cybernetic transportation systems in urban environments[END_REF] for details). Decision stage is also able to dynamically change the route when an obstacle is detected or a lane-change is required. The control block manages the orders given at this stage, comparing them with the vehicle position in order to generate the final commands (lateral and longitudinal control) to the actuators (motor driver, steering wheel). This work aims on modifying the longitudinal control in order to improve the energy consumption taking into account the road layout.

IV. SPEED PROFILE ALGORITHM

This section describes the proposed algorithm for reducing the energy consumption of a vehicle from one point to another taking a specific time. This implemented algorithm considers the road layout that the vehicle will face on the selected route and the distance to cover. It also takes into account the vehicle dynamics to compute the driving losses using an energy model before starting the route.

A. Algorithm Inputs

The algorithm uses different information coming from the pre-defined route to be covered and the on-board sensor systems. All this information is received from the perception and decision stages. The inputs required by this speed profile algorithm are specified below:

1) Pre-loaded Route: In the same way that a commercial GPS works, it is necessary to have a preloaded file containing the points that describe the trajectory to be followed by the vehicle in a 3D map. The processing of this method is based on a pre-loaded data that have been added to the SLAM technique for localization. This control points are used to determine the different stretches to generate the proper reference speed in each of them. Altitude is used to compute the road layout, which is required in order to apply the energy model.

2) Vehicle Properties: Some of the vehicle characteristics are required for the energy model. In order to estimate the total energy needed by the vehicle to complete the route, a dynamic model is applied, which describes the amount of energy required to go on a certain speed, over a route with a specified slope and length. Equation 1 describes this model, where M is the vehicle mass, g is the gravitational acceleration, θ denotes the road slope, v the vehicle speed, A f the vehicle front area, C d and µ r the aerodynamic drag and rolling resistance coefficient respectively. As described on [START_REF] Jurik | Energy optimal real-time navigation system: Application to a hybrid electrical vehicle[END_REF], equation 1 estimates the energy demand for each route segment taking into account both aerodynamic, gravitational and rolling resistance losses.

E i (θ i , d i , v i ) = ρ 2 •A f •C d •v 2 i •d i +M •g•d i •(µ r • cos θ i + sin θ i ) (1)
3) Instant Power Consumed: For determining the power consumption on real time, an instant power measurement system was installed on the prototype. It allows to know how much power is demanding the vehicle on real time, permitting to check the behavior of the solution proposed. For the instant power measurement is required both the current and voltage from the batteries. For this reason, two modular systems were added to measure both variables. The first one is composed by a Hall Effect sensor biased by a ±12 volt power source and its main function is to translate the output current from the batteries to a low current output with a rate of 2000 : 1. Then, the current is conducted through a load whose voltage drop is measured by an I/O analog card connected to the PC via an USB port. For the voltage measurement a voltage divisor is implemented with a ground insulator on its output. The signal is acquired by the same card, being processed by the PC. Figure 2 shows the electronic scheme used for the installation of the current and voltage measurement systems.

B. Speed Profile Generation

After loading the route selected by the commuter and the vehicle dynamic characteristics for the calculations, the next step is to divide the trajectory into segments-as shown on figure 3-with the same road slope. These route divisions have the same theoretical power consumption required. Once the division is set, the total energy required for the whole route is calculated using (equation 1) and adding every segment contribution to the total. Then, the whole energy estimated for completing the route divided by the time is determined. It brings as result the mean power that must be consumed all along the path, according to equation 2. Therefore, the following conditions are satisfied: the time passed from the start until the end is the desired by the commuter and the power consumed through the whole travel is not greater than the required, so it is possible to get the best energy consumption rate.

P = n ∑ i=0 E i (θ i , d i , v i ) t total (2)
In order to obtain a more regular speed profile, a weighing process based on the distance and the difference between the segment slope and the mean slope (equation 3) is applied, assigning to each segment the theoretical best power that should be consumed along it-see equation 4. This consideration brings as result a greater power consumption assigned to upgrade segments and, on the other hand, downgrades are planned with less power cost. It produces a speed profile plan more stable with smaller differences among different layouts. This is also useful because makes the speed profile more likely as a common driver performance would do, bringing more comfort to the passenger.

θ = n ∑ i=0 θ i • d i d total
(3)

P i = P • (1 + k • (θ i -θ)) (4)
Once each segment has its optimal power assigned, based on the road slope and the vehicle properties, the ideal speed is calculated by the employment of the energy model. This process sets the velocity profile for the entire route, completing it in the desired time. Figure 4 shows the complete algorithm flowchart, where each part is explained below. This flowchart describes the complete algorithm performance. First it starts reading the vehicle properties and the route characteristics, so it is able to apply the energetic model and estimate the optimal speed profile for the whole trajectory. It is also described how the vehicle generates the reference speed, once it has started the travel. 1) Dynamic Operation: Once the vehicle starts the route, the developed module is in charge of determining on which segment the vehicle is currently driving. Then, the optimal segment speed is generated according to the previously defined algorithm. When a new segment is detected, a smooth transition function is applied-see equation 5-to carry out an efficient velocity change.

v i+1 = v i + K • t • g • [ cos ( π 2 -∆θ ) + µ r • sin ( π 2 -∆θ )] (5)

V. EXPERIMENTAL RESULTS

This section presents the results obtained with the proposed algorithm using a pre-defined route. The experiments were carried out at Inria's facilities. The selected scenario includes upgrade, flat and downgrade segments, turning it into a good candidate for testing the performance of the proposed system. The figure 5 shows an image taken from Google Earth with the route selected (yellow line) and the elevation profile. The tests were done using a Cybus prototype, measuring the reference speed, the real speed and the instant power consumed by the vehicle on each travel. In order to have similar power conditions on each scenario, all the tests were made by employing fully-charged batteries.

A. Comparative analysis

Using the trajectory showed on figure 5, two experiments are proposed to test the performance of the proposed system. The first scenario is the one explained on section II, where a commuter drives manually a vehicle, attempting to maintain a constant reference speed from a starting point to a destination, which is supposed to be a very efficient way to drive. The second scenario is when the intelligent speed profile algorithm is applied over the longitudinal control, acting over the energy consumption. The algorithm was tested multiple times for assuring its repeatability and because of this, one average and representative test was selected to be presented graphically in this work. The conditions in each scenario are the same, either the trajectory or the time established to cover the whole route. On each of the case graphics, the reference speed-red, the measured speed-blue-and the instant power consumed-green-are plotted. 1) Manual Driving: The figure 6 illustrates the reference speed generated by the driver-that is sent to the low level control, the measured speed and the instant power consumed by the vehicle. The power consumed results to be very irregular, leading to an inefficient driving way and causing higher energy consumption.

2) Speed Profile Algorithm: The route described on figure 5 is divided in segments in order to apply the Intelligent Speed Profile Algorithm (ISPA). Figure 7 shows how the reference speed changes along the trajectory, trying to pursuit the best energy consumption over each road layout. A good response from the energy consumption perspective is shown. However, there are some power peaks caused by the low level control on the longer downgrade segment.

3) Results Analysis: Figure 6 shows the performance of the vehicle while it is driven manually. The reference speed managed by the driver results irregular compared to the one generated by the automated system proposed. It means that the power consumed in the first case is higher since the speed control is not efficient. On the other hand, the speed reference given by the ISPA is stable and does not oscillates as much as in the first case, which means that the energy management results even more efficient and proper to the road layout.

Table I shows the energy consumption for each of the routes. The ISPA algorithm saves 17.08% compared to the manual driver. The amount of energy consumed by the car was determined by integrating the instant power consumed while the vehicle was traveling along the route on each of the scenarios. The power peaks and the speed control problems observed on both figures (6 and 7) are due to the low level controller. The reason is that this experimental platform has been designed for applications which do not require a high precision speed control. Actuators and sensors were selected for this architecture in order to get a low cost platform.

VI. CONCLUSIONS AND FUTURE WORKS

This paper presents an algorithm for performing longitudinal vehicle control on an EVs with energy saving consideration. A solution to the electric vehicles autonomy problem is proposed by applying the concept of eco-driving and generating a reference speed that achieves big energy savings without changing any of the vehicle main resources (batteries, motor and power electronics).

This research proposes an intelligent algorithm able to generate a speed profile. It would make the vehicle cover a route in a more efficient way than a common driver would. The algorithm acts over a pre-defined trajectory and the time specified to cover it. The goal is to get the lower energy consumption in the fixed time.

As future work, traffic data will be introduced to modify the algorithm according to the traffic changes on real time. Additionally, higher speed ranges will be also considered. Finally, a high level system that modifies the speed reference in order to have a more efficient performance, solving the low level control problems will be investigated.

Fig. 1 .

 1 Fig. 1. Cybus vehicle, driving on INRIA's facilities

Fig. 2 .

 2 Fig. 2. Improvements required to the control architecture, acquisition and conditioning stage.

Fig. 3 .

 3 Fig. 3. Road splitting illustration, in order to apply the energetic model over each segment.

Fig. 4 .

 4 Fig. 4. Complete algorithm flowchart.

Fig. 5 .

 5 Fig. 5. INRIA's facilities with the route remarked on yellow.

Fig. 6 .Fig. 7 .

 67 Fig.6. Reference speed generated manually by the driver; and instant power consumed by the vehicle.

ACKNOWLEDGMENTS

Authors express their gratitude to the EU CityMobil-2 project and the RITS Team for its support in the development of this work.