M Simon

Mme Elisa Vianello

M Blaise

Philémon, Florent, Cyril Marie Marc

Gaëlle, Gaël, Éric Anne Paul

Jean-Marie Fanny

Dodji

Mehrdad

REMERCIEMENTS

Après plus de trois ans de travail, ma thèse s'achève sur ces quelques pages. Celle-ci n'aurait pas pu avoir lieu sans la participation de différentes personnes que j'aimerai citer ici. Tout d'abord, je voudrais remercier mon directeur de thèse, Blaise Yvert, qui a été à mes côtés tout au long de ce travail laborieux mais riche d'enseignements. J'ai pu apprécier son optimisme et ses encouragements. Je remercie également mes co-encadrants, Elisa Vianello et Jean-François Bêche, avec qui j'ai pu avoir des discussions intéressantes à différents stades de ma thèse.

Je remercie Luca Berdondini et Simon Thorpe, qui ont accepté d'être rapporteurs pour cette thèse, pour leur relecture attentive de ce manuscrit et leurs commentaires constructifs. Je remercie de même les autres membres du jury, Elisa Vianello, Jean-François Bêche et Christophe Pouzat, pour avoir accepté d'écouter et d'évaluer mon travail. J'aimerais également remercier toutes les personnes avec qui j'ai pu collaborer durant mon travail. Je remercie Thilo Werner, encadré par Elisa Vianello, avec qui j'ai pu avoir de quelques échanges au début de ma thèse. Je remercie Takashi Kohno et Timothée Levi, qui ont eu la gentillesse de m'accueillir dans leur laboratoire au Japon pendant deux semaines, pour travailler sur une implémentation FPGA. Je remercie également Joël Fournier que j'ai eu le plaisir d'encadrer pour un stage sur ce même

DEVELOPMENT OF AN STDP NEURAL NETWORK FOR UNSUPERVISED ONLINE SPIKE-SORTING

PHD THESIS BY MARIE BERNERT, SUPERVISED BY BLAISE YVERT, DEFENDED ON JANUARY 24 TH 2019 ABSTRACT Pattern recognition is a fundamental task for living beings and is perform very efficiently by the brain. Artificial deep neural networks are making quick progress in reproducing these performance and have many applications such as image recognition or natural language processing. However, they require extensive training on large datasets and heavy computations. A promising alternative are spiking neural networks, which closely mimic what happens in the brain, with spiking neurons and spiketiming-dependent plasticity (STDP). They are able to perform unsupervised learning and have been used for visual or auditory pattern recognition. However, for now applications using STDP networks lag far behind classical deep learning. Developing new applications for this kind of networks is all the more at stake that they could be implemented in low power neuromorphic hardware that currently undergoes important developments, in particular with analog miniaturized memristive devices able to mimic synaptic plasticity. In this work, we chose to develop an STDP neural network to perform a specific task: spike-sorting, which is a crucial problem in neuroscience. Brain implants based on microelectrode arrays are able to record the activity of individual neurons, appearing in the recorded signal as peak potential variations called action potentials. However, several neurons can be recorded by the same electrode. The goal of spike-sorting is to extract and separate the activities of different neural cells from a common extracellular recording taking advantage of the fact that the shape of an action potential on an electrode depends on the neuron it stems from. Thus spike-sorting can be seen as an unsupervised pattern recognition task where the goal is to detect and classify different waveforms. Most classical spike-sorting approaches use three separated steps: detecting all action potentials in the signal, extracting features characterizing their shapes, and separating these features into clusters that should correspond to different neural cells. Though online methods exists, most widespread spike-sorting methods are offline or require an offline preprocessing step, which is not compatible with online application such as Brain-computer interfaces (BCI). Moreover, the development of ever larger microelectrode arrays creates a need for fully automatic and computationally efficient algorithms. Using an STDP network brings a new approach to meet these requirements. We designed a network taking the electrode signal as an input and giving out spikes that correspond to the spiking activity of the recorded neural cells. It is organized into several layers, designed to achieve different processing steps, connected in a feedforward way. The first layer, composed of neurons acting as sensory neurons, convert the input signal into spike train. The following layers are able to learn patterns from the previous layer thanks to STDP rules. Each layer implement different mechanisms that improve their performances, such as resource-dependent STDP, intrinsic plasticity, plasticity triggered by inhibition, or neuron models having rebound spiking properties. An attention mechanism has been implemented to make the network sensitive only to parts of the signal containing action potentials. This network was first designed to process data from a single electrode, and then adapted to process data from multiple electrodes. It has been tested both on simulated data, which allowed to compare the network output to the known ground truth, and on real extracellular recordings associated with intracellular recordings that give an incomplete ground truth. Different versions of the network were evaluated and compared to other spike-sorting algorithms, and found to give very satisfying results. Following these software simulations, we initiated an FPGA implementation of the method, which constitutes a first step towards embedded neuromorphic implementation.

RESUME

La reconnaissance de motifs est une tâche cruciale pour les êtres vivants, exécutée avec efficacité par le cerveau. Les réseaux de neurones profonds artificiels reproduisent de mieux en mieux ces performances, avec des applications telles que la reconnaissance d'images ou le traitement du langage. Ils nécessitent cependant un apprentissage intensif sur de grands jeux de données et couteux en calculs. Les réseaux de neurones à impulsions, plus proches du fonctionnement du cerveau avec des neurones émettant des impulsions et des lois d'apprentissage dites STDP dépendant du temps entre deux impulsions, constituent une alternative intéressante. Ils permettent un apprentissage non supervisé et ont déjà été utilisés pour la reconnaissance visuelle ou auditive, mais les applications restent limitées par rapport à l'apprentissage profond classique. Il est d'autant plus intéressant de développer de nouvelles applications pour ces réseaux qu'ils peuvent être implémentés sur des circuits neuromorphiques connaissant aujourd'hui des développements importants, notamment avec les composants analogiques « memristifs » qui miment la plasticité synaptique. Ici, nous avons choisi de développer un réseau STDP pour un problème crucial en neuroscience: le spike-sorting. Les implants cérébraux composés de matrices de microélectrode permettent d'enregistrer l'activité individuelle de multiples neurones, prenant la forme de pics de potentiel dans le signal, appelés potentiels d'action. Une même électrode enregistre l'activité de plusieurs neurones. Le spike-sorting a pour but de détecter et trier cette activité, en utilisant le fait que la forme d'un potentiel d'action dépend du neurone qui l'a émis. Il s'agit donc d'un problème de reconnaissance de motifs non supervisée. Les méthodes classiques de spike-sorting consistent en trois étapes : la détection des potentiels d'action, l'extraction de traits caractéristiques de leurs formes, et le tri de ces caractéristiques en groupes correspondant alors aux différentes cellules neurales. Bien que les méthodes onlines existent, les méthodes les plus répandues nécessitent un traitement offline, qui n'est pas compatible avec les applications temps réelles telles que les interfaces cerveau-machine (BCI). De plus, le développement de matrices de microélectrodes toujours plus denses nécessite des méthodes automatiques et efficaces. Utiliser un réseau STDP apporte une nouvelle méthode pour répondre à ces besoins. Le réseau que nous avons conçu prend en entrée le signal de l'électrode et produit en sortie un train d'impulsions qui correspond à l'activité des cellules enregistrées. Il est organisé en différentes couches, connectées en série, chacune effectuant une étape du traitement. La première couche, constituée de neurones senseurs, convertit le signal d'entrée en train d'impulsions. Les couches suivantes apprennent les motifs générés par la couche précédente grâce aux lois STDP. Chaque couche est améliorée par l'implémentation de différents mécanismes, tels que le STDP avec ressources, l'adaptation de seuil, la plasticité déclenchée par l'inhibition, ou un modèle de neurone déchargeant par rebond. Un mécanisme d'attention permet au réseau de ne traiter que les parties du signal contenant des potentiels d'action. Ce réseau a été conçu dans un premier temps pour traiter des données mono-électrode, puis adapté pour traiter des signaux provenant d'électrodes multiples. Il a été testé d'abord sur des données simulées qui permettent de comparer la sortie du réseau à la vérité, puis sur des enregistrements réels de microélectrodes associés à des enregistrements intracellulaires donnant une vérité partielle. Les différentes versions du réseau ont été ainsi évaluées et comparées à d'autres algorithmes, donnant des résultats très satisfaisants. Suite à ces résultats simulés sur ordinateur, nous avons travaillé à une implémentation FPGA, constituant une première étape vers une implémentation embarquée neuromorphique.

LIST OF FIGURES

LIST OF TABLES

I. INTRODUCTION: CONTEXT AND GOAL OF THE THESIS

Stakes of recording the brain

Understanding the brain is maybe one of the most interesting challenges for scientists nowadays. Will our intelligence, some day, be able to understand its own mechanics? The long path towards this goal begins by observing what is going on in our head. The human brain is constituted of tens of billions of neurons, and thousands of times more synapses connecting them. These two elements constitute the computational bricks of the brain, connected together in an incredibly complex network that neuroscientists are trying to unravel. Anatomical observations allow us to understand how the neurons and their connections are organized in the brain. This makes us progress towards understanding how it works, but needs to be completed by functional observations, in other words measuring the neuronal activity, to understand how the information is encoded, transmitted and processed in the brain.

Neuronal activity is both electric and chemical. At its resting state, a neuron has its membrane potential at an equilibrium (around -70 mV). This potential then varies depending on the activity received from other neurons through synapses. When it increases up to a threshold value, the neuron fires. Different voltage-gated ion channels become activated, which modifies the transmembrane currents and triggers an abrupt increase of the neuron's potential followed by an abrupt decrease before returning to its resting state. This phenomenon is called action potential or spike. This action potential propagates along the neuron's axon, through similar ion channels' mechanisms. The axon is terminated by synapses through which the action potential is chemically transmitted to other neurons triggering the opening of channels that in turn modify the receptor neurons' potentials (Figure I-1). Action potentials are thus the nervous system's information carrier. Recording the spiking activity of different neurons is thus crucial to understand how the information is encoded and processed in the brain.

Figure I-1: Emission and propagation of an action potential in neurons

It is known that specific neural cells participate in the encoding of specific behaviors, stimuli or concepts. For example, in the visual cortex, some neural cells are sensitive to visual stimuli oriented in a specific direction, and different cells have different preferred directions. Similarly, it has been found that, in the motor cortex, some neurons code for different directions of the arm movement and that a complex movement can be decoded from the collective firing of a group of cells (Georgopoulos et al. 1986;Schwartz 1994). Spatial representation and navigation is encoded by neural cells in the hippocampus, called place cells, that code for specific locations in an environment [START_REF] Moser | Place Cells, Grid Cells, and the Brain's Spatial Representation System[END_REF].

Another study also shows that single facial characteristics are encoded by single cells [START_REF] Chang | The Code for Facial Identity in the Primate Brain[END_REF]. Recording individual neural spiking activity is thus crucial to analyze how neural information is encoded. Recording several neurons simultaneously also allows to understand how the spiking activity of different neurons interact to encode and to process more complex concepts [START_REF] Buzsáki | Neural Syntax: Cell Assemblies, Synapsembles, and Readers[END_REF].

Besides improving our understanding of brain functions, decoding the information from neural recordings has practical applications such as neuro-prostheses and neuro-rehabilitation with Brain Computer Interfaces (BCIs). BCIs consist in using the decoded activity from a neural recording to control an artificial actuator. Hopes are to be able to replace defective functions with neuroprostheses, for example with a BCI controlling a robotic arm for tetraplegic patients [START_REF] Hochberg | Reach and grasp by people with tetraplegia using a neurally controlled robotic arm[END_REF] or a BCI controlling a speech synthesizer for aphasic patients [START_REF] Bocquelet | Key considerations in designing a speech brain-computer interface[END_REF]. Decoding brain activity can also be used for neuro-rehabilitation, which allows to restore motor function by stimulating muscles according to the decoded activity using either electrodes on the surface of the skin [START_REF] Bouton | Restoring cortical control of functional movement in a human with quadriplegia[END_REF] or implanted ones [START_REF] Ajiboye | Restoration of reaching and grasping movements through brain-controlled muscle stimulation in a person with tetraplegia: a proof-of-concept demonstration[END_REF]. In these systems, the processing of neural activity recording should be done in real-time. Moreover, the recording's quality need to be high enough to ensure the possibility to decode complex behaviors. Ideally, this implies being able to record individual spiking activity from a large number of neurons. Experimentally, it has been shown that the number of cells recorded actually improves the overall decoding quality [START_REF] Wessberg | Real-time prediction of hand trajectory by ensembles of cortical neurons in primates[END_REF][START_REF] Ifft | A Brain-Machine Interface Enables Bimanual Arm Movements in Monkeys[END_REF].

Recording technologies have evolved to meet these requirements. Nowadays, microelectrode arrays (MEA) are able to record the individual activity of hundreds of neurons and the technology is still improving. To fully take advantage of these advances, new algorithms have to be developed for efficiently processing the corresponding data online and in real time.

Microelectrode arrays recordings and spike-sorting.

Neural activity recording technologies are diverse, each with their own advantages and drawbacks. Among the most known non-invasive methods, we can cite EEG, MEG, fMRI or PET scan. EEG measures the electrical fields generated on the scalp by the coordinated activity of neurons. MEG measures the magnetic fields outside the head. fMRI and PET scan measure metabolic changes such as the blood flow or oxygenation variations through the brain, correlated with neural activity. These non-invasive methods are useful for diagnostics and functional experiments but do not allow the recording of individual cells. In contrast, the patch clamp method consists in placing a micropipette in contact with a neural cell's membrane to record its intracellular potential. This method allows to record individual cells, but only a few neurons can be recorded simultaneously and the method is not suitable for in vivo et al., 2013;Angotzi, Malerba, Zucca, & Berdondini, 2015;Lopez et al., 2016;Pothof et al., 2016;Rios, Lubenov, Chi, Roukes, & Siapas, 2016;Seidl et al., 2012) (Figure I-2.b). The increasing number electrodes on the same device allows on one hand to record more and more neurons simultaneously, opening opportunities for complex decoding.

On the other hand, dense MEAs generate a huge flow of data, which require suitable processing algorithms, especially for real-time applications. Ideally, most processing should be done at the level of the electrode, to reduce the flow of data to transmit to the rest of the experimental chain. In particular, one type of processing that is usually done on MEA recordings is spike-sorting. As stated previously, an extracellular electrode is able to record the spiking activity of nearby neurons. Thus, the activity of the few neural cells surrounding the electrode is recorded on the same electrode signal by superposition. As the shape of the action potential recorded by the electrode depends on how the cell is positioned relatively to the electrode, it is possible to separate the spiking activity of the few recorded cells thanks to the different action potential shapes. The process is called spike-sorting. Many algorithms exist for spike-sorting, reviewed in Section II. Most of them are satisfying for offline processing and for a limited number of electrodes. However close-loop experiments require an immediate feedback from the neural data decoding, which implies real-time spike-sorting. This constraint is even more difficult to meet when using a high number of electrodes, as it increases the amount of data to process. Beside online applications, a high number of electrodes also make human supervision on spike-sorting impossible, thus requiring fully automatic methods. Therefore there is a need for new efficient online and automatic spike-sorting methods.

Pattern recognition and artificial neural networks

The principle of spike-sorting is to detect action potentials in a signal, and to classify them according to their shapes. Spike-sorting is thus a pattern recognition problem, a field that is widely explored in artificial intelligence. The most known forms of pattern recognition are image processing or natural language processing. Many solutions exists to solve this kind of problem, sometimes inspired by the natural ability of our brain to solve these tasks.

In particular, artificial neural networks are very popular in artificial intelligence and widely used for pattern recognition tasks. They are usually constituted of formal neurons, modeled by a mathematical activation function transforming a real input value, which is the sum of other neurons' outputs, into a real output value (Figure I-3.a). The neurons are connected together through synapses characterized by a weight, which defines how much a presynaptic neuron influences the postsynaptic neuron. The mathematical properties of these networks make the synapses' weights optimization possible through a gradient descent method, called backpropagation (Rumelhart et al. 1986), so that the network 'learns' to achieve a specific task. This neural network model is very convenient but not biologically realistic. In contrast, spiking neural networks (SNNs) use spikes to convey information. The spikes are modeled as discrete binary events transmitted between neurons through synapses. A spiking neuron receives spikes from inputs neurons, integrates them through a temporal dynamic and in turn emits other spikes . Though SNNs can also be trained using a backpropagation algorithm [START_REF] Bohte | Error-backpropagation in temporally encoded networks of spiking neurons[END_REF], it is possible to use biologically realistic learning rules. In particular, it has been shown experimentally, by stimulating pairs of real neurons, that the weight of a synapse changes depending on the time difference between a postsynaptic spike and a presynaptic spike (Bi & Poo 1998). This phenomenon is called spike-timing-dependent plasticity (STDP) (Figure I-4) and can be implemented in SNNs, which are then called STDP networks. STDP rules allows spiking networks to learn patterns by positively reinforcing synapses contributing to the postsynaptic activity. A review of STDP network models is given in Section III. STPD networks have been used for a few pattern recognition tasks, but applications to real world problems are still sparse. Indeed, there is at the moment no standard way to train an STDP network to achieve a specific task. However, STDP networks have other advantages compared to formal neural networks. First, STDP networks naturally allow unsupervised learning, which is essential for spikesorting as the ground truth is unknown. Second, formal neural networks usually require training on a huge amount of data, whereas STDP networks can learn on few examples.

The advent of neuromorphic hardware for low-power computing

Though artificial neural networks are widely used for pattern recognition tasks, their main inconvenient is that they involve many neurons and many synapses working in parallel, which is computationally very demanding, especially when they are executed on traditional computers with sequential architectures. This is why neuromorphic hardware, whose purpose is to natively implement neural networks in integrated chips, currently undergoes important developments. In particular, memristive components are being developed, which are able to mimic a synapse with STDP at a miniaturized scale (Figure I-5) [START_REF] Jo | Nanoscale Memristor Device as Synapse in Neuromorphic Systems[END_REF]Indiveri et al. 2013;[START_REF] Indiveri | Neuromorphic architectures for spiking deep neural networks[END_REF][START_REF] Park | Electronic system with memristive synapses for pattern recognition[END_REF]Saïghi et al. 2015;[START_REF] Rajendran | Neuromorphic Computing Based on Emerging Memory Technologies[END_REF][START_REF] Barbera | Interplay of multiple synaptic plasticity features in filamentary memristive devices for neuromorphic computing[END_REF]Sourikopoulos et al. 2017). This kind of devices thus opens an opportunity for STPD networks to be implemented in low-power miniaturized devices. Therefore both the unsupervised learning properties of STDP networks and their possibility to be implemented in such devices offer important perspectives for pattern recognition in the future. Goal of the thesis: online spike-sorting with an STDP network

In this context, the goal of this thesis work was to design a new unsupervised and online method for spike-sorting, using an STDP network. In contrast to (Zhang et al. 2015), where an STDP network was used as part of the spike-sorting method, here we aim at performing the entire spike-sorting process with an STDP network, without any pre-processing or post-processing. We also want the network to process the electrode signal online. This means that the electrode signal is directly streamed as an input to the network, and that the output spike train should correspond to the sorted spiking activity, each output spike corresponding to an action potential in the electrode signal. A preliminary study was done towards this goal (Werner et al. 2016). This network was tested on data with very high signal-tonoise ratio (SNR) but was not suitable at typical SNR found in cortical neural recordings. Moreover, it required a bank of filters to decompose the input signal, which added computation cost to the method.

Here, no further processing is applied beyond the initial band-pass filter eliminating the slowly evolving local field potentials (LFP).

Designing such a network implied finding a suitable network structure, choosing neurons and synapses models for each functional parts of the network, and parameterizing correctly each element to achieve the desired goal. My work was mainly focused on processing single electrode signals, which constitutes the first step towards processing richer recordings. However, a design adapted for multiple electrodes was also tested. The problem was treated from an algorithmic point of view, leaving aside the hardware implementation but keeping in mind that the designed network should be adaptable to a neuromorphic implementation. Nevertheless, a preliminary work was done to implement the network

II. SPIKE-SORTING STATE OF THE ART

Spike-sorting principle

The signal recorded by an extracellular microelectrode contains a local field potential, action potentials, and noise. The local field potential is believed to correspond to the average current generated by cellular elements in a local area, mostly reflecting synaptic activity [START_REF] Buzsáki | The origin of extracellular fields and currents--EEG, ECoG, LFP and spikes[END_REF].

It varies slowly and can be removed with a high-pass filter (above about 200-300 Hz). After filtering, we obtain a noisy signal that contains action potentials emitted by different neurons close to the electrode. The role of spike-sorting is to detect and sort these action potentials, to obtain the individual spiking activity of each neural cell. The principle of spike-sorting mainly relies on the fact that the shape of an action potential recorded by an electrode depends on the geometries of both the neuron and the electrode, and their relative positions. Thus, on an electrode signal, action potentials with the same shape most probably come from the same neuron whereas action potentials with different shapes come from different neurons (Figure II-1). Spike-sorting methods thus use the action potential waveforms, or features extracted from these waveforms to sort the activities of different neurons. Possibly, additional information such as timing information like inter-spike interval [START_REF] Delescluse | Efficient spike-sorting of multi-state neurons using inter-spike intervals information[END_REF] or spatial information [START_REF] Rossant | Spike sorting for large, dense electrode arrays[END_REF][START_REF] Hilgen | Unsupervised Spike Sorting for Large-Scale, High-Density Multielectrode Arrays[END_REF]) can be used.

Classical methods

Once the signal has been correctly filtered, it is constituted of the superposition of noise and action potentials emitted by different neurons. Most spike-sorting methods are decomposed into three distinct steps: a detection step to find the action potentials in the signal, a feature extraction step to extract features characterizing the detected action potential shapes, and a clustering step to group the feature vectors into different clusters corresponding to the different neural cells recorded in the signal . This section describes the most used algorithms for each of these steps, as well as some more global approaches to spike-sorting. A good review can also be found in [START_REF] Rey | Past, present and future of spike sorting techniques[END_REF].

Detection

The simplest way to detect action potentials in a microelectrode signal is to use a simple threshold. Each time the recorded potential crosses this threshold, it is considered that an action potential is present. This threshold can be positive or negative or both depending on the expected sign of the action potential peak. The threshold is most of the time chosen between 3 and 4 times the standard deviation of the noise, which gives a good compromise between false negatives and false positives. A first step is thus to estimate this standard deviation. In case of a Gaussian noise, the standard deviation can be estimated based on the median, with 𝜎 = 𝑚𝑒𝑑𝑖𝑎𝑛(|𝑠|) 0.6745 ⁄ , where s is the signal (Quiroga et al. 2004;[START_REF] Rossant | Spike sorting for large, dense electrode arrays[END_REF]). This method using median is robust to the presence of action potentials in the signal.

More sophisticated methods apply a preprocessing to the signal before thresholding. The idea is to better take into account the characteristics of an action potential. Indeed even if their shape differ from one neuron to another, they share common properties such as a similar frequency content, or similar features in their shapes. Some examples of such methods are the use of an energy operator (Rutishauser et al. 2006), or the use of wavelets [START_REF] Nenadic | Spike detection using the continuous wavelet transform[END_REF][START_REF] Escola | Wavelet-based scale-dependent detection of neurological action potentials[END_REF]. In case of a template matching method (see Section II.B.4), it is also possible to use the templates for detection [START_REF] Bankman | Optimal Detection, Classification, and Superposition Resolution in Neural Waveform Recordings[END_REF]. Template matching methods are often used in combination with a preliminary simpler detection method to establish the templates. [START_REF] Delescluse | Efficient spike-sorting of multi-state neurons using inter-spike intervals information[END_REF]) used a detection method where a single template was computed for detection.

In case of multiple electrodes, the detection has to take into account spatial information. Indeed an action potential is recorded on several neighboring electrodes but not all electrodes. Thus the signal shape is relevant only on the electrodes where the action potential appears. The number of electrodes detecting an action potential from a neural cell can vary depending on the position and nature of the cell. As an example, [START_REF] Rossant | Spike sorting for large, dense electrode arrays[END_REF]) used a detection algorithm where a high threshold is applied first to detect the presence of action potential. Then a lower threshold is applied and spatially connected components containing at least one high-threshold crossing potential are considered to correspond to one action potential. This dual-threshold method allows to limit false positives with the high threshold while merging correctly all electrodes involved in a same action potential thanks to the low threshold.

Once an action potential has been detected, its shape is stored for further processing. To be coherent between two occurrences of a similar action potential, it is necessary to always store the same number of samples around a reference time point. This reference point is often chosen as the maximum or minimum of the action potential, which is more robust than taking the point at which the waveform crosses the threshold as it can vary because of noise. Additionally, the signal is often up-sampled before alignment to be robust to sampling jitter.

2.

Feature extraction Some spike-sorting algorithms such as in (Rutishauser et al. 2006) or [START_REF] Pouzat | Using noise signature to optimize spike-sorting and to assess neuronal classification quality[END_REF] or more generally template matching algorithms use directly the whole action potential waveform for clustering. However this require to manipulate high-dimensional vectors, especially in the case of multiple electrodes, which can impair the execution time and even the clustering quality, a phenomenon known as the "curse of dimensionality". For this reason most spike-sorting methods use a reduced number of features, extracted from the waveforms. This can be very simple features such as the amplitude or the width of the waveform. As an example, [START_REF] Delescluse | Efficient spike-sorting of multi-state neurons using inter-spike intervals information[END_REF]) use the peak amplitudes on the four recording sites of a tetrode. Other methods use more complex features such as wavelets (Quiroga et al. 2004;[START_REF] Hulata | A method for spike sorting and detection based on wavelet packets and Shannon's mutual information[END_REF]. It is thus possible to use predefined features, however most methods use an automatic dimensionality reduction algorithm such as principal component analysis (PCA) [START_REF] Shoham | Robust, automatic spike sorting using mixtures of multivariate t-distributions[END_REF][START_REF] Rossant | Spike sorting for large, dense electrode arrays[END_REF][START_REF] Hilgen | Unsupervised Spike Sorting for Large-Scale, High-Density Multielectrode Arrays[END_REF], which projects the waveforms into a subspace that accounts for most of the variations between waveforms. In case of multiple electrodes, spatial information can also be used for clustering. In [START_REF] Rossant | Spike sorting for large, dense electrode arrays[END_REF]), a vector indicating on which electrodes the action potential has been detected is used, in addition to PCA features. In [START_REF] Hilgen | Unsupervised Spike Sorting for Large-Scale, High-Density Multielectrode Arrays[END_REF], the spatial position of each action potential is estimated through a method based on barycenters and used as a feature.

Clustering

After the detection and the features extraction steps, each detected action potential is represented by a vector in an N-dimensional space. Two action potentials that are close to each other in this space are likely to have a similar waveform and thus to stem from the same neural cell. The goal is thus to sort the feature vectors into clusters, corresponding to action potential waveforms stemming from different neural cells. This last step is therefore a clustering problem, for which many solutions exist.

One classical method often used in spike-sorting is the expectation-maximization algorithm. The data to cluster is modelled as stemming from a mixture of a given stochastic distribution, most of the time a Gaussian distribution. This is an iterative algorithm, where the parameters of the data distribution are updated at each iteration. In the expectation step, each data point's label is estimated given the current model parameters. Then in the maximization step, the parameters are optimized to best fit the estimated labels. These two steps are repeated until convergence (Figure II-3) has been achieved. An improved version of this algorithm has been used for example in [START_REF] Rossant | Spike sorting for large, dense electrode arrays[END_REF], where each data point is associated to a mask giving information about which dimensions are the most relevant for clustering. [START_REF] Shoham | Robust, automatic spike sorting using mixtures of multivariate t-distributions[END_REF]) also use the expectation-maximization algorithm on a mixture of t-distributions. Another widely used method is the K-mean algorithm. K-mean is also an iterative algorithm where the centers of the clusters are updated at each iteration: the data points are labeled according to the nearest center, and then the centers are updated as the barycenter of the corresponding points. This method has been used for example in [START_REF] Chah | Automated spike sorting algorithm based on Laplacian eigenmaps and k-means clustering[END_REF]. [START_REF] Oliynyk | Automatic online spike sorting with singular value decomposition and fuzzy C-mean clustering[END_REF]) use a fuzzy C-means algorithm, which is a variation of the K-means algorithm where the belonging of a point to a cluster is weighted.

The expectation-maximization as well as the K-mean algorithms require the user to initially choose the number of clusters to find. This is not satisfying for spike-sorting, as the number of clusters is a priori unknown and should ideally be determined automatically by the algorithm. A way to overcome this problem is to automatically test different numbers of cluster and select the best one afterward. However other algorithms do not have this default. This is the case for example of the mean-shift algorithm, used for example in [START_REF] Marre | Mapping a Complete Neural Population in the Retina[END_REF]) and [START_REF] Hilgen | Unsupervised Spike Sorting for Large-Scale, High-Density Multielectrode Arrays[END_REF]). Its principle is that for each point of the space, we define a window around this point. The barycenter of the data points within this window is computed and the window is shifted to be centered on this barycenter. The process is repeated until the window stabilizes on a local maxima (Figure . This way, each point can be assigned to the nearest local maxima, which correspond to a cluster (Figure . This method is almost non-parametric as the only parameter to choose is the size of the window. Another example of non-parametric method is the super-paramagnetic clustering used in (Quiroga et al. 2004). The main idea of this algorithm is to randomly assign a cluster to a point, and this new cluster assignment is propagated to neighbor points with a probability depending on the distance between points. Thus points that are close to one another will tend to change their labels together. This process is repeated many times, to ensure a relevant classification.

Finally, (Zhang et al. 2015) use an STDP network as part of their clustering algorithm. This network is inspired from pattern recognition for static inputs. It has one input layer and one output layer. Each detected waveform is given as an input of the network after being converted in an input spike train through a binary encoding of the signal values. Each time a waveform is presented, an output neuron, corresponding to a cluster, is activated. During the training phase the synaptic weights are also updated for each presented waveform. The number of clusters is monitored externally by activating or deactivating neurons of the output layer. Clusters are added if too many points are not classified and clusters are removed if their corresponding neuron does not fire for a long time.

Template matching and other global approaches

Separating the spike-sorting problem into three steps provides a simple methodology. However these steps are not independent and the performance of one step might depend on how the previous steps are solved. A global approach to the spike sorting problem might lead to better performances. As a simple example, it is much easier to detect an action potential in a noisy signal if we know its exact waveform. Most methods that use such a global approach are based on the use of templates corresponding to each different action potential waveforms, and are thus called template matching methods. These methods often rely on an initial clustering step to initialize the templates, usually using a classical three-step method as presented previously. However, it is not necessary in this preprocessing step neither to find all the action potentials, nor to assign all the found action potentials to a cluster, but simply to find the centroids of the clusters that will constitute the templates. [START_REF] Yger | Fast and accurate spike sorting in vitro and in vivo for up to thousands of electrodes[END_REF]) is an example of such a template matching method. After establishing a set of templates, the scalar product of the signal with each time-shifted version of each template is computed. The template and shift time with the highest scalar product is selected and considered as a found and classified action potential. This process is then iterated, as part of a greedy algorithm. In [START_REF] Franke | An online spike detection and spike classification algorithm capable of instantaneous resolution of overlapping spikes[END_REF], the signal is considered as the sum of the convolution of the ground truth spike trains with waveforms.

The templates estimated by the preprocessing step are used to establish filters that approximate the deconvolution of the signal. [START_REF] Ekanadham | A unified framework and method for automatic neural spike identification[END_REF]) presents an even more global approach, in the sense that they optimize simultaneously both the template waveforms and the action potential times and amplitudes, to maximize the probability to have such amplitude and spike times given the waveforms and observed signal. More precisely, they alternate a step optimizing the spike times and amplitudes and a step optimizing the action potential waveforms.

Online vs. offline methods

Most spike-sorting methods require some offline processing. Though offline spike-sorting can be used for post-experiment data analysis, it is necessary for closed-loop experiments to have an online realtime spike-sorting method. For a classical three step methods, if at least one of the steps requires an offline processing, then the spike-sorting method cannot be executed online without adaptation. The detection step can usually be executed online, as the thresholding and the possible preprocessing operator are local operations that can be applied online. Extracting predefined features from a waveform, such as its amplitude, can also be done online. On the other hand, automatic dimensionality reduction algorithms such as PCA require a consequent number of data points, and thus cannot be applied online. However, is it possible to compute the projection as an offline preprocessing step and then apply this projection online. Most clustering algorithms, such as expectation-maximization, K-means or mean-shift, also require an offline processing step for the same reason. A noticeable exception is given by (Rutishauser et al. 2006) whose clustering algorithm can be applied online. At each detection of an action potential, the corresponding waveform is assign to a cluster according to the distance with the cluster centroid, and the centroid of the assigned cluster is then updated. Template matching algorithm are often suitable for online execution, though the initialization of the template might require an initial short offline preprocessing step. This is the case of the method presented in [START_REF] Franke | An online spike detection and spike classification algorithm capable of instantaneous resolution of overlapping spikes[END_REF] which can be applied online once the templates have been defined.

Another factor that can jeopardize real-time spike sorting is the computation time. In addition to intrinsically allow an online processing, a real-time spike-sorting method should also be able to detect and classify an action potential within a few milliseconds, which is the duration of an action potential.

In particular, the clustering part can be very time-consuming if too many dimensions are used. This time constraint becomes particularly critical when the number of recording electrodes is important as in currently-developed cortical implants, generating large amounts of data to process.

Using multiple electrodes

Using large-scale multi-electrode implants allows to record more neural cells, and brings more information for spike-sorting but requires a suitable spike-sorting method. In the case of an electrode array where electrodes are far enough from each other to record completely different neurons, such as Utah arrays, each electrode can be processed independently as a single electrode. In contrast, a few electrodes very close to one another, such as tetrodes, may record the same neural cells, and the set of electrodes can thus be processed together as a single electrode by simply concatenating the features. In between, using dense microelectrode arrays brings an interesting spatial dimension to the spike-sorting problem. Indeed, in that case, each electrode detects the activity of a few neural cells, as for a single electrode, but each neural cell is also recorded by several electrodes but not all the electrodes of the array (Figure II-5). Hence action potentials are spatiotemporal events. They are limited both in time, as they last about a millisecond, and in space, as there are visible on a few neighboring electrodes. This spatial dimension is all the more important as, because of the high number of recorded neural cells, many action potentials will overlap in time. However they can be easily discriminated if they do not overlap in space. This spatial aspect thus affects all the steps of the spike-sorting algorithm. [START_REF] Lefebvre | Recent progress in multi-electrode spike sorting methods[END_REF] gives good review about existing multi-electrode spikesorting algorithms. Detection should not only give the timestamp of the action potential event but also information about its location. A common way to perform a non-redundant detection is to search for local minima (or maxima) both in time and space, which is done for example in [START_REF] Yger | Fast and accurate spike sorting in vitro and in vivo for up to thousands of electrodes[END_REF]. Each event thus correspond to one timestamp and one electrode. [START_REF] Rossant | Spike sorting for large, dense electrode arrays[END_REF]) has a different approach, where events are described as spatially connected components over a neighborhood graph. The events are detected as crossing a high threshold and then extended to neighboring electrodes crossing a lower threshold. This spatial information should then be used for clustering. In [START_REF] Rossant | Spike sorting for large, dense electrode arrays[END_REF], the electrodes detected as belonging to the action potential event are represented in a mask vector, whose components are equal to one for these electrodes, and to zero for the others. This mask is then used in a masked expectation-maximization algorithm to indicate where the relevant values are, the irrelevant ones being replaced by a random distribution corresponding to noise. [START_REF] Yger | Fast and accurate spike sorting in vitro and in vivo for up to thousands of electrodes[END_REF]) used a very practical approach. As each event is associated to one electrode, the clustering is done independently on each electrode. Some clusters can be merged afterward if similar action potentials sometimes have their peaks on one electrode and some other times on another neighboring one. An interesting approach is used in [START_REF] Hilgen | Unsupervised Spike Sorting for Large-Scale, High-Density Multielectrode Arrays[END_REF], as the location of an action potential is processed like another feature. Indeed for each action potential event, a barycenter is computed and the coordinates are used as features in a mean-shift algorithm. Template matching methods are not much affected by the use of multiple electrodes. Indeed, once the templates have been established, they implicitly contain the information about the action potential location, as the template is null on electrodes where the action potential is not visible. The spatial aspect is thus mainly used in the initialization of the templates.

Another particularity of spike-sorting on multiple electrodes is that the noise can be correlated across different electrodes. Knowing the correlation matrix between the electrodes, it is possible to apply a linear transformation to the multiple-electrode signal to obtain a new signal with the same number of dimensions but where the dimensions are not correlated. This process, known as noise whitening, is used in many spike-sorting methods [START_REF] Delescluse | Efficient spike-sorting of multi-state neurons using inter-spike intervals information[END_REF][START_REF] Marre | Mapping a Complete Neural Population in the Retina[END_REF][START_REF] Ekanadham | A unified framework and method for automatic neural spike identification[END_REF][START_REF] Yger | Fast and accurate spike sorting in vitro and in vivo for up to thousands of electrodes[END_REF]) and can also be used to remove temporal correlations.

Common difficulties

In the previous section we presented the general principles of spike-sorting and some classical algorithms to solve it. However, while designing a spike-sorting method, one can be confronted to several difficulties specific to neural recordings. In particular the fact that the action potentials emitted by one neural cell have each time the same shape is not always true. Here we present some classical known difficulties encountered in spike-sorting and methods to overcome them.

Bursts of action potentials

Neural cells sometimes fire in burst, which means they emit several spikes within a short period of time. The problem is that the amplitude of the action potential tends to decrease at each occurrence within a burst . The assumption that the shape of the action potential does not vary is thus not true in this case. To tackle this problem, some spike-sorting algorithms use the weakest assumption that the action potential from a given neural cell always has the same base waveform, but modulated in amplitude.

The method presented in [START_REF] Yger | Fast and accurate spike sorting in vitro and in vivo for up to thousands of electrodes[END_REF]) is a template matching algorithm that uses this assumption.

An action potential with a different amplitude can still be detected with the corresponding template, and its amplitude is computed after detection and classification. [START_REF] Franke | An online spike detection and spike classification algorithm capable of instantaneous resolution of overlapping spikes[END_REF]) and [START_REF] Ekanadham | A unified framework and method for automatic neural spike identification[END_REF]) also use templates modulated by an amplitude in their algorithm. In both case the amplitude estimation is also used to estimate correctly the templates. [START_REF] Delescluse | Efficient spike-sorting of multi-state neurons using inter-spike intervals information[END_REF]) used a different strategy, as the only features used in their algorithm are the action potentials' amplitudes and not their precise shapes. Their algorithm relies on an explicit model of the action potential amplitude depending on the inter-spike interval, an information that is not used in the previously presented algorithms.

Non stationary data

Another situation where the assumption that waveforms do not change is violated is the case of long recordings, during which electrodes can slowly drift, causing slow changes in the action potential waveforms (Figure II-7). A common way to solve the problem is to slice the recording into chunks that are short enough to assume that there are no significant waveform changes within one chunk. This is done for example in [START_REF] Franke | An online spike detection and spike classification algorithm capable of instantaneous resolution of overlapping spikes[END_REF]. Their method is an online template matching method, for which the templates are updated at the end of every chunks. [START_REF] Bar-Hillel | Spike sorting : Bayesian clustering of non-stationary data[END_REF]) also divides the recording into chunks. The action potentials are modeled as a chain of Gaussian mixtures, and the method combines an expectation-maximization algorithm with a Bayesian method to link the mixtures of each chunk. In [START_REF] Calabrese | Kalman filter mixture model for spike sorting of non-stationary data[END_REF], another variation of the expectation maximization is used, by combining it with a Kalman filter applied on the Gaussian mixture parameters.

Temporal waveform overlap

When action potentials from different neural cells occur in a short time interval, their waveforms may overlap in the signal. The resulting waveform is the sum of the different waveforms, which often makes it impossible to recognize for traditional algorithms . A first type of approach to solve this problem is to proceed by iterations, as this is done in [START_REF] Yger | Fast and accurate spike sorting in vitro and in vivo for up to thousands of electrodes[END_REF]. Their method is a greedy template matching algorithm. At each iteration the template and time position obtaining the highest template matching score is selected and the corresponding waveform is then removed from the signal. Thus if at least one of two overlapping action potentials is detected, the second one can also be detected once the first one has been removed from the signal.

The second type of approach is to use an algorithm based on a model that directly takes into account that action potential waveforms are summed. This is the case of [START_REF] Franke | An online spike detection and spike classification algorithm capable of instantaneous resolution of overlapping spikes[END_REF]. The first step of the algorithm is to filter the signal with filters based on templates, which is a linear operation. The second step is an independent component analysis (ICA), which by nature take into account the fact that signals are summed. Their study shows that the method is effectively robust to overlapping action potentials. In [START_REF] Ekanadham | A unified framework and method for automatic neural spike identification[END_REF]), the signal is modeled as the sum of templates multiplied by an amplitude coefficient, the amplitude being null when there is no action potential. The goal is thus to find the templates and their amplitudes for each sampling time. The amplitudes and the templates are alternatively optimized, using respectively a gradient descent and a least-squares method. The common point between these two methods, that make them robust to overlapping action potentials, is that they do not consider action potentials independently but all together. Noticeably, methods able to disentangle overlapping waveforms are methods that have a global approach in contrast to methods decomposed in the three classical steps. Indeed once a detection step has been applied, the detected waveform is considered to correspond to one action potential. Therefore an action potential corrupted by another action potential cannot be classified correctly.

Spike-sorting software implementations

Spike-sorting algorithms are numerous and diverse, and software implementations of spike-sorting methods are even more diverse. A list of existing spike-sorting software can be found on the following web page, maintained by Simon Kornblith and reproduced in Table II-1: http://simonster.github.io/SpikeSortingSoftware/. Some software applications implement a specific spike-sorting algorithm, some others offer several combinations of the different processing steps. For similar algorithms, the choice of the language and implementation optimizations can have a strong impact on the execution time. The user interface design also plays an important role in the software usage. Despite the diversity of existing software applications, only a few allow a fully unsupervised online spike-sorting, and the execution time often becomes important for large MEA.

III. ARTIFICIAL STDP SPIKING NEURAL NETWORKS STATE OF THE ART

The most known type of artificial neural networks are formal neural networks, which are very popular in artificial intelligence. STDP networks differ from this type of neural networks on two main points. First, STDP networks are spiking neural networks (SNN). They are constituted of neurons emitting spikes, which are discrete events characterized by their time of emission, whereas formal neurons output a real value with no temporal information. In SNNs, information is thus conveyed through sequences of spikes, also called spike trains. Second, formal neural networks are usually trained to perform a task through a global supervised or reinforcement algorithm, optimizing the weights of the synapses connecting the neurons. STDP networks implement a spike-timing-dependent plasticity (STDP), which is a local learning rule, inspired from biological synapses (Bi & Poo 1998), which modifies a synapse weight depending on the time difference between spikes emitted by the presynaptic and the postsynaptic neurons. From these two points of view, STDP networks are thus closer to biological reality than formal neural networks. For this reason, they are often used to investigate how information is processed in the brain either in an effort to reproduce a specific neural function, or to study their computational properties. They also have been used to solve concrete pattern recognition problems in different application fields. In this section we will review different ways of encoding information in spiking neural network, neuron models and synaptic plasticity models and their properties, some important network properties and some existing applications of STDP networks to pattern recognition tasks.

Neural code: how to encode information in a neural network

Understanding how the information in encoded in the brain is a fundamental question in neuroscience, for which there is, at the moment, no clear answer. Through time, many possible coding schemes have been proposed. When designing an artificial neural network, whether to test a model or for a computational application, the choice of how the information is encoded is crucial as it deeply impacts the network mechanics.

Rate coding and its limitations

Historically, the most widely used neural code is rate coding. The idea of rate coding is that the information is contained in the firing rate of each neuron and not in the precise spike times. When dealing with spiking neurons, their firing rates can be computed by averaging the number of spikes emitted within a short time period. Other possibilities are to average the firing rate over a population of neurons, which requires an entire population to code for the same variable, or to average over several runs of an experiment. The advantage of rate coding is that the firing rate takes continuous values, varying continuously in time and therefore is easy to manipulate. An illustrative example is the population vector, which can be used to represent the direction of movement encoded by an ensemble of cortical motor neurons (Georgopoulos et al. 1986;Schwartz 1994). This gave birth to the second generation of artificial neural networks. In contrast with the first generation where the neurons' outputs are binary, here the neurons' outputs are real values, obtained through an activation function of the weighted sum of their inputs. This kind of neural network has the strong advantage that they can be trained for a specific task using a backpropagation algorithm (Rumelhart et al. 1986).

The efficiency of the backpropagation algorithm explains its popularity in artificial intelligence applications, however it is not biologically realistic and cannot explain learning in the brain. A strong argument supporting the fact that the firing rate cannot be the only way the information is encoded in the nervous system is that the speed at which some tasks are executed is incompatible with the time necessary to estimate a firing rate (Gerstner et al. 1993;[START_REF] Thorpe | Rapid Visual Processing using Spike Asynchrony[END_REF][START_REF] Johansson | First spikes in ensembles of human tactile afferents code complex spatial fingertip events[END_REF][START_REF] Vanrullen | Spike times make sense[END_REF]. For example a visual task takes about 150ms to complete whereas, it would take about 100ms at each layer to integrate a reliable firing rate [START_REF] Thorpe | Rapid Visual Processing using Spike Asynchrony[END_REF].

Another argument is the fact that rate coding constitutes a loss of information relatively to pulse coding, where individual spike times are taken into account. As individual spike times carry more information than an average firing rate, spiking networks are thus theoretically more powerful computationally (Maass 1997a;Maass 1997b;Gerstner et al. 1993). Although the rate-coding paradigm has shown its efficiency with formal neural networks, a pulse-coding paradigm is naturally more relevant for spiking neural networks, and probably closer to biological reality.

Different forms of pulse coding

Pulse coding can be declined into different forms, depending on how the information contained in the spike times is interpreted. A first important aspect is how an analog stimulus can be encoded into a spike train. The second aspect of pulse coding is how information carried in the spike train is used by the network.

A first method to encode an analog stimulus into spikes is the latency code [START_REF] Vanrullen | Surfing a spike wave down the ventral stream[END_REF][START_REF] Masquelier | Unsupervised learning of visual features through spike timing dependent plasticity[END_REF][START_REF] Thorpe | Rapid Visual Processing using Spike Asynchrony[END_REF]. The principle of latency coding is an intensitylatency conversion, which means that the latency of the first spike emitted by a neuron relatively to the onset of the stimulus depends on the intensity of the stimulus (see Figure . This is quite natural when using integrate-and-fire neurons as neurons integrating stronger stimulus fire first. [START_REF] Johansson | First spikes in ensembles of human tactile afferents code complex spatial fingertip events[END_REF] show experimental evidence that this kind of code is actually found in sensory neurons. Latency coding requires a reference time, from which the latency is computed. It has been proposed that one possible reference signal could be the oscillatory brain activity, thus constituting a variation of latency coding called phase coding (Hopfield 1995;[START_REF] Mclelland | Neuronal oscillations and the rate-to-phase transform: mechanism, model and mutual information[END_REF][START_REF] Masquelier | Oscillations can reconcile slowly changing stimuli with short neuronal integration and STDP timescales[END_REF]. It is indeed possible, by using a LIF neuron and an oscillatory signal to convert an analog stimulus or a rate code into a phase code. Again, phase coding has also been observed experimentally [START_REF] Mclelland | Neuronal oscillations and the rate-to-phase transform: mechanism, model and mutual information[END_REF]. The second important point in neural code is how spike time information is used in the network. In some case, the relevant information is carried by the strongest stimuli and thus by the early-arriving spikes. This paradigm is used in [START_REF] Masquelier | Unsupervised learning of visual features through spike timing dependent plasticity[END_REF], where the information of the first spike time among sets of neurons is used for image processing. More generally, network where the relevant information is carried by the spikes' firing order are said to use a rank-order code [START_REF] Vanrullen | Surfing a spike wave down the ventral stream[END_REF][START_REF] Thorpe | Rapid Visual Processing using Spike Asynchrony[END_REF] and are used when the earliest spikes are the most important ones. This is emphasized in [START_REF] Vanrullen | Surfing a spike wave down the ventral stream[END_REF] where an image processing algorithm is modeled using rank-order coding and where most of the original image can be reconstructed using only a small fraction of the spikes, i.e. those emitted early after the stimulus onset. In some other cases, the exact spike times are crucial for the network mechanics. For example, a network can use spike synchrony as a relevant pattern, which means that several neurons coding for different variables need to fire at the same time to trigger a spike in another neuron [START_REF] Singer | Synchronization of cortical activity and its putative role in information processing and learning[END_REF]. This is natural for Leaky-Integrate-and-Fire (LIF) neurons that need to receive enough spikes simultaneously to fire. More generally, the relevant information can be contained in the spike time differences between several neurons, meaning that input neurons need to fire with precise spike time differences for the spike sequence to be recognized. This property was termed as "polychrony" by [START_REF] Izhikevich | Polychronization: computation with spikes[END_REF]). To achieve this polychrony, a crucial element is the transmission delay between neurons, which makes it possible for spikes emitted by different neurons at different times to arrive simultaneously on the same readout neuron, as illustrated in Figure III-2 (Gerstner et al. 1993;Hopfield 1995;[START_REF] Izhikevich | Polychronization: computation with spikes[END_REF]. Overall, whereas the rate code is simple to use due to its analog nature, pulse codes are more difficult to manipulate and require to make choices on how the time information is used but can lead to possibly more powerful computational properties.

Neuron models and their properties

Neurons are one of the two key elements, with synapses, of an artificial neural network. In spiking neural network, they obey to a temporal dynamic which can be inspired from what happens with real neural cells in the brain.

Several types of neurons exist in the brain and they exhibit different behaviors. [START_REF] Hodgkin | The local electric changes associated with repetitive action in a non-medullated axon[END_REF] studied the response of axons to stimuli and established a behavioral classification: class 1 axons, which have a wide range of spiking frequency response depending on the stimulus strength, and class 2 axons, whose frequency response is relatively insensitive to the stimulus strength. Since then, the behavioral classification of neurons hasbeen refined, the most known classes of neurons being the regular spiking neurons (RS), the intrinsically bursting neurons (IB), the fast spiking neurons (FS) and the low-threshold spiking neurons (LTS) [START_REF] Connors | Intrinsic firing patterns of diverse neocortical neurons[END_REF][START_REF] Gibson | Two networks of electrically coupled inhibitory neurons in neocortex[END_REF][START_REF] Izhikevich | Simple model of spiking neurons[END_REF][START_REF] Pospischil | Minimal Hodgkin-Huxley type models for different classes of cortical and thalamic neurons[END_REF]. The behavior of a neuron is determined by its response to different kinds of stimuli. [START_REF] Izhikevich | Which Model to Use for Cortical Spiking Neurons[END_REF] gives a good review of the different possible responses to different stimuli. In particular, the response can be:

 tonic spiking or bursting when the neuron fires or bursts continuously when stimulated ( When designing an artificial spiking neural network, one needs to find a neuron model able to reproduce the desired behavior. A very biologically realistic model was established by [START_REF] Hodgkin | A Quantitative Description of Membrane Current and its Application to Conduction and Excitation inNerve[END_REF], which modeled precisely all the ionic currents within an axon during the generation of an action potential. This model is highly configurable and can reproduce any kind of observed behavior. However its complexity makes it difficult to use and computationally demanding.

Depending on the goal to achieve, a neuron model does not necessarily need to be highly biologically meaningful, but to reproduce efficiently some neural behavior. Neuron models can be divided into several main types. First, models that, like the Hodgkin-Huxley model, reproduce precisely the neuron's membrane potential dynamics, including the action potential dynamic. Some of these models are biologically meaningful, such as [START_REF] Morris | Voltage Oscillations in the Barnacle Giant Muscle Fiber[END_REF], while others focus on reproducing efficiently the neuron dynamic [START_REF] Fitzhugh | Impulses and Physiological States in Theoretical Models of Nerve Membrane[END_REF][START_REF] Rose | The assembly of ionic currents in a thalamic neurons: I. The threedimensional model[END_REF]Nanami & Kohno 2016).

Second, models that describe the subthreshold potential dynamics but handle spikes as discrete events occurring when the neuron potential reaches its threshold. These models, sometimes called integrateand-fire models, have the advantage to greatly reduce the computational cost, as the rapid dynamic of an action potential does not need to be computed [START_REF] Izhikevich | Which Model to Use for Cortical Spiking Neurons[END_REF]). The most widely used neuron model of this type is the Leaky-Integrate-and-Fire (LIF) model, whose potential evolution is simply described by the following equation:𝜏 𝑑𝑢 𝑑𝑡 ⁄ = -𝑢(𝑡) + 𝑅𝐼(𝑡). The evolution of the potential u combines a leak with a time constant τ and an integration of the input current I. When the potential u reaches a fixed threshold level, a spike is emitted and the potential is reset to a predefined value. Many variations of this model have been developed, with more variables to induce interesting firing properties [START_REF] Smith | Fourier analysis of sinusoidally driven thalamocortical relay neurons and a minimal integrate-and-fire-or-burst model[END_REF][START_REF] Izhikevich | Resonate-and-fire neurons[END_REF][START_REF] Brette | Adaptive Exponential Integrate-and-Fire Model as an Effective Description of Neuronal Activity[END_REF]). An noticeable model of this type was developed by Izhikevich [START_REF] Izhikevich | Simple model of spiking neurons[END_REF], which, in spite of its simplicity, is able to mimic many different neuronal behaviors. Another type of model that process spikes as discrete events is the Spike Response Model (SRM). The SRM expresses the potential of the neuron as a function of time that depends on the previously received and emitted spikes. The spike times are still defined as the times when the potential reaches a threshold.

Many different neural behaviors and many different neuron models exist. Choosing a model depends on which behavior needs to be reproduced with which degree of accuracy and with which efficiency.

Synaptic plasticity

The ability of neural networks to learn is due to their synaptic plasticity, in other word, the ability of synapses to change their weight. Indeed, in a spiking neural network, synapses transmit spikes from a presynaptic neuron to a postsynaptic neuron, and the weight of a synapse affects its ability to influence its postsynaptic neuron. At the network's level, the synaptic plasticity can thus modify the network's response to different stimuli. According to Hebb's postulate, the weight of a synapse evolves following local plasticity rules depending on the activities of the post and presynaptic neurons. In most models, these weight changes are induced by presynaptic and postsynaptic spikes, and depend on the time difference between them. This kind of plasticity is thus called spike-timing-dependent plasticity (STDP). The most common STDP rules induce changes triggered by pairs of pre-and postsynaptic spikes, but changes can also happen on single spike or depending on more than two spikes. The changes can be persistent, for long-term plasticity, or non-persistent for short term plasticity. The following sections summarize the most common models of plasticity.

1.

Long-term STDP

The most known spike-timing-dependent plasticity rule is the long-term plasticity induced by pairs of presynaptic and postsynaptic spikes. This form of plasticity was first observed experimentally by (Bi & Poo 1998). When presynaptic and postsynaptic spikes separated by a given time interval are repeatedly induced, a persistent change in the strength of the synapse is observed. The amount of change depends on the time interval separating the presynaptic spike and the postsynaptic spike, which constitutes the core principle of pair-based STDP rules.

In the experiment from (Bi & Poo 1998), the weight change is positive when the presynaptic spike precedes the postsynaptic spike, corresponding to a long term potentiation (LTP), and negative in the other case, corresponding to a long term depression (LTD) (see Figure . The weight change is large when this time interval is short and tends to zero when the time interval is long. This is coherent with Hebb's postulate [START_REF] Hebb | The Organization of Behavior[END_REF]) that the connection between a cell repeatedly participating is the activation of another cell is reinforced. In STDP cases, presynaptic spikes shortly preceding postsynaptic spikes are likely participating in triggering the postsynaptic spike, and thus reinforce the synapse's weight. The dependence between the spike time interval and the weight's change can be, in this case, modeled as a double exponential function. Since then, other shapes for this time dependence have been observed and used in models such as symmetric dependences or anti-hebbian plasticity where the time dependence is reversed compared to the classical time dependence [START_REF] Woodin | Coincident pre-and postsynaptic activity modifies GABAergic synapses by postsynaptic changes in Cl-transporter activity[END_REF][START_REF] Luz | Balancing feed-forward excitation and inhibition via hebbian inhibitory synaptic plasticity[END_REF][START_REF] Vogels | Inhibitory synaptic plasticity: spike timing-dependence and putative network function[END_REF] The weight's change applied on a synapse can either depend or not, on the current weight of the synapse. A common way to model a weight dependence is to make the weight depression proportional to the current weight w, and the weight potentiation proportional to (1-w). This kind of weight dependence is often called multiplicative STDP, in contrast to additive STDP for a synaptic plasticity rule that does not depend on the current weight. When used in a spiking neural network, multiplicative and additive STDP do not have the same properties [START_REF] Rubin | Equilibrium properties of temporally asymmetric Hebbian plasticity[END_REF][START_REF] Gütig | Learning input correlations through nonlinear temporally asymmetric Hebbian plasticity[END_REF]. Indeed, when using additive STDP, synapses converge towards either their maximum or their minimum weights, whereas multiplicative STDP leads to continuously distributed equilibrium weights.

When dealing with realistic spike trains, more than one postsynaptic and presynaptic spikes contribute to the synapse plasticity. A pair-based STDP model thus has to specify which spikes' pairs have to be taken into account. There are many possibilities such as the all-to-all pairing scheme, where all possible combinations are taken into account, or a nearest-neighbor scheme where only neighboring spikes are taken into account. These different pairing schemes can be implemented using local variables [START_REF] Morrison | Phenomenological models of synaptic plasticity based on spike timing[END_REF], which avoids having to store past spikes.

Experiments such as triplet and quadruplet experiments [START_REF] Froemke | Spike-timing-dependent synaptic modification induced by natural spike trains[END_REF][START_REF] Wang | Coactivation and timing-dependent integration of synaptic potentiation and depression[END_REF] or frequency dependence experiments [START_REF] Sjöström | Rate, timing, and cooperativity jointly determine cortical synaptic plasticity[END_REF] show the limitation of the pair-based model to explain synaptic plasticity when multiple spikes are involved. For example, in a triplet protocol, three spikes are presented to the synapse, either in a pre-post-pre scheme or a post-pre-post scheme.

According to a pair-based model, the resulting weight change should be the same with both schemes, which is not what is observed. New models have been developed to explain these experiments, such as the suppression model [START_REF] Froemke | Spike-timing-dependent synaptic modification induced by natural spike trains[END_REF], or the triplet model [START_REF] Pfister | Triplets of Spikes in a Model of Spike Timing-Dependent Plasticity[END_REF]. Both take into account previous postsynaptic and presynaptic spikes to modulate the impact of a pre-post or post-pre pairs on the synapse weight.

Short-term plasticity

Long-term STDP induces persistent weight changes depending on the correlation between postsynaptic and presynaptic activities. These change are persistent in the sense that in the absence of activity, the synapses' weights remain unchanged. On the contrary, short-term plasticity (STP) is a plasticity triggered by transmission of presynaptic spikes that induces non-persistent changes, as the synapse's weight return to a baseline value in the absence of activity. As for long-term plasticity, shortterm plasticity has been observed on real synapses through numerous experiments [START_REF] Markram | Differential signaling via the same axon of neocortical pyramidal neurons[END_REF][START_REF] Zucker | Short-term synaptic plasticity[END_REF]. Short-term plasticity may result in either a depression, when each presynaptic spike decreases the synapse weight ( ). These phenomena can be explained by the fact that the transmission of spikes through a synapse depends on some resources, whose quantity is modified at each spike transmission. This idea of resource dependence has been used to developed models of short-term plasticity, which are able to reproduce both facilitation and depression (Abbott et al. 1997;[START_REF] Tsodyks | Synchrony generation in recurrent networks with frequency-dependent synapses[END_REF]. As changes induced by short-term plasticity only take effect during a short period and rapidly fade with time, it cannot be considered as directly participating in learning a task. However it has interesting properties that contribute to the network's efficiency. Indeed, short-term plasticity is able to regulate the postsynaptic potential induced by input spikes, which leads to a better discrimination of changes in the input spike train (Abbott et al. 1997).

Homeostasis and metaplasticity

To be computationally relevant, the learning process in a neural network should both be stable, which means synapses' weights and neurons' activity should not diverge nor become totally null, and induce some selectivity, which means that depending on the input activity and the network structure, some synapses are potentiated whereas others are depreciated in order to have specific responses to different stimuli. Most STDP rules constitute a positive reinforcement, as synapses that contribute to make a postsynaptic neuron fire are potentiated. This process can thus be unstable if not correctly tuned. A standard way to obtain a stable network is to choose an STDP rule that induce more LTD than LTP in the absence of correlation (Masquelier et al. 2008;[START_REF] Humble | Spatio-temporal pattern recognizers using spiking neurons and spike-timing-dependent plasticity[END_REF][START_REF] Srinivasa | Unsupervised discrimination of patterns in spiking neural networks with excitatory and inhibitory synaptic plasticity[END_REF]. Some other STDP rules can inherently be stable yet lack the competitive property. This is the case for example of multiplicative STDP for which the synapses' weights naturally converge but in a unimodal distribution [START_REF] Rubin | Equilibrium properties of temporally asymmetric Hebbian plasticity[END_REF][START_REF] Gütig | Learning input correlations through nonlinear temporally asymmetric Hebbian plasticity[END_REF]. More generally, obtaining a compromise between the stability and the competition requires a fine tuning of the learning rules [START_REF] Yger | Models of Metaplasticity: A Review of Concepts[END_REF][START_REF] Babadi | Stability and Competition in Multi-spike Models of Spike-Timing Dependent Plasticity[END_REF][START_REF] Gütig | Learning input correlations through nonlinear temporally asymmetric Hebbian plasticity[END_REF].

Therefore, it can be interesting, both for obtaining realistic models or for computational properties, to introduce complementary plasticity rules regulating the STDP rule. These kinds of rules are known as homeostatic mechanisms, as they allow to obtain homeostasis, i.e. stability of the network firing rates through learning. Whereas STDP rules are homosynaptic, depending on the pre-and postsynaptic activities on a given synapse, homeostatic mechanisms can be heterosynaptic, as changes on one synapse can influence neighboring synapses. They also act at a longer time scale than the STDP rule they regulate [START_REF] Zenke | Synaptic Plasticity in Neural Networks Needs Homeostasis with a Fast Rate Detector[END_REF][START_REF] Yger | Models of Metaplasticity: A Review of Concepts[END_REF]. There is some evidence that plasticity acting at different timescales and on more than single synapses actually takes place in the brain, though such plasticity can take many different forms, whether modifying directly the synapse weight, or modifying the neuron excitability or modulating STDP rules [START_REF] Abbott | Synaptic plasticity: taming the beast[END_REF]. They can also have different purposes such as stabilizing the neuronal activity, but also improving the selectivity or allowing reinforcement learning.

Homeostatic mechanisms acting directly on the synapse weight are also known as synaptic scaling.

The principle of synaptic scaling is to limit or to impose the total weight of synapses arriving on the same postsynaptic neuron, thus stabilizing learning by preventing the total weight to grow up indefinitely and encouraging selectivity by preventing all synapses from fading. Models implementing a synaptic scaling often simply renormalize the synaptic weight at each weight change [START_REF] Malsburg | Self-organization of orientation sensitive cells in the striate cortex[END_REF]Lazar 2009;Aswolinskiy & Pipa 2015;[START_REF] Humble | Spatio-temporal pattern recognizers using spiking neurons and spike-timing-dependent plasticity[END_REF]. Homeostatic mechanisms can also modify the neuron's excitability, a process sometimes called intrinsic plasticity. (Zhang & Linden 2003) suggest that intrinsic plasticity plays an important role in the brain. Its main function is to regulate the activity of individual neurons. In models such as (Lazar 2009;Aswolinskiy & Pipa 2015), the intrinsic plasticity adjusts the neuron's threshold depending on its activity so it reaches a mean firing rate target. Finally, some regulation mechanism modulates the STPD rule itself. This kind of plasticity is called metaplasticity. One of the first model of metaplasticity was developed in [START_REF] Bienenstock | Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex[END_REF] to explain the development of neuronal selectivity. It involves in this case a Hebbian plasticity that depends on the instantaneous pre-and postsynaptic activities. This plasticity is further modulated by a moving average of the postsynaptic activity, which thus constitutes a form of metaplasticity. [START_REF] Zenke | Synaptic Plasticity in Neural Networks Needs Homeostasis with a Fast Rate Detector[END_REF]) used a similar mechanism on an STDP rule: the amount of depression induced by the STDP rule depends on the average output activity. Another example of metaplasticity is presented in (Hunzinger et al. 2012). In this study the amount of synaptic change depends on a resource variable that is decreased at each synaptic change and then slowly recovers. The resource variable being shared across synapses, this typically constitutes a heterosynaptic metaplasticity. An interesting application of metaplasticity is to implement a reinforcement learning in an STDP network. [START_REF] Izhikevich | Solving the distal reward problem through linkage of STDP and dopamine signaling[END_REF]) and [START_REF] Legenstein | A learning theory for reward-modulated spike-timingdependent plasticity with application to biofeedback[END_REF]) both presented a model where the STDP weight change is not immediately applied but converted into a fading eligibility trace. The model also includes a reward signal that is increased each time a reward is received and also fades with time. The weight change is then computed as the product of this eligibility trace with the reward signal. Following the same idea, (Aswolinskiy & Pipa 2015) also modulate the STDP rule with a reward variable.

Network properties

Neurons and synapses are the two elementary components of an artificial neural networks. Each of these elements have their properties and their temporal dynamics. When combined together to form a neural network, a new dynamic emerges, which can be complex to study. When studying networks, two complementary approaches exist. On one hand, some studies focus on studying the network's neural activity with fixed synaptic connections, often chosen following a random distribution. On the other hand, other studies focus on the evolution of synaptic weights given assumptions on the neurons' activity.

One type of neural networks that has been largely studied are sparsely connected recurrent networks, possibly taking an external stochastic input, as a model of what happens in the cortex (van Vreeswijk & Sompolinsky 1996; [START_REF] Amit | Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex[END_REF][START_REF] Brunel | Dynamics of Sparsely Connected Networks of Excitatory and Inhibitory Spiking Neurons[END_REF][START_REF] Ostojic | Two types of asynchronous activity in networks of excitatory and inhibitory spiking neurons[END_REF]). It has been shown that these networks can reach, under certain conditions, a stable state called the asynchronous state, where neurons can be described as independent Poisson neurons, firing at a constant and homogeneous firing rate. The sparsity of the network allows to assume that the neurons activities are independent, as their inputs are independent from one another. Assuming that the inputs of each neuron come from independent Poisson neurons, each emitting spikes according to a Poisson process, the total input current follow a white Gaussian distribution. As a consequence, the neuron behaves as a Poisson neuron whose firing rate depends on the mean and the variance of its input [START_REF] Amit | Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex[END_REF]. This mean-field approach allows to deduce the firing rate of the asynchronous state. An important condition from the asynchronous state to be stable is that the excitation and the inhibition should be correctly balanced in the network (van Vreeswijk & Sompolinsky 1996; [START_REF] Amit | Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex[END_REF][START_REF] Brunel | Dynamics of Sparsely Connected Networks of Excitatory and Inhibitory Spiking Neurons[END_REF][START_REF] Ostojic | Two types of asynchronous activity in networks of excitatory and inhibitory spiking neurons[END_REF]. Indeed, at this condition, the average input of a neuron is subthreshold and its action potentials are mainly due to fluctuations, ensuring an irregular firing. In other conditions, states where with oscillations and synchronization between neurons can emerge [START_REF] Brunel | Dynamics of Sparsely Connected Networks of Excitatory and Inhibitory Spiking Neurons[END_REF]. Other studies also showed that with a strong synaptic coupling the neural activity is chaotic, with high fluctuations of the firing rates both in time and across neurons, though the population firing rate is still constant on average [START_REF] Ostojic | Two types of asynchronous activity in networks of excitatory and inhibitory spiking neurons[END_REF]. Overall, random networks exhibit various possible behaviors and constitute a good basis to understand the dynamic of neural networks.

The second important aspect in neural network is synaptic learning. Learning in neural networks modifies the structure of the network depending on sensory inputs. The most widely used method to study synaptic weights evolution is to make assumptions on the firing rates and the time correlations between and within groups of neurons. This then allows to compute the average weight change of a particular synapse, or of groups of synapses, depending on the STDP rule used. This method can be used to study various STDP rules' properties, such as the weight distribution depending on the weight dependence of the STPD rule [START_REF] Rubin | Equilibrium properties of temporally asymmetric Hebbian plasticity[END_REF][START_REF] Gütig | Learning input correlations through nonlinear temporally asymmetric Hebbian plasticity[END_REF], the stability and the selectivity of different triplet rules [START_REF] Babadi | Stability and Competition in Multi-spike Models of Spike-Timing Dependent Plasticity[END_REF] or the weight evolution in a recurrent neural network [START_REF] Burkitt | Spike-timing-dependent plasticity for neurons with recurrent connections[END_REF]). As stated previously, balance between inhibition and excitation is important for the stability of a network, which seems to be confirmed by experiments [START_REF] Kirkwood | Balancing Excitation and Inhibition[END_REF]. Though this balance can be self-maintained in a random recurrent network (van Vreeswijk & Sompolinsky 1996), networks with a feedforward structure and a varying sensory input require a precise balance. Studies on inhibitory STDP rules show that it is possible to obtain a precise excitatory-inhibitory balance through learning on inhibitory synapses [START_REF] Vogels | Inhibitory Plasticity Balances Excitation and Inhibition in Sensory Pathways and Memory Networks[END_REF][START_REF] Luz | Balancing feed-forward excitation and inhibition via hebbian inhibitory synaptic plasticity[END_REF]. Beyond theoretical results, [START_REF] Srinivasa | Unsupervised discrimination of patterns in spiking neural networks with excitatory and inhibitory synaptic plasticity[END_REF] showed that the use of inhibitory STDP to balance inhibition and excitation leads to better results in a pattern recognition task.

Applications to pattern recognition tasks

One difficulty with STDP networks is that there is no standard method to train a network to achieve a specific task through STDP. Some studies have thus focused on learning random spike train patterns to demonstrate STDP network learning capabilities. (Masquelier et al. 2008) demonstrated that a very simple monolayer STDP network is able to learn, in an unsupervised manner, to recognize the beginning of a pattern (Figure III-7). This work was extended to demonstrate its ability to discriminate different patterns or different parts of a pattern [START_REF] Masquelier | Competitive STDP-based spike pattern learning[END_REF]. Following the same idea, (Hunzinger et al. 2012) showed that a resource-dependent plasticity improves the performance of STDP learning on a pattern recognition task. Application to more concrete pattern recognition tasks, such as visual recognition, are also emerging. For example, [START_REF] Masquelier | Unsupervised learning of visual features through spike timing dependent plasticity[END_REF]) used an STDP network to recognize faces and vehicles in pictures. Their network is well structured, alternating learning layers with shared weights and pooling layers, and used a classical classifier as a readout function. [START_REF] Brader | Learning real-world stimuli in a neural network with spikedriven synaptic dynamics[END_REF]) used a one-layer STDP network with a teaching signal for a digit recognition task. [START_REF] Bichler | Extraction of temporally correlated features from dynamic vision sensors with spike-timing-dependent plasticity[END_REF]) also trained a network to recognize patterns in a video stream, though the recognized patterns do not include time aspects such as movements. In this case the network was designed to be implemented in neuromorphic devices mimicking STDP at a miniaturized scale. Most of these applications use a feedforward structure, sometimes with only one layer. [START_REF] Srinivasa | Unsupervised discrimination of patterns in spiking neural networks with excitatory and inhibitory synaptic plasticity[END_REF]) used an interesting network structure, based on the idea of reservoir computing (Figure III-8). This network is constituted of pools of recurrently connected excitatory and inhibitory neurons, the different pools being themselves connected in a feedforward manner. This STDP network was applied to a visual pattern recognition task, and shows that the introduction of plasticity on inhibitory synapse improves the discrimination performances. Beyond visual recognition, (Suri et al. 2013) also show an example of application to auditory pattern recognition. At the moment, STDP applications to real-word pattern recognition tasks are still limited, but the existing applications show interesting learning properties such as unsupervised learning. More work needs to be done to understand how to efficiently design an STDP network for a specific task.

IV. NETWORK MODEL

We built an STDP network for spike sorting which takes a single electrode signal as input, and outputs a spike-train that corresponds to the sorted action potential recorded in the signal. It is organized into layers connected in a feed-forward manner, with all-to-all connections ( Here we focus on the different possible solutions to implement each different functional elements of the network for a single electrode case, and how the network structure can be adapted to process multi-electrodes' signals.

Encoding the input signal

The STDP network takes an analog signal representing a potential varying in time as an input. It is thus necessary to encode this input signal into a spike train for further processing by the network. This can be interpreted as a sensory functionality, by analogy with sensory neural cells that convert light intensity (in the retina) or pressure (in the skin) into spike trains for further processing by the central nervous system. Following this analogy, a first approach to encode the signal would be to use a neuron whose firing rate would increase with the value of the input signal, thus implementing a firing-rate code. This would give information whether the amplitude of the input signal is high or low, however, it would be difficult to discriminate precisely different amplitudes, as two different amplitudes would be encoded by the same neuron, with only a quantitative difference in its firing rate. Instead, our input layer is composed of a set of sensory neurons, each neuron being sensitive to a different range of signal amplitudes, defined by an individually selected central value c, , and a margin DV m , common to all input neurons. When the input signal value is within a neuron's sensitivity range, which is to say between c-DV m and c+DV m , the neuron fires, otherwise it remains silent. Those sensitivity ranges are regularly spaced, partially overlapping, and their union covers the entire possible range of signal values (Figure IV-2.a). By analogy with the visual system, this works as if the signal amplitude was an object moving along one dimension and activating neurons depending on its position (Figure ). An important property of this way to encode the input signal is that the number of spikes emitted by the input layer does not depend on the exact signal value, thus all signal values generate the same excitation level on the next layer.

In our case, the input signal is a sampled signal. Thus, a natural way to implement the sensory neurons is to compute their activation at regular time step, each time a new sample arrives (Figure IV-2.c). At each sampling step, each sensory neuron either fires once if the signal sample is within its sensitivity range or remains silent (Figure left). Another possibility is that each neuron emits several spikes at each sampling step, depending on how close the signal value is from the sensitivity range center. For example, the number of spikes emitted can be maximum when the signal value equals the sensitivity range center, and then decreases linearly until the signal value reaches the limits of the sensitivity range, for which no spike is emitted (Figure right). These two ways of encoding the signal, which we call later binary encoding and triangular encoding, have different properties that are studied in Section IV.B. In our different network implementations, only the binary encoding method has been tested, for its simplicity.

The sampling frequency used for encoding is an important parameter for the spike-sorting performance of the network. Indeed, to ensure that the next layers will properly learn the spike pattern generated by the input layer, it is necessary that for two different occurrences of the same waveform, some common sensory neurons get activated, regardless of the noise and sampling time jitter. A good criterion is to choose the sampling step short enough so that, even for the fastest variations of the signal, two consecutive samples always activate some common neurons. In practice and on the different recordings we testes, we found that an 80-kHz sampling frequency was a good compromise, between sufficient information and too much data generated. This was thus the value used in all our implementations, expect for the first one (MiniNet) where it was only 20 kHz.

To discriminate between different action potentials in the signal, we need to take into account not only the amplitudes of an action potential but its entire waveform, in other words the signal amplitude variations in time. Thus the input layer should give information about the value of the signal at different time points. To do so, we introduce synapses with different transmission delays stemming from each sensory neuron, as previously done for example in (Gerstner et al. 1993), (Hopfield 1995) or (Ghosh-Dastidar & Adeli 2007). From the point of view of the layers that receive the input layer spike train through these synapses, this looks like the original set of sensory neurons is duplicated into several sets of neurons acting alike yet with different delays. Each of the delayed set of sensory neurons encodes the signal value with a given delay from current time (Figure IV-2.e). The idea is that the plasticity rules will then potentiate some of these delayed synapses and depreciate others, as this is believed to happen during the development of the barn owl auditory system [START_REF] Gerstner | A neuronal learning rule for sub-millisecond temporal coding[END_REF].

The difference between two delayed synapses and the number of delays determine the time resolution of the encoded signal and the size of the encoded signal time window at each sampling step. The number of delays is a compromise between the number of synapses and the discriminative power of the input spike patterns. Noticeably, if the encoded time window becomes too large (typically larger than an action potential), it loses its discriminative power as the signal value outside the action potential is within the noise range and thus close to zero and non-discriminative. In practice we found that a 0.05-ms time resolution was enough, and we chose a time window of 0.5 ms (thus 10 different delays), which is usually shorter than one action potential.

Attention mechanism

The input signal the network has to process fluctuates around zero most of the time, as the action potentials occur sparsely. At some point in the process it is thus necessary to discriminate the parts of the signal corresponding to an action potential from the rest of the signal. Indeed, by the way the signal is encoded, there is no intrinsic difference between parts of the signal containing an action potential and parts containing only noise, and both can be recognized as patterns. As this is well described in (Masquelier et al. 2008), when a neuron learns a pattern through an STDP rule, the neuron tends to fire earlier each time the pattern is presented. In this study, this process naturally stops when the neuron reaches the beginning of the pattern, as the spike train before the pattern is random and cannot be learnt. In our case, a neuron learning an action potential waveform would each time recognize earlier parts of the waveform and would end up recognizing the null pattern between action potentials. Even though once some neurons have learned the null part of the signal, they could prevent other neuron from firing when the signal is null, we found better for a more stable learning that an attention mechanism detecting the action potentials intervene at an early stage of the process, just after the input layer, preventing the null parts of the signal to be learnt. This attention mechanism was implemented using a short-term plasticity. In the first versions of the network this short-term plasticity was applied directly on the synapses between the input and the intermediate layer, but we later introduced an attention neuron whose role was to specifically detect where action potentials occurred in the signal.

Short-term plasticity

We implemented an attention mechanism thanks to a short-term plasticity (STP) rule. Indeed STP, and more precisely short-term depression, has the property to weaken the weights of the synapses for which a presynaptic spike occurs at high frequency (Abbott et al. 1997), as this is the case for sensory neuron coding for near-zero values. As a result, the postsynaptic neuron will be less excited when the input signal is within the noise range. In our implementation the short-term plasticity rule is governed by the following equation:

𝑑𝑤 𝑑𝑡 = 1 𝜏 𝑠𝑡𝑝 (1 -𝑤) -∑ 𝑤 * 𝑓 𝑑 * 𝛿(𝑡 -𝑡 𝑠 ) 𝑠 ,
where w is the synaptic weight, τ stp the STP recovery time constant, f d the STP depression factor and t s the presynaptic spike times. We chose τ stp one order higher than the typical duration of an action potential so that the synaptic weight did not change significantly during an action potential. Given this equation we can compute the equilibrium weight for a presynaptic neuron firing at regular pace, with a frequency f. When such a synapse, with a weight w eq , fires, its weight becomes (1-f d )w eq . After a time interval 1/f, its weight should return to w eq through the exponential recovery before firing again. This can be expressed as (1 -𝑤 𝑒𝑞 ) = 𝑒𝑥𝑝(-1 𝑓 * 𝜏 𝑠𝑡𝑝 ⁄ )(1 -(1 -𝑓 𝑑 )𝑤 𝑒𝑞 ). Thus, the equilibrium weight is given by:

𝑤 𝑒𝑞 = 1 -exp (- 1 𝑓 * 𝜏 𝑠𝑡𝑝 ) 1 -(1 -𝑓 𝑑 )exp (- 1 𝑓 * 𝜏 𝑠𝑡𝑝 )
As expected, w eq is close to 1 when f is close to zero, and decreases when f increases. With the hypothesis that the input signal follows a Gaussian noise distribution, we can deduce the mean firing rate of each sensory neuron and thus approximate its weight.

Attention neuron

In the first version of the network the short-term plasticity was applied on the synapses connecting the input layer to the intermediate layer, in combination with the STDP rule. However, this method has the default to make low-amplitude waveforms difficult to recognize, as their low STP weights tends to shrink the receptive field size of the intermediate neurons learning it (see Section IV.C.3). A better idea is to implement an attention neuron, connected to the input layer through STP synapses, whose role is to specifically detect any action potential present in the input signal. When an action potential is present in the signal, this neuron fires continuously from the beginning to the end of the action potential. When no action potential occurs, it remains silent (Figure IV-5). Its spikes can then be used by the rest of the network as a signal that an action potential is present in the signal.

The attention neuron was implemented as a LIF neuron model. Its potential evolves dynamically according to the following equation:

𝑑𝑉 𝑑𝑡 = - 1 𝜏 𝑚 * 𝑉(𝑡) + ∑ ∑ 𝑤 𝑖 (𝑡 𝑖,𝑠 )𝛿(𝑡 -𝑡 𝑖,𝑠 ) 𝑠 𝑖 ,
where V is the neuron potential, τ m is the membrane time constant, i indexes the incoming synapses, s indexes the spikes transmitted by the synapse, w i is the synapse's weight and t i,s the spike's time. Additionally, the neuron fires when the potential V reaches its threshold. In this case, the potential is not reset so that the potential value only depends on the input signal and not on the attention neuron's activity. As the EPSP received by the attention neuron increases when values outside the noise range occur within the signal, the attention neuron's potential also increases. By setting correctly its threshold, the attention neuron can thus detect action potentials (Figure IV-6).

The important information for the attention mechanism is the amplitude of the signal and not its exact shape. Thus, at first sight, it is not necessary to have different delays to connect the attention neuron to the input layer, as introduced in Section IV.A. Though the exact shape of the signal is not important, action potentials have the property of bringing the signal outside the noise range for a relatively long time. To take advantage of this, for a better detection, one possibility is to set the membrane's time constant to a value similar to the duration of an action potential, so that the neuron's potential depends on the input values in a time window corresponding to an action potential duration. The other possibility is to use the delayed synapses from the input layer. In this case, the potential is already summed over such a time window, so a short membrane time constant insufficient. In practice, the later method proved to give better results.

The choice of the attention neuron's threshold is crucial, as it will determine the false positive and false negative rates. Our first idea was to fix a false positive rate, and to set the threshold to reach this goal. However it is difficult to precisely approximate the potential's distribution of the attention neuron for a noisy signal, especially for the tail of the distribution, which is key for having a low false positive rate. Only long simulations could help choosing the threshold with this criterion. However our goal is to have a network versatile for any situation, thus we would like to have a simpler criterion that allows to set the threshold easily. As an alternative idea, we define a stereotypic waveform with a minimum amplitude and duration, which would be the minimum waveform the attention neuron can detect. For simplicity this waveform has a square shape, which makes it easy to compute an approximation of the maximum potential reached by the attention neuron when encountering such a waveform, using the numerical approximation of the EPSP generated by each different input signal value (see Section IV.B.2).

In cases where a waveform has a peak followed by another peak of the opposite sign and thus crosses zero, we were confronted to the problem that the attention neuron potential decreased under the threshold in the middle of the action potential. Thus instead of firing continuously from the beginning to the end of the action potential, the attention neuron stopped firing in the middle of the action potential. To solve this problem, we implemented a hysteresis mechanism by increasing the attention neuron's threshold by a self-excitation value each time it fires. Thus the excitation due to the input signal needs to go down under a value that is lower than the neuron's threshold for it to stop firing.

Learning of waveform elements by the intermediate layer

The input layer generates a spike train representing the signal shape in a sliding time window. The intermediate layer receives this spike train through synaptic connections. Its purpose is thus to learn to recognize a pattern in this spike train, corresponding to parts of action potentials waveforms. To do so, an STDP rule is used on the synapses projecting from the input layer to the intermediate layer. Importantly, the intermediate layer is gated by an attention mechanism allowing it to fire and learn only when an action potential is present in the signal. In the different versions of the network, this attention mechanism was implemented either by using an STP rule on the synapses connecting the input and the intermediate layers or using an attention neuron. When the attention mechanism is implemented through the attention neuron, the latter projects excitatory synapses onto the intermediate layer, thus inducing an additional excitation when the attention neuron fires, due to presence of an action potential. In both cases, the intermediate neurons' threshold is adjusted so that the intermediate layer can fire only when an action potential is present in the signal.

LIF neuron

The neurons in this intermediate layer follow a LIF neuron model, which internal potential V varies according to the following equation:

𝑑𝑉 𝑑𝑡 = - 1 𝜏 𝑚 * 𝑉(𝑡) + ∑ ∑ 𝑤 𝑖 (𝑡 𝑖,𝑠 )𝛿(𝑡 -𝑡 𝑖,𝑠 ) 𝑠 𝑖
, where τ m is the membrane time constant, i indexes the incoming synapses, s indexes the spikes transmitted by the synapse, w i is the synapse weight and t i,s the spike times. Additionally when the potential reaches the neuron's threshold, it fires and its potential is reset to zero. Each time an intermediate neuron receives a spike from the input layer, its potential is increased proportionally to the corresponding synapse's weight. When no spike is received, the potential returns back to zero, with an exponential decay corresponding to the membrane time constant τ m . Thus if the neuron receives enough spikes in a short time compared to τ m , its potential increases enough for the neuron to fire. This type of neuron can thus detect spike coincidence, with a time precision that corresponds to τ m . We thus set τ m to be the same order of magnitude as the chosen time resolution for encoding. In our case, the number of spikes coming from the input layer is constant, thus the difference is made by the synapses weights, learnt with the STDP rule described in the next section. For a neuron to fire, enough spikes coming from potentiated synapses need to arrive simultaneously. Before the learning phase, intermediate neurons need to fire to begin the learning process. To do so, at start, the synapses weight are initialized randomly, with a mean weight that is high enough to be able to generate some spike on the intermediate layer. After the learning phase, the potentiated synapses come from different input neurons for each different intermediate neuron, thus each intermediate neuron can recognize a different spike pattern.

2.

The STDP rule used

The principle of an STDP rule is that presynaptic spikes contributing to make the postsynaptic neuron fire trigger a potentiation of the synapses. This type of rule was observed in the brain by (Bi & Poo 1998) (see Section III.C.1). This well-known STDP rule induces a potentiation of the synapse when a presynaptic spikes occurs shortly before a postsynaptic spike and a depression when a presynaptic spike occurs shortly after a postsynaptic spike. Here, we chose to use a very simple STDP rule, where the synapse is potentiated when a presynaptic spike occurs within a given coincidence time window before a postsynaptic spike, and is depreciated if a postsynaptic spike occurs alone ( For the additive STDP, each time a postsynaptic spike occurs, the synapse's weight is decreased by Δw - . Additionally if this postsynaptic spike coincides with a presynaptic spike, the weight is increased by Δw + , resulting in a total weight change of -Δw -+ Δw + . The synapse's weight is then clipped between 0 and 1. Following this rule the mean weight variation is given by:

〈 Δ𝑤 Δ𝑡 〉 = 𝑓 𝑝𝑜𝑠𝑡 (𝑝 𝑝𝑎𝑖𝑟 ∆𝑤 + -∆𝑤 -)
Where f post is the postsynaptic neuron's firing rate and p pair the probability that a presynaptic spike is present in the coincidence time window before each postsynaptic spike. The synapse's weight will converge towards 1 if this mean variation is positive and towards zero otherwise. There is thus a threshold probability p lim = Δw -/Δw + , for which if p pair > p lim , the synapse's weight converges towards 1, and if p pair < p lim , the synapse's weight converges towards 0 (Figure IV-7.b, left).

In the multiplicative case, the weight's change depends on the synapse's current weight. For a postsynaptic spike alone, the synapse's weight is increased by w*Δw -, where w is the current synapse's weight. For a postsynaptic spike coinciding with a presynaptic spike, the weight is increased by (1w)*Δw + . In this case, the mean weight variation is:

〈 Δ𝑤 Δ𝑡 〉 = 𝑓 𝑝𝑜𝑠𝑡 (𝑝 𝑝𝑎𝑖𝑟 (1 -𝑤)∆𝑤 + -(1 -𝑝 𝑝𝑎𝑖𝑟 )𝑤∆𝑤 -)
The synapse's weight is naturally bounded and the equilibrium weight, for which the mean weight variation 〈 Δ𝑤 Δ𝑡 〉 is null, is given by:

𝑤 𝑒𝑞 = 1 1 -𝑝 𝑝𝑎𝑖𝑟 𝑝 𝑝𝑎𝑖𝑟 ∆𝑤 _ ∆𝑤 + + 1
In the case Δw -=Δw + , the equilibrium weight simply become w eq =p pair right).

Thus, given p pair , the equilibrium value of the synapse is known for these two STDP versions. We can then assume that, when the equilibrium has been reached, if a neuron of the intermediate layer has properly learnt a pattern from the input layer, it fires each time this pattern occurs and remains silent otherwise. Thus, for a specific synapse, the value of p pair is close to the probability that the corresponding presynaptic neuron fires when the learnt pattern occurs. The multiplicative STDP seems to have nice properties with an equilibrium weight that is naturally bounded and can take continuous values. However, it can happen that a postsynaptic neuron learns two different patterns simultaneously, especially if they are partially overlapping. In this case, the weight of a synapse activated exclusively by one of the two learnt patterns will converge towards a lower weight as the synapse is depreciated when the other pattern occurs, and there is no simple way to force the neuron to choose one of the two patterns. With the additive STDP, this problem can be avoided by setting p lim > 0.5. In this case, simultaneous learning cannot happen because for at least one of the two learnt patterns the corresponding synapses' weights would converge towards zero. For this reasons for the rest of this work we focused on the additive STDP, whose simple properties make it easy to manipulate.

Receptive field size

At the beginning of the simulation, the synaptic weights connected to the input layer are initialized randomly. The mean weight is just high enough for neurons from the intermediate layer to fire and thus begin to learn patterns. A neuron has learnt a pattern once the weights of the synapses projecting on this neuron have all converged towards either one or zero. Then, for some input values the neuron reacts by firing whereas it remains silent for other values. The values for which the neuron fires constitute its receptive field.

The properties of the intermediate neurons' receptive fields are important for properly configuring this layer. They are studied in details in Annex A. Here, the input values are N-dimensional vectors corresponding to the signal values at the N encoded delays. The receptive field of each neuron tends to be centered on local density maxima of the possible input values, which typically corresponds to the mean shapes of action potentials emitted by different neural cells. Indeed the possible input values correspond to the different action potential waveforms, to which a Gaussian noise is added. We studied the receptive field's shape for both types of encoding methods, binary and triangular, described in Section IV.A. In both cases the receptive field has interesting properties when choosing Δ𝑤 + = 2Δ𝑤 -for the STDP rule. In both cases, it is bounded if 𝑇ℎ > 𝐸 𝑚𝑎𝑥 * (𝑁 -1), where Th is the neuron's threshold, N the number of different synaptic delays and E max is the maximum possible excitation generated by a single delay given the type of encoding. For the binary encoding method, when this bounded condition is met, the receptive field is an L1-norm ball of diameter 2DV m N-Th, where DV m is half the sensitivity range size. A receptive field with an L2-ball shape, obtained with the triangular encoding method, suits well the hypothesis of a white Gaussian noise. However this property is reached on conditions where the intermediate neuron's threshold is close to its maximum possible potential. In this condition, small relative variations on the neuron's potential greatly impact its behavior, which constitute a lack of robustness, especially since the results on the receptive field size were obtained using several approximations. Similarly, having a bounded receptive field is a nice property, but also requires a high threshold. We thus decided to use the binary encoding method, which is simpler to implement, and to allow an unbounded receptive field for more robustness in its size. The fact that the receptive field is unbounded should not impact the network behavior much, assuming that an input signal value falling in the "unbounded parts" of the receptive field has a low probability to occur. We chose to allow 3 out of the 10 used dimensions to be unbounded thus setting the threshold to 2*7*DV m , or more generally 2*0.7*N*DV m for N dimensions, DV m is half the sensitivity range for encoding. The receptive field size was then adjusted using DV m to cover most of the pattern variations due to the noise. However the receptive field should not be too big, as we do not want a receptive field to cover two distinguishable patterns. In our different network implementations, different values of DV m where tested, going from 1.5σ noise to 2σ noise , where σ noise is the noise standard deviation.

Our receptive field study does not take into account the time integration of the consecutive sampling steps. To correct this, we assumed that the values received during an interval corresponding to the membrane time constant do not change significantly and we simply multiplied the threshold as if the same value was received at each sampling step.

Winner-Take-All property mechanism

To ensure that only one neuron in the same layer fires at a time, and consequently that each neuron learns a different pattern, we implemented a classical winner-take-all (WTA) mechanism. Its principle is that when a neuron fires, the other neurons from the same layer are inhibited, preventing them from firing at the same time. Our implementation is a hard winner-take-all, in other words, at each simulation step, if at least one neuron has a potential higher that the firing threshold, the neuron with the maximum potential is selected, fires and inhibits the other neurons by resetting their potentials. As a result, instead of firing at the same time, the neurons fire one after the other, separated by a time interval that depends on their membrane time constant and the inhibition's strength. In our implementation, this time interval is about 0. Each time the intermediate layer emits a spike, it means that the input value is in the receptive field of the neuron that fires. However, we have no information whether the input value is centered in the receptive field or close to its boundary. Two input values can be very close but recognized by two different neurons because they are from either side of a receptive field boundary. The information given by the intermediate layer would be improved if we allowed several neurons to fire at the same time, with partially overlapping receptive field. Then, the number of common spiking neurons for two different input values would give information about how close to each other these two values are.

To test this idea, we implemented a 2-WTA mechanism, where we allowed 2 neurons to fire before inhibiting all neurons in the layer. Even if the two neurons fire at the same time, they should not learn the same thing so that their receptive fields do not completely overlap. We assume that in fact the most excited neuron fires just before the other, and implemented a resource-based STDP inspired from (Hunzinger et al. 2012), where synaptic changes are proportional to a global resource variable. Each time a synaptic weight change occurs, the resource variable is decreased. The resource then progressively comes back to its initial value following a time constant. The evolution of the resource variable r is described by the following equation:

𝑑𝑟 𝑑𝑡 = 1 𝜏 𝑟𝑒𝑠 (1 -𝑟) -∑ 𝑟 * 𝑓 𝑟𝑒𝑠 * 𝛿(𝑡 -𝑡 𝑠 ) 𝑠
where τ res the resource recovery time constant, f res the resource consumption factor and t s the postsynaptic spike times of the intermediate layer. In our implementation, the time constant is set short enough so that between two firing steps of the intermediate layer, the resource completely recovers. Then the first spike triggers a synaptic weight change that takes almost all the resources. Thus, for an input value, two neurons fire but only the most excited learns. We showed by simulations with a random input signal that this indeed creates overlapping receptive fields (Figure IV-10). To test if this method actually improves the spike-sorting performance, we tested this intermediate layer implementation on our single electrode simulated data sets (see Section V.A.1). For this purpose we used as attention neuron's activity the attention neuron spike trains obtained with the one of our network implementation, ANNet. We then evaluated the intermediate layer output using the entropy criterion and the distance criterion described in Section V.B.3. These tests showed that the 2-WTA strategy gives better information for classification than the 1-WTA. We also tested different values for DV m for the 2-WTA and 1-WTA. Indeed as receptive fields can overlap, they can be larger for the same distance separating their center. Though the entropy criterion did not show important differences, the distance indices show that the 2-WTA might lead to better results . Following these tests, in the last version of the network, we also decided to choose a sensitivity range with DV m =2, which seems to give better results .

Output layer

As for the intermediate layer, the goal of the output layer is to recognize pattern in the spike train emitted by the previous layer, here the intermediate layer. It is also connected to the previous layer through an all-to-all synaptic connection, implementing an additive STDP rule, and is composed of LIF neurons in its first versions. For each occurrence of an action potential in the input signal, the intermediate layer outputs a spike sequence, which characterizes the action potential waveform. As action potentials with similar waveform recurrently occur in the signal, similar spike sequences are emitted by the intermediate layer, constituting a pattern the output layer should learn to recognize. The length of these spike sequences can vary depending on the amplitude and duration and the action potential. In spite of these possible length variations, the output layer, which processes these spike sequences, should emit one spike for each spike sequence received. Moreover, the output layer should be robustly able to differentiate partially overlapping spike patterns, for example two sequences that have the same beginning but end differently. In an extreme case a spike pattern can contain a subsequence corresponding to another spike pattern to recognize. To achieve this goal, several mechanisms can be used, which will be described in the next sections.

Intrinsic plasticity

Intrinsic plasticity (IP) is a form of plasticity at the neuron level (Zhang & Linden 2003;Lazar 2009;Aswolinskiy & Pipa 2015). In our case, its purpose is to adapt the neurons' threshold to the size of the learnt pattern (e.g., length of the intermediate layer spike sequence). Indeed if the neuron's threshold is too low compared to the size of the pattern it has learnt, it will not only spike for this incoming patterns but also for shorter ones corresponding to other signal waveforms. In other words, it will be too tolerant to pattern differences and not be able to become specific to a single pattern. Also if the threshold is too low, the neuron might fire before the end of the intermediate spike sequence and be unable to discriminate patterns with the same beginning but different endings. By contrast, if the threshold is too high compared to the pattern size, it will not be able to detect the pattern. Thus, it is crucial to adapt the threshold of each output neuron to the size of the pattern it is tuned to, especially when this size is not the same across patterns.

To implement an IP, we considered two different mechanisms. In a first version, for each input spike received within a given time window before the neuron's spike, the threshold is decreased by a constant ΔTh -, and for each input spike received within a given time window after the neuron's spike, the threshold is increased by a constant ΔTh + (Figure . With this method, the neuron's threshold adjusts to fire after receiving a proportion 𝛼 = Δ𝑇ℎ + (Δ𝑇ℎ + + Δ𝑇ℎ -) ⁄ of the learnt pattern. This method has the strong inconvenient that it can only be used with architectures that allow the output neurons to fire before the end of the intermediate layer spike pattern. It was thus only used in our first network implementation MiniNet and soon replaced by the following IP mechanism. In our second method, each time the neuron fires, its threshold Th is decreased proportionally to its value, reaching a new value (1-F IP )Th. Moreover, for each spike received within a given time window around the neuron's spike, the threshold is increased by a constant ΔTh + (Figure IV-12.b). The equilibrium is reached when the increase and the decrease compensate each other, giving an equilibrium threshold 𝑇ℎ 𝑒𝑞 = 𝑁 * ∆𝑇ℎ + (1 -𝐹 𝐼𝑃 ) ⁄ for a pattern containing on average N spikes. As a result, if the neuron has learnt to recognize a pattern, the equilibrium threshold will be proportional to the average size of this pattern. During our tests, we also implemented a variation of this mechanism, where the total threshold is the sum of a fixed threshold and a variable one subject to IP. However this variation was not used for any complete network implementation. Is it thus possible to adapt each neuron's threshold individually to be proportional to the learnt pattern size. For commodity, and to be coherent with the WTA mechanism that is also applied on this output layer and selects the neuron with the highest potential, we can consider equivalently that the neuron's potential is divided by its threshold and that the neuron fires when its potential reaches 1. This threshold adaptation allows the neuron to be sensitive to the relative difference between the learnt pattern and the presented pattern rather than the absolute difference, as its potential is normalized by the learnt pattern size. Combining IP with synapses that can have negative as well as positive values would in theory allow to distinguish two patterns one included into the other. Indeed, a neuron that has learnt the large pattern will not fire for the small pattern as its threshold is high, and the neuron that has learnt the small pattern will not spike for the large pattern as synapses with negative weight, not present in the small learnt pattern, are activated and decrease the neuron potential.

The difficulty with this IP is how to deal with the neurons that have not yet learnt a pattern. Indeed these neurons should be able to learn new patterns, even the shortest ones, but should not fire before other neurons that have already learnt a pattern and might have a higher threshold. It is not possible to reach these conditions by just setting the initial threshold appropriately, and we thus need another mechanism to take full advantage of the IP.

Lateral STDP

The idea of the lateral STDP is to implement a synaptic plasticity that prevents a neuron from recognizing a pattern when another neuron is learning this pattern. To achieve this, we took advantage of the fact that when a neuron fires, other neurons from the same layer receive an inhibition signal. Thus, when a neuron receives a lateral inhibition, it triggers synaptic weight changes on its incoming synapses (Figure IV-13). For the classical STDP implemented on the same synapse, the weight convergence depends on the probability to have a presynaptic spike when a postsynaptic spike occurs. For the lateral STDP we want the weight convergence to depend on the probability to have an inhibition each time a presynaptic spike occurs. Thus, each time a presynaptic spike occurs alone, the synapse weight is increased by a constant and when the presynaptic spike is paired with an inhibition the weight is decreased by a constant. As a result, similarly to the classical STDP, the synapse weight will converge towards 0 if the probability to have an inhibition each time a presynaptic spike occurs is above a threshold probability and towards 1 otherwise. This makes a neuron more likely to fire when a pattern has not been learnt by any neuron, as no lateral inhibition occurs in this case, and less likely to fire when a pattern is learnt by another neuron. However, when the neuron is learning a pattern, we want the synaptic weights to converge according to the classical STDP. A simple way to ensure this when the two types of STDP are implemented on the same synapses, is to set synaptic weight changes for the lateral STDP to be much lower than the weight change for the classical STDP, so that the classical STDP takes over the lateral STDP.

This lateral STDP, combined with the classical STDP, can also be combined with IP. Indeed, the neuron's threshold can be initialized low, so that neurons can learn new short patterns. Then, once a neuron begins to learn a pattern, even if its threshold increases, other neurons will be less likely to fire when the pattern is presented thanks to the lateral STDP. This solution was tested in our first implementation of the network (MiniNet), improving the F-score from 0.81 to 0.89 on our simulated preliminary dataset (see Section V.A.1). We also tested it on our last network implementation (LTSNet), where it shows a systematically better result on our simulated single electrode dataset .

Delays and inhibition by the attention neuron

One possibility to make the output neurons wait till the end of spike sequence before firing is to inhibit them when the attention neuron is firing. For the neurons to be able to fire after the end of the spike sequence, they need to receive an excitation after the inhibition by the attention neuron. To do so, we introduced delayed synapses between the intermediate layer and the output layer, the same way there are delays between the input layer and the intermediate layer. This also has the advantage to bring information about the spike timing inside the spike sequence. The interval between two consecutive synaptic delays determines the time resolution for this timing information. The membrane time constant of the output neuron is chosen of the same order as this time resolution, as we want to detect a coincidence between spikes arriving at the same time but with different delays.

With this structure, illustrated in Figure IV-15, the output layer receives spikes during a few milliseconds after the end of the detected action potential. Randomly initialized neurons are thus excited during a few milliseconds. By contrast neurons that have learnt to recognize a spike coincidence pattern are excited only when the learnt pattern occurs, that is during about 0.02ms, which is the time resolution used in the intermediate layer. As described in (Masquelier et al. 2008), during learning the neuron fires always earlier in the received spike sequence, until it recognizes the beginning of the sequence. To prevent the neuron from learning what comes before the end of the action potential, the inhibition we used from the attention neuron is a presynaptic inhibition that prevents the spikes from arriving to the neuron (see Figure . These spikes are thus not taken into account for the STDP rule. The neurons that have learnt a pattern are excited during a short coincidence time when this pattern is presented whereas neurons that have not learnt anything are excited during a few milliseconds. The neuron's threshold can thus be initialized high, so that before learning the neuron fires late. As a neuron begins to learn a pattern, it fires earlier in the sequence while its threshold decreases to become proportional to the pattern size.

This structure was used in our second version of the network (ANNet), but is not totally satisfying as it is very complex, rely on a precise timing from the attention neuron and require many synapses. The LTS neuron model presented in the next section allowed us to replace this complex structure with a much simpler solution.

LTS neurons

The structure of the output layer and its plasticity rules make it complex to implement. We thus sought a more elegant and less power-consuming way to achieve the work performed by the output layer.

The output layer described so far was designed based on the fact that the main properties of the LIF neuron model is to fire when it receives an excitation. However, other interesting behaviors have been observed in the brain. For example, LTS neurons have the property to generate a potential rebound, and even spikes if the rebound is high enough, after being inhibited (Nanami & Kohno 2016) (see Figure . This is particularly interesting in our case, as we want neurons from the output layer to wait the end of the pattern before firing. Though this model has interesting properties, simulating its equation requires a very short time step as it models the potential spiking dynamic, which is computationally demanding. We designed a simpler model, falling in the Integrate and Fire category, which means that even though the potential evolution is described by a differential equation, the spikes are modeled as discrete events that occur when the potential reaches a threshold. This way to model spikes also has the advantage that the spikes can be used to reset the neuron after firing or for the WTA mechanism. Indeed, a lateral inhibition cannot be used as WTA for this type a neuron, as inhibition can trigger spikes through a rebound.

In our simplified model, the neuron's potential is governed by the following equation:

{ 𝜏 𝑚 𝑑𝑉 𝑑𝑡 = -𝑉 + 𝑞 + 𝑔𝐼 𝑠𝑡𝑖𝑚 𝜏 𝑚 𝜀 𝑑𝑞 𝑑𝑡 = -𝑞 + 𝑓(𝑉) 𝑤𝑖𝑡ℎ𝑓(𝑉) = { 𝛼 𝑛 𝑉𝑖𝑓𝑉 < 0 𝛼 𝑝 𝑉𝑖𝑓𝑉 ≥ 0
where V is the neuron potential, q is an adaptation variable that triggers the rebound after inhibition, τ m is the membrane time constant, ε is a constant that makes q vary slower than V, I stim is the stimulus current that corresponds to the received spikes and g is a constant. When the potential V reaches the neuron's threshold, both V and q are reset. After adjusting the parameters to the values shown in Table IV-1, we obtained similar properties as the DSSN model (Figure ). These parameters were used in all our tests, except for g, which plays the same role as the threshold for the LIF neuron model. Indeed, when g=1, the parameters are adjusted so that our LTS neuron fires for a total stimulus of 1.

For the LTS neuron to fire for a threshold stimulus Q th , g is set to g=1/Q th . To test this neuron model independently from the rest of the network, we tested it on arbitrary patterns. We generated different datasets each containing two different spike patterns for the output layer to discriminate. These datasets are generated with the following process. 5000 pattern occurrences are generated, separated by a time interval of 20ms, each pattern occurrence lasting between 0.5 and 1ms. For each pattern occurrence, one of the two patterns is chosen randomly and variations are randomly introduced respecting a maximum difference with the reference pattern. In each dataset, one pattern is composed of ten spikes, mimicking 10 different afferents. The second pattern's size varies between 10 and 20. The two patterns share some common spikes (stemming from the same afferents), the number of which varies from 0 to 10. Additionally, at each occurrence of a pattern in the simulation, the presented pattern is randomly modified by adding and deleting some spikes. These initial tests show that an output layer constituted of simplified LTS neurons can distinguish patterns robustly (

Adaptation to polytrodes

After testing our STDP network on single electrode data, we worked on adapting the network architecture to multiple electrodes. The principle is to take advantage of the fact that an action potential might be recorded by several electrodes, which give additional information. First preliminary tests where done with the real tetrode data (see Section V.A.2), which are recordings from 4 electrodes close enough from each other to record the same cell. These preliminary tests, done with the ANNet version of the network (see Section VI.B), show that adapting the structure can actually improve the performance. We then performed extended tests on a set of simulated polytrode recordings. The general idea to adapt the single network structure to multiple electrodes is the following. The basic network structure is constructed by duplicating the single network structure to process each electrode in parallel. With this basic structure, the shared information between electrodes is not used. Then, to take advantage of the redundancy between neighboring electrodes, lateral connections between the parallel single-electrode networks are introduced (Figure IV-19). We worked on the benefits of adapting the architecture for two specific parts of the network: the attention neuron and the output layer.

Test on attention neuron

Each electrode signal is encoded into a spike train by an independent input layers. These input layers are projecting to several attention neurons, whose role is to detect action potentials at different positions in the electrode array. With a purely parallel structure, each input layer projects to one attention neuron whose role is then to detect action potentials on the corresponding electrode. We wanted to test if connecting each attention neuron to several input layers, encoding neighboring electrodes could improve the detection performance. To do so, we designed a network where there is as many attention neurons as input layers, and each attention neuron is connected to three input layers (see Figure . This network was tested on our simulated polytrode dataset (see Section V.A.1), and run with different thresholds for the attention neuron, in order to compute a ROC score (see Section V.B.2). The results show that the structure with lateral connections did not give better results than the purely parallel structure (see Figure , and was thus not used in our final implementation. However other structures could be tested, including for example inhibitory connections.

Output layer structure

Our base structure for processing several electrodes is constituted of several parallel subnetworks, each of them processing the signal of one electrode. As for the attention neuron, at each layer of the network the different sublayers can be connected to several of the preceding sublayers to process more information. We decided to adopt such a structure for the output layer rather than the intermediate layer, as it requires fewer synapses. Ideally, the output layer should output one spike for each recorded action potential, regardless of the electrodes on which it is recorded. Each output sublayer implements a WTA mechanism which prevent several neurons from the same sublayer to fire at the same time. As an action potential can be recorded on several neighboring electrode, several output sublayers could fire for the same action potential. These structures were tested for the last version of our network (PolyNet), on our simulated polytrode dataset, which simulated a recording with ten electrodes arranged in a line. Table IV-2 shows the mean F-scores obtained. When there are less WTA connections between output sublayers than connections between intermediate and output sublayers, the performance is degraded. This can be explained by the fact that lateral connections between intermediate and output sublayers increase the number of output sublayers likely to fire for one action potential. Therefore the number of connections for the WTA mechanism needs to be adjusted to take this effect into account. Our hope was that introducing lateral connections for both the intermediate to output layer connection and the WTA mechanism would increase the performance. Though this is not the case, we can see on some simulated recordings that, when no lateral connections are used an action potential can be detected by several output sublayers (

V. PERFORMANCE ASSESSMENT

Testing datasets 1.

Simulated data

To assess the spike-sorting performance of the network, we need to test it on data with known ground truth and controlled parameters. To do so we generated simulated data, both in the case of a single electrode and of multiple electrodes.

a) Single electrode datasets

In case of a single electrode, our method was strongly inspired from [START_REF] Adamos | Performance evaluation of PCA-based spike sorting algorithms[END_REF]. For each simulated cell, the ground truth is generated according to a Poisson process. The action potential shape corresponding to each simulated cell is modeled according to the following equation:

𝑉(𝑡) = 𝐴 * cos (2𝜋 𝑡 -𝑡 𝑝ℎ 𝜏 1 ) * exp(-( 2.3548𝑡 𝜏 2 ) 2 )
Where A, t ph , τ 1 and τ 2 are parameters that determine the shape of the action potential for one simulated cell. For commodity, we introduce an intermediate parameter A max , which corresponds to the maximum amplitude of the waveform, from which A is computed. Each time a simulated neural cell fires, the corresponding waveform is added to the signal. A correlated Gaussian noise is also added to the signal, generated through a dynamical Ornstein-Ulhenbeck process, following the equation

𝑑𝑋 𝑡 = -𝑋 𝑡 𝑑𝑡 𝜏 𝑛𝑜𝑖𝑠𝑒 + 𝑑𝑊 𝑡 ⁄
, where τ noise is the time constant of the process, and W t is a Wiener process, which means the dW t are independent and identically distributed, following a Gaussian distribution. For our simulations, the time constant is set to τ noise = 0.1 ms. We chose to generate all our recordings with a sampling frequency of 20 Hz. For each simulated recording the signal-to-noise ratio (SNR) is computed as 𝑆𝑁𝑅 = 〈|𝐴 𝑚𝑎𝑥 |〉/𝜎 𝑛𝑜𝑖𝑠𝑒 , where 〈|𝐴 𝑚𝑎𝑥 |〉 is the average action potential amplitude and σ noise is the noise level.

The very first tests on our STDP network were conducted on a preliminary dataset, generated through this method. Twenty simulated recordings were generated, lasting 200 s, each containing two different waveforms (see Figure V-1) occurring at 10Hz and 12Hz respectively, with a noise level σ noise of 2, leading to an SNR of 3.5. This preliminary dataset has two main drawbacks. First the noise level does not vary between recordings, and second the chosen waveforms have a width of about 2ms, which is quite large for an action potential and makes them easier to detect. We thus generated a more extensive dataset, which then constituted our main simulated single electrode dataset. The recordings of this dataset also lasts 200 s. They contain three different waveforms defined by the parameters shown in Table V-1 (see also Figure V-2), occurring at 3.3Hz each. Seven different noise levels were used, corresponding to seven different SNR (see Table V-2). For each noise level, ten recordings were simulated, leading to a total of 70 simulated recordings, each of 200 s duration.

b) Polytrode dataset

For tests with multiple electrodes, we needed to model the fact that an action potential generated by a neural cell is recorded by nearby electrodes with an amplitude and a shape that differ depending on the distance and the geometry of the neuron relatively to the electrode.

We thus implemented a simple model of extracellular action potentials, able to reproduce different variations of their temporal shape depending on the electrode position. An intracellular action potential is a positive peak of the membrane potential. This signal results from voltage-sensitive sodium and potassium channel ion fluxes flowing through the neuron's membrane at the initial segment, characterized by a positive transmembrane current that first flows inward and then outward. These active currents are compensated by capacitive and leak currents in the rest of the arborization. Detailed models of action potentials show that this transmembrane current peak, which is strongest in the soma, is also found in remote neuron parts, with a decreased amplitude, a larger width and a delayed peak [START_REF] Gold | On the Origin of the Extracellular Action Potential Waveform: A Modeling Study[END_REF]. Considering that the extracellular space is homogeneous and conductive, the potential at one point of the extracellular space is proportional to the sum of the local transmembrane currents divided by the distance. The difference in shape of the same action potential recorded at different points of the extracellular space can thus be explained by the shape differences and the delays between the transmembrane current waveform in the different parts of the neuron. In The noise we used is a Gaussian noise with spatial and temporal correlations. The spatial correlation between two points decreases exponentially with the distance. The exponential decrease is characterized by a constant d noise . Thus, we computed the correlation matrix between all electrodes and generated a spatially correlated Gaussian noise at each time step. The temporal correlation was then obtained using an Ornstein-Ulhenbeck process with a time constant τ noise , as for the single electrode simulations.

For our simulated polytrode dataset, we generated recordings lasting 100 s, sampled at 20 kHz, with a noise level σ noise = 1. For this dataset, we chose a short spatial correlation for the noise (d noise = 1 μm), so that the correlation between two electrodes was negligible. The temporal noise correlation was the same as for the single electrode simulations (τ noise = 0.1 ms). For each recording we simulated 10 electrodes, arranged along a line, and spaced by 30 μm. In the different recordings, we simulated 2, 3 or 4 neural cells, randomly positioned along a line distant of 20 μm from the electrodes' line (Figure V-4). For each neural cell, the segment along which the intracellular action potential propagates is 100-μm long and its direction is chosen randomly in the half-space directed towards the electrodes (Figure V-4). The transmembrane current waveform, shown in Figure V-3, is the same for all simulated cells and propagates at the speed of 50mm.s -1 , which is quite slow but allows to see significant variations in shape for the action potential waveforms. Given the characteristics of the simulated neural cells and the simulated electrodes, we can compute the action potential waveform generated on each electrode by each cell. Then for each neural cell, the waveforms' amplitude is adjusted so that the maximum amplitude for each cell matches the chosen SNR. We generated recordings with an SNR of 3, 4, 5 or 6. The maximum amplitude chosen for each neural cell to obtain the different SNR are detailed in Table V-3.

Real recordings

Our network was also evaluated withreal recordings from hippocampus region CA1 of anesthetized rats, available from the Buszaki Laboratory [START_REF] Henze | Intracellular features predicted by extracellular recordings in the hippocampus in vivo[END_REF][START_REF] Henze | Simultaneous intracellular and extracellular recordings from hippocampus region CA1 of anesthetized rats[END_REF]) (datasets d533101 and d11221.002). These two recordings are tetrode recordings, associated with an intracellular recording giving the ground truth for one neural cell (Figure V-6). The d11221.002 recording has a sampling frequency of 20 kHz. The d533101 recording, having an original 10-kHz sampling frequency, was up-sampled, for convenience, at 20 kHz using a Wittaker-Shannon interpolation. Before being fed to the network, the signals were band-pass-filtered using a first-order Butterworth filter (300Hz -3000Hz).

Performance indices 1. Spike-sorting performance

When assessing the performance of a spike-sorting algorithm, one can be confronted to three types of errors: 1) false positive errors, when the algorithm detects an action potential where there is none, 2) false negative errors, when an action potential is not detected, and 3) clustering errors, when an action potential is detected but classified in a wrong cluster. False positives and false negatives are both detection errors. We thus defined several performance indices that quantify each type of error as well as the global performance.

The ground truth and the algorithm output are respectively constituted of N and K clusters.. We denote H ij the number of elements of true cluster i detected and classified in output cluster j, FN i the number of elements of true cluster i not detected by the algorithm and FP j the number of elements in output cluster j not corresponding to any true action potential. The number of elements in true cluster i is denoted T i , and the total number of true action potentials is denoted T. The number of elements in output cluster j is denoted O j , and the total number of detected action potentials (including false positives) is denoted O.

To evaluate the detection performance of the algorithm, we take into account the total number of false positives 𝐹𝑃 = ∑ 𝐹𝑃 𝑗 𝐾 𝑗=1

, the total number of false negatives 𝐹𝑁 = ∑ 𝐹𝑁 𝑖 𝑁 𝑖=1

, and the total number of action potentials correctly detected without taking into account their classification, 𝐷 = ∑ ∑ 𝐻 𝑖𝑗 𝐾 𝑗=1 𝑁 𝑖=1

. We can then define the precision P D , recall R D and F-score F D as performance indices for detection:

{ 𝑃 𝐷 = 𝐷 𝐷 + 𝐹𝑁 = 𝐷 𝑂 𝑅 𝐷 = 𝐷 𝐷 + 𝐹𝑃 = 𝐷 𝑇 𝐹 𝐷 = 2𝐷 2𝐷 + 𝐹𝑃 + 𝐹𝑁 = 2𝐷 𝑇 + 𝑂 = 2 1 𝑃 𝐷 + 1 𝑅 𝐷
To evaluate the clustering performance, we first need to find a correspondence between the true clusters and the output clusters, as we do not know a priori which output cluster corresponds to which true cluster. We define a correspondence M as a set of (i,j) pairs, where i is a truth cluster index and j an output cluster index. To be valid, M should contain neither the same true cluster twice nor the same output cluster twice. Given a correspondence M, the number of correctly classified elements is

𝐻 𝑀 = ∑ 𝐻 𝑖𝑗 (𝑖,𝑗)∈𝑀
. The optimal correspondence is the one maximizing H M , computed through an exhaustive search, and we denote 𝐻 * = 𝑚𝑎𝑥 𝑀 (𝐻 𝑀 ) the optimal number of hits. H * thus corresponds to the number of correctly detected and correctly classified action potentials. We can then define a clustering performance index:

𝐶 = 𝐻 * 𝐷

To evaluate the global performance of the algorithm, we combine C and F D to obtain an F-score that takes into account all types of errors:

𝐹 = 2𝐻 * 𝑇 + 𝑂 = 𝐹 𝐷 * 𝐶
Given a true cluster i and an output cluster j, we can also defined the precision P ij , recall R ij and F-score F ij for this specific pair, as follows:

{ 𝑃 𝑖𝑗 = 𝐻 𝑖𝑗 𝑂 𝑗 𝑅 𝑖𝑗 = 𝐻 𝑖𝑗 𝑇 𝑖 𝐹 𝑖𝑗 = 2𝐻 𝑖𝑗 𝑇 𝑖 + 𝑂 𝑗
These paired scores are used to analyze in detail how well each action potential waveform is detected and classified. They are also used to assess the performance withthe real dataset, for which the truth is only known for one neural cell. The scores used in this case is the paired scores obtained for this true known cell and the best matching output neuron.

ROC curve for the attention neuron

The role of the attention neuron is to detect action potentials within the input signal. Depending on how its threshold is chosen, the attention neuron generates more or less false positive and false negative errors. To assess the efficiency of the attention neuron independently from the choice of its threshold, we decided to use a variation of the Receiver Operating Characteristic (ROC) curve. Usually, a ROC curve shows the evolutions of both the false positive rate and the true positive rate (or recall) when the threshold evolves. In our case, we cannot compute a false positive rate since we have no negative labels (all noisy part of the signal can be qualified as negative). Instead we used the recall (true positives over positives) and the precision (true positives over predicted positives). The idea of the ROC curve is the following. When the threshold is very low, the recall is 1 whereas the precision is 0. As the threshold increases, the recall decreases while the precision increases, until they reach respectively 0 and 1. This can be represented in a curve showing the trajectory of the precision and recall values (see Figure V-7). For perfectly separable data, there is a point for which both recall and precision are equal to 1. In this case, the area under the ROC curve is 1. Otherwise, the recall begins to decrease before the precision reaches 1, leading to an area under the curve lower than 1. Applied to the attention neuron, this area under the ROC curve thus constitutes an index of the performance on a particular recording, independently of the threshold choice. This index was used for example in Section IV.E.2 to compare different structures for the attention neuron.

Intermediate layer quality

The output of the intermediate layer is an important intermediate result. To evaluate the quality of the intermediate layer, we defined several indices to evaluate its output knowing the ground truth.

Each time an action potential occurs in the input signal, the intermediate layer generates a sequence of spikes stemming from different intermediate neurons. Though the exact timing of these spikes could be exploited to classify the spike sequence, not all versions of our output layer use this information. Thus for the sake of simplicity, to describe the spike sequence generated for each action potential occurrence, we simply count how many times each intermediate neuron fires. For each action potential occurrence we know its label L from ground truth and the corresponding intermediate layer output is described by a vector X of N natural integers where N is the number of intermediate neurons and X i is the number of spikes emitted by the i th intermediate neuron.

Our first quality index I H is based on entropy. We want to compute the entropy of the action potential label given the intermediate layer output, which tells us how much information is missing in the intermediate layer output to find the correct label. To do so, we computed an approximation of the probability to have X=x, a priori or given a label, using the density of points around x. We can then compute the conditional entropy with the following formula:

𝐻(𝐿|𝑋) = ∑ 𝑝(𝑥)𝑝(𝑥|𝑙)log ( 𝑝(𝑥)𝑝(𝑥|𝑙) 𝑝(𝑙) ) 𝑥,𝑙
We then normalize our index with respect to the entropy of the label:

𝐼 𝐻 = 𝐻(𝐿|𝑋) 𝐻(𝐿)
This entropy index gives us an objective index on the intermediate layer quality in case of an ideal classification, however it does not reflect how the output layer works. Thus we constructed a more concrete index based on the distances between intermediate layer spike sequences. We grouped the intermediate layer outputs, described by the same N-dimensional natural integer vector, into K clusters C i according to their ground truth label. We first computed the centroid of each cluster 𝑐 𝑖 = 𝑚𝑒𝑎𝑛({𝑋/𝑋 ∈ 𝐶 𝑖 }). To estimate the size S i of each cluster i we took the 0.9 quantile of the distance of each point of the cluster to its centroid:

𝑆 𝑖 = 𝑞 0.9 ({|𝑋 -𝑐 𝑖 |/𝑋 ∈ 𝐶 𝑖 })

For each cluster i we also computed an inter-cluster distance D i as the minimum distance to other clusters based on the centroids' distance:

𝐷 𝑖 = min ({|𝑐 𝑖 -𝑐 𝑗 |/𝑗 ∈ ⟦1, 𝐾⟧, 𝑗 ≠ 𝑖})
We then took the ratio of the median cluster size and the median inter-cluster distance as our distance index I D .

𝐼 𝐷 = 𝑚𝑒𝑑𝑖𝑎𝑛({𝑆 𝑖 } 𝑖∈⟦1,𝐾⟧ ) 𝑚𝑒𝑑𝑖𝑎𝑛({𝐷 𝑖 } 𝑖∈⟦1,𝐾⟧ )
Comparison with other spike-sorting methods

Tests with Osort and Wave_clus

To assess our method, we chose to test two other spike-sorting methods: Osort (Rutishauser et al. 2006) and Wave_clus (Quiroga et al. 2004). Both of these methods give the possibility to sort the data automatically, without supervision. These two methods were tested on our single electrode simulated dataset (see Section V.A.1) and the real tetrode data (see Section V.A.2).

Concerning Osort, the detection was done using the positive amplitude thresholding for the simulated data and the negative amplitude thresholding for the real data. The extraction threshold was set to 4. The alignment was done, using the findPeak method, on the maximum for the simulated data and on the minimum for the real data. The sampling frequency was set to 20 000, to match the input data sampling frequency. Other parameters were set to their default values.

Concerning Wave_clus, all the parameters were set to their default value, except for the threshold's type, which is set to positive for simulated data and negative for real data, and for the number of samples stored before and after the event that are both set to 30. Wave_clus gives the option to adjust the temperature parameter after the first clustering pass. We did not use this option and used the automatically chosen value, as our goal is to compare fully automatic methods. Wave_clus proposes an option to assign unsorted action potentials to found clusters. For each recording we used this option once, after the first clustering pass.

Statistical tests

The single electrode simulated data and the real tetrode dataset were both used to compare different methods, in particular to compare the STDP network to Osort and Wave_clus. For the simulated data, several recordings with similar characteristics were generated (see Section V.A.1). Each of the compared methods are run once on each simulated recordings. The different methods' results on a set of recordings with similar characteristics are then compared. For the real tetrode data (see Section V.A.2), the STDP method was run 8 times on each channel, Wave_clus was run 8 times on each channel, and Osort was run once on each channel as its result was deterministic. The different methods' results on the same recording are then compared. As the variances were significantly different for the different groups (as assessed by a Bartlett test), a Welch test was used for 2-by-2 comparisons, except for the comparison with Osort on the tetrode data where a one-sample t-test was used. A Bonferroni correction was used for each set of multiple comparisons.

VI. IMPLEMENTATIONS AND RESULTS

MiniNet 1. Implementation

The first version of our network, consisted in one input layer, one intermediate layer implementing both an STDP rule and an STP rule, and one output layer implementing an STDP rule (Figure VI-1). In this version of the network, the encoding frequency was 20 kHz, corresponding to our recordings' sampling frequency, without up-sampling. The number of encoding delays used in the input layer was large, corresponding to a signal window of 1.2 ms. As improvements we also implemented a lateral STDP and an intrinsic plasticity on the output layer. The main characteristics of this network are summarized in

Δw pre

Weight change for each presynaptic spike -0.9* Δw lat

Results

The network was first tested on our preliminary simulated dataset (see Section V.A.1), which consists in signals containing two action potentials. These action potentials have an SNR of 3.5 and a quite large duration of about 2 ms. This first network, with neither lateral STDP nor IP, gave satisfactory results on the preliminary dataset, with a mean F-score of 0.81. Figure VI-3 shows an example of results on one recording from this dataset. It can be seen that the learning converge after about 100 s, and that the waveform 1 is recognized by the output neuron 3 with an F-score of 0.83, and the waveform 2 is recognized by the output neuron 1 with an F-score of 0.88.

Adding a lateral STDP to the output layer improved the performance of the network. Indeed, with lateral STDP, the mean score was brought from 0.81 to 0.88. As preliminary tests, an intrinsic plasticity was implemented on the output layer in combination with the lateral STDP. It was tested qualitatively, on signals containing action potentials with slightly different amplitudes and durations. These tests showed that the neuron's threshold is actually able to adapt to the size of the learnt pattern. Although this first version worked well on the preliminary dataset, the performances were not as good with more realistic data, especially when action potentials were shorter. One reason was that the input layer was adapted for slow action potentials, because of the slow sampling frequency used for encoding. This was corrected in the next versions. Another problem was the interaction between the STP and the STDP on the intermediate layer, which made it difficult to robustly recognize lowamplitude action potentials. Indeed, without STP, an intermediate neuron's potential depends on how close the input signal is to the shape it has learnt. When the STP is added, its potential also depends on the signal's amplitude. The reduced potential for low amplitudes requires low-amplitude waveforms to be very close to the learnt waveform to be recognized. This led us to introduce an attention neuron (which was added in the patent) in the next versions of the network.

Results

ANNet was tested both on our single-electrode simulated dataset (see Section V.A.1) and on real recordings (see Section V.A. Figure VI-6 shows an example of the network's output for a simulated recording and for a real recording, as well as the F-score evolution along the whole recording. It can be seen that, for the simulated recording, three output neurons are active and accurately reproduce the true activity.

Concerning the real recording, for which the ground truth is only known for one neural cell, one of the output neuron reproduces the known ground truth. Though it cannot be objectively evaluated, two other action potentials seem to be detected by two other output neurons. The performance of ANNet was compared to two other spike-sorting algorithms, Osort (Rutishauser et al. 2006) and Wave_clus (Quiroga et al. 2004) (see Section V.C). On simulated data with a high SNR, the ANNet's scores were lower than the two other algorithms, though the difference remained relatively small. In contrast, for SNR lower than 6, ANNet performed significantly better than Osort and Wave_clus ( Finally, the robustness of this network was also tested on two additional simulated recordings. In the first one, the firing rates were different between the three simulated cells (

Preliminary tests on tetrode data

Our ANNet implementation of the network, designed initially for single electrode, was tested on real tetrode recordings. We tested two different methods to adapt the feedforward network structure to process the tetrode data (see Section V.A.2). In a first adapted architecture, the input layer was duplicated four times for each electrode, and these four inputs layers project to a common attention neuron and a common intermediate layer ( On the d533101 recording, with the highest SNR, the tests show a clear improvement of the results when taking into account all four channels, with even better results when using the second structure. On the d11221.002 recording, there was no significant improvement, and the performance dropped with the second structure, with a high variation between different runs. This is thus difficult to conclude on these two datasets. The next version of our network, adapted to multiple electrode, was thus tested on more data generated through simulation, with more electrode and a controlled SNR and controlled number of cells, to check the effect of different network structures (see Section VI.D). V inhib Post-inhibition potential 0 q reset Reset value of q after firing 0 q inhib Reset value of q after an inhibition 0

Results

This implementation was tested on our polytrode simulated dataset (see Section V.A.1). Figure shows an example of output on a sample of 250 ms of a simulated polytrode signal with four simulated neural cells and an SNR of 6. Each action potential in the signal triggers a spike from a unique neuron in the entire output layer, most of the time in the sublayer corresponding to the channel where the action potential amplitude is the highest. Figure VI-18 shows the scores obtained on the same signal for each pair of true cells and output neurons. Most action potentials emitted by one specific cell are detected by one specific output neuron, though cell 1 is detected by the same neuron as cell 3 at the beginning of the simulation, and a few false positives and classification mismatches can be observed. This is consistent with the mean clustering, precision and recall scores observed with different SNR . Although the recall is quite good, the precision and the clustering score are low, due to the high number of false positives and classification mismatches. Indeed when taking each output neuron separately, the number of corresponding mismatches and false positives is low, as shown for example in Figure VI-18.b, but due to the high number of output neurons (150), the total number of mismatches and false positives become important. The misclassifications could be explained by the concurrence between neighboring output sublayers, as they do not receive exactly the same stimulus.

The first output sublayer to fire is not systematically the one corresponding to the highest action potential amplitude, as we would ideally expect. This can be seen for example in channel 4 of Figure VI-17 on which the action potential detected for the true red waveform has a lower amplitude lower than the one occurring on channel 5.

Estimation of the resources used

For each of our different implementations, we made a rough estimation of the resources used for a neuromorphic implementation, in terms of number of neurons, synapses and transmitted spikes. For the input layer, the estimation was done for encoding a signal range of 20σ noise . The number of spikes generated and transmitted is expressed in spikes per second on the input layer, and then in spikes per action potential present in the signal. The results obtained are gathered in Table VI-6, Table VI-7 andTable VI-8, for the three single-electrode networks, respectively. Depending on how the neuromorphic implementation is done, some of these implantation bricks can be more limiting than others. However, with our network structure, the limiting factor is likely to be the synaptic connection between the input and the intermediate layers, as the numbers of synapses and of spikes transmitted are very high.

Another aspect that can be relevant for power consumption in a neuromorphic implementation is the number of synaptic weights' changes. For the connection between the input layer and the intermediate layer, the number of weights' changes can be estimated at respectively 12720, 5700 and 10000 changes per action potential for respectively MiniNet, ANNet and LTSNet. An estimation of 1pJ per weight change [START_REF] Xiong | Low-power switching of phase-change materials with carbon nanotube electrodes[END_REF][START_REF] Chin | Ultra-Low Switching Power RRAM Using Hopping Conduction Mechanism[END_REF]) and an electrode signal containing about 100 action potentials per second lead to an approximation power consumption of 1µW. For the polytrode implementation the quantity of resources used is multiplied by the number of electrodes, except for the intermediate to output layer connection, for which the number of synapses is also multiplied by the number of lateral connections between sublayers (3 in our case). However, this connection should not be the limiting element in this structure.

Preliminary FPGA Implementation

The different versions of the network were tested by simulations on a workstation. This allows to test the spike-sorting performance of the network but not the efficiency of the implementation, as it is designed to be executed on parallel hardware. Neuromorphic technologies not being mature enough to implement our network, we chose as an intermediate step towards an embedded implementation, to begin an implementation on FPGA devices. This technology has both advantages to be flexible and to offer parallelism architecture opportunities. To this end we started a collaboration with Takashi Kohno and Timothée Levi at the University of Tokyo, whose team has already been working on

VII. CONCLUSION AND PERSPECTIVES

A.

Main contributions

The goal of this thesis was to design a fully automatic online spike-sorting algorithm using an STDP network, in the perspective of an implementation on neuromorphic hardware. The network developed during this work meets in large these requirements. First, the algorithm is entirely neuromorphic. Apart from a simple band-pass filtering, that can be achieved by simple passive analog circuitry, no preprocessing is done before feeding the input signal to the network. The conversion of the analog signal into spike trains is done by a set of simple units that can be assimilated to sensory neurons. No post-processing is needed either, as the network is designed to output spike trains that correspond to the sorted cells' spiking activity. Second, the method is fully automatic. The network has been designed and parameterized taking into account the main characteristics common to all extracellular recordings, in particular the duration of an action potential. The parameters of the network depend only on two characteristics of the input signal that may change between recordings: the noise level and the amplitude range, which can both be easily estimated. Third the algorithm can theoretically be executed online. Indeed the network is designed to process the input signal on the fly. The theoretical latency of the response depends on the dynamic of the neuron models and the synaptic transmission delays used in the network and does not exceed a few milliseconds. In practice, the execution time depends on the physical implementation of the network, in particular on what type of hardware it is executed. Though the algorithm was designed for a neuromorphic implementation, the practical implementation on neuromorphic hardware is beyond the scope of this work. Though, preliminary implementation on FPGA gave good results (see Section VI.F).

An important point when designing an algorithm to solve a problem is to check if the problem is solved correctly. This is not trivial for spike-sorting, as the ground truth is most of the time unknown. A common way to assess the performance of a spike-sorting algorithm is to test it on simulated data, mimicking real recordings, but for which the ground truth is thus known. In this work we generated our own simulated data, using two different methods (see Section V.A.1). The first method, used for single electrode recordings, uses predefined action potential waveforms for each true cell. The second method, used for multiple electrode recordings, uses an extracellular action potential model based on a simplified neural geometry. The simplicity of this model allows to quickly generate multiple electrode data without advanced knowledge in electrophysiology, in contrast with more realistic models that require to simulate complex neural structures [START_REF] Hines | NEURON : A Tool for Neuroscientists[END_REF][START_REF] Hagen | ViSAPy: A Python tool for biophysics-based generation of virtual spiking activity for evaluation of spike-sorting algorithms[END_REF]. The performance assessment on simulated data was supplemented with tests on real recordings, with a partially known ground truth. These tests showed that our algorithm reaches the state-of-the-art performance on single electrode recordings, using a radically new method (see Sections VI.B and VI.C).

It also has a correct qualitative behavior on multiple-electrode recordings (see Section VI.D). In particular we were able to ensure that each action potential in the signal generates only one output spike, in spite of being processed through several parallel channels. Moreover, as expected for an STDP network, our network needs only a few occurrences of a pattern (about a hundred) to learn to recognize it, in contrast to deep neural networks that require to be trained on large datasets.

Beyond the spike-sorting application, many mechanisms were developed for this network, to deal with the specificities of the spike-sorting problem. These mechanisms could be used for other applications.

First the patterns to recognize in the signal had a strong temporal aspect as the relative times of the values taken by the signal are crucial to discriminate the patterns (i.e. the action potential waveforms). This is not common in STDP applications. Indeed LIF neurons are naturally able to detect coincidences, but need additional mechanisms to process this temporal aspect. This problem can be simply solved by using delays in the synaptic transmissions [START_REF] Natschläger | Spatial and temporal pattern analysis via spiking neurons[END_REF]Ghosh-Dastidar & Adeli 2007;[START_REF] Ghosh-Dastidar | A new supervised learning algorithm for multiple spiking neural networks with application in epilepsy and seizure detection[END_REF], which we did in our network (see Section IV.A and IV.D.3). The network was successfully able to distinguish two waveforms that were symmetric along time (one positive and then negative and the other negative and then positive), showing that the temporal aspect is correctly taken into account. Such network structure using transmission delays has already been used for EEG pattern recognition (Ghosh-Dastidar & Adeli 2007), though in this study, learning is not done through STDP. On the other hand, some studies applied STDP networks to auditory or visual pattern learning, but without using transmission delays (Suri et al. 2013), which limit the network to recognizing short timescale patterns and not complex sequences. The ability to recognize temporal patterns using transmission delays could thus be useful for applications such as video processing or sound and language processing. It would allow for example discriminating similar objects moving in different directions in a video stream, or recognizing whole words or sentences in an audio recording.

Another important aspect of the network is the attention mechanism (see Section IV.B). Artificial neural networks are often presented isolated patterns, removing the need to select the relevant parts of the input stimuli. Recognizing patterns in a composed input stimulus, possibly containing several patterns among irrelevant inputs, is a much more complex problem. Convolutional neural networks solve this problem by scanning all the input stimulus to find local patterns. However, it would be more interesting to process only the relevant parts of the input stimulus through an attention mechanism.

In this work, we implemented an attention mechanism thanks to a short-term plasticity rule, able to detect changes in the input stimulus. This mechanism could possibly be extended to visual processing, for example to detect moving objects in a video stream.

Finally, when designing our spike-sorting network, we were confronted to the problem of input patterns with different sizes due to action potentials of different durations. A neuron learning a pattern should thus adapt its error tolerance to the size of the learnt pattern. This was done using an intrinsic plasticity rule that adapts the value of the neuron's threshold to the size of the learnt pattern (see Section IV.D.1). As our patterns are temporal, we also needed a mechanism to ensure that neurons would not fire before the end of the pattern and thus miss information to recognize it. Our most satisfying solution to solve this problem was to use an LTS neuron model that generates a potential rebound after an inhibitory stimulus. During tests on our output layer structure, we showed that combining these two mechanisms (see Section IV.D.4) makes the network model able to discriminate patterns with very different sizes, and even two patterns strictly included one into another. A possible alternative application of this approach could be the processing of complex visual scenes, where relevant objects can have different sizes in the input picture.

B. Discussion and future work

Our STDP network works well to process single electrode signals. However, the long-term goal is to develop an STDP network able to process signals from dense MEAs. Our implementation of a network processing a multiple-electrode signal is preliminary. We were able to design a network architecture that fires one spike for each action potential, on a channel where this action potential is visible, which is the intended behavior. However, this network still generated too many misclassification errors, in part due to the fact that action potentials stemming from the same neural cell are not robustly classified in the same channel. Future efforts should thus focus on adjusting the architecture to solve this problem. Possibilities are to simply adjust parameters such as the connection weights between layers, to introduce new plasticity mechanisms, or to add a layer dedicated to this problem. For example, the lateral STDP already implemented on the output layer could be adjusted depending on which layer the WTA inhibition comes from. Another aspect that could be investigated is the influence of the number of neurons in each layer on the network performance. Our guess is that, provided that there is already enough neurons to learn all presented patterns, adding additional neurons should not impact the spike-sorting performance. Indeed, all inactive neurons that have not learnt any pattern behave the same way, and their number should not impact the behavior of neurons responding to a pattern. However this hypothesis has not been investigated experimentally.

In this work, we mostly investigated the simplest cases of spike-sorting situations. However, as it has been highlighted in Section II, some specific difficulties emerge when performing spike-sorting on realistic extracellular signals. One of them is the problem of non-stationary data, when the action potentials' shapes slowly change during the recording. As our network is continuously learning, we could expect that it would be robust to slow changes in waveforms. However this still needs to be confirmed by running the network on long non-stationary recordings. Another problem in spikesorting is to robustly classify action potentials occurring in bursts, in which case they display decreasing amplitudes. Without modification, our network would classify as different action potentials occurring in a burst, although they would stem from the same cell. A possibility to solve the problem would be to add a layer that would classify them as stemming from the same neural cell, by taking advantage of the temporal correlation between them. Finally, a major difficulty is to be able to recognize action potentials overlapping in time. Methods able to do so explicitly model the fact that action potentials sum up. Our way to encode the signal into a spike train does not take into account this property, making it difficult to separate action potentials occurring at the same time on the same electrode.

Solving this problem would require changing the encoding paradigm, or adding a mechanism to the input layer to take into account additivity.

All our simulations were done on a computer workstation whose sequential architecture is not adapted to the execution of artificial neural networks. As a first step towards a neuromorphic implementation, part of the network was implemented on an FPGA. Even though this work needs to be completed, it showed the feasibility of an FPGA implementation. Future work should implement first a complete single electrode network, then a multiple electrode network, and study how much electrodes can be simultaneously processed given the FPGA resources. On a long-term perspective, this network could be implemented on neuromorphic hardware using memristive devices. Synapses implemented through memristors or RRAMs are able to mimic STDP and short-term plasticity [START_REF] Ohno | Short-term plasticity and long-term potentiation mimicked in single inorganic synapses[END_REF]Saïghi et al. 2015;Werner et al. 2016), which are both used in our network. Neurons, often implemented using CMOS technologies, can implement integrate-and-fire models as well as more complex models (Indiveri et al. 2013;Sourikopoulos et al. 2017). However mechanisms such as intrinsic plasticity have not yet been tested on neuromorphic device, and thus require further developments.

The limiting element in a neuromorphic implementation is the number of synapses. Though efforts were done to not use an excessive number of synapses, the use of several transmission delays between two layers increases their number. A first idea to reduce the number of synapses could be to reduce the number of delays between two layers, possibly compensating this loss by implementing more layers, each acting at a different time scale. Indeed, it is known in the field of deep-learning, using formal neural networks, that deep structures are more optimal than large structures. It would be interesting to investigate if the same effect is observed with STDP networks. Another possibility would be to implement a plasticity rule on the synaptic transmission delays, such as for example delay plasticity based on spike timing [START_REF] Eurich | Dynamics of self-organized delay adaptation[END_REF][START_REF] Lücken | Pattern reverberation in networks of excitable systems with connection delays[END_REF], removing the need to use several synapses with different delays.

C. Impact and perspectives

Artificial neural networks, applied to pattern recognition problems, have undergone significant development in the last decades, thanks to the availability of huge datasets and powerful computers. The advent of neuromorphic chips could open a new era in pattern recognition, by allowing to implement such applications into embedded low power devices. However many developments remain to be done in this field, both from the hardware and software points of view, as the available techniques are in their early stages. In particular, STDP networks constitute a new field of artificial intelligence, with few concrete applications. Many different mechanisms are being explored to better regulate the learning dynamic of these networks. Some of them have been used in our network, such as short-term plasticity, intrinsic plasticity, transmission delays, or winner-take-all mechanism. Others were not explored in this work, such as metaplasticity, plasticity on inhibitory synapses, plasticity on delays, but could however be useful for the application to spike-sorting. Managing to implement a spike-sorting method into an embedded neuromorphic chip would strongly benefit to BCI applications, for which real-time processing is crucial. Indeed, no available spike-sorting methods completely meet the need of automatic real-time processing for the ever growing number of electrodes in recording devices. Thus human BCI experiments currently do not use spike-sorting in practice for sake of simplicity, although it has been shown to improve decoding performances [START_REF] Todorova | To sort or not to sort : the impact of spike-sorting on neural decoding performance[END_REF]). Ideally, spike-sorting would be done at the electrode level thanks to such device, thus avoiding to transmit a massive flow of data to an external processing unit. Beyond spike-sorting, the STDP network paradigm could be extended to downstream processing of MEA recordings, such as behavior decoding for fundamental neurosciences or BCI, or to process other types of neural data recordings. Improving neural data processing through neuromorphic implementation would allow analyzing data from many neurons simultaneously and thus improving our understanding of brain functioning. More generally, spike-sorting is a pattern recognition problem and the mechanisms developed in this work to process MEA recordings could be used for completely different applications, as for instance visual pattern recognition or language processing.

VIII. ANNEXES Intermediate layer receptive field shape

The input signal is encoded into a spike train by the input layer, which is then transmitted to the intermediate layer through synapses implementing an STDP rule. At the level of the synapse, we know that the synapse weight will converge towards one or zero depending on the probability to have a presynaptic spike in the STDP coincidence window each time a postsynaptic spike occurs. Here we study the learning process steady state at the level of a neuron from the intermediate layer, in other word, the neuron receptive field and the weight of its incoming synapses after learning. At each sampling step, the spikes received by the intermediate, transmitted from the input layer through delayed synapses, represent the signal value at N different delays from current time. A pattern to learn can thus be seen as an N dimensional vector, encoded into spikes by the input layer. We assume in this section that the only variation between action potential waveforms from the same cell is due to a white Gaussian noise added to the signal. Thus at each occurrence of a pattern, a Gaussian noise is added to the N dimensional vector representing this pattern, which will induce variations in the set of spikes encoding it. Here we will also neglect the effect of time integration and assume that the potential of the neuron is mostly due to the last set of spikes received from the input layer.

Let's focus first on the one-dimensional case. We denote x the input value received by the network. Each sensory neuron n can be associated with a center value c n , that corresponds to the center of its sensitivity range. For an input value x, the number of spikes emitted by a sensory input neuron is given by an activation function a cn (x) that is the same for each sensory neuron but shifted according to their center value:

𝑎 𝑐 𝑛 (𝑥) = 𝑎 0 (𝑥 -𝑐 𝑛 )

Let W n be the weight of the synapse connecting the n th sensory neuron to the studied intermediate neuron. Given the way the sensory neurons are organized, we can define a weight density:

𝑤(𝑐 𝑛 ) = 𝑊 𝑛 𝐷𝑉 𝑖 ⁄

Where DV i is the difference between two consecutive sensory neuron center values. In this onedimensional case, assuming that the sensory neurons cover a value range large enough not to have border effects, the excitation of the intermediate neuron for an input value x can be written: For simplicity and assuming that DV i is small, we can rewrite this expression in a continuous form:

𝐸 1 (𝑥) = ∫ 𝑤(𝑐)𝑎 0 (𝑥 -𝑐)𝑑𝑐

+∞ -∞

Whereas a 0 is known by definition, we need to do hypotheses on the form of w. We assume for the moment that only one pattern x 0 is presented to the network (or that the different patterns presented are far enough from each other not to interact). Without losing generality we can assume that x 0 =0.

Thus the input value x of the network follows a normal distribution. Given the symmetry of the problem and the STDP rule properties, we assume that w(x) equals 1 for 𝑥 ∈ [-𝑉 𝑝𝑜𝑡 ; 𝑉 𝑝𝑜𝑡 ] and 0 otherwise for a value V pot to be determined. Using the binary encoding presented in paragraph IV.A, we have a 0 (x)=1 for 𝑥 ∈ [-𝐷𝑉 𝑚 ; 𝐷𝑉 𝑚 ] and a 0 (x)=0 otherwise, where DV m is half the size of the sensitivity range. We can then compute the value of the neuron's potential. Both a 0 and w are equal to one on a segment and to 0 otherwise. When |𝑥| < |𝐷𝑉 𝑚 -𝑉 𝑝𝑜𝑡 |, the segments [-𝑉 𝑝𝑜𝑡 ; 𝑉 𝑝𝑜𝑡 ] and [-𝐷𝑉 𝑚 -𝑥; 𝐷𝑉 𝑚 -𝑥] are included one into the other and 𝐸 1 (𝑥) = 2min (𝐷𝑉 𝑚 , 𝑉 𝑝𝑜𝑡 ). When |𝑥| > |𝐷𝑉 𝑚 + 𝑉 𝑝𝑜𝑡 | the segments are disjoints and 𝐸 1 (𝑥) = 0. In between E 1 (x) follow a linear evolution.

E 1 (x) can thus be written as follow:

𝐸 1 (𝑥) = { 2min (𝐷𝑉 𝑚 , 𝑉 𝑝𝑜𝑡 )𝑖𝑓|𝑥| < |𝐷𝑉 𝑚 -𝑉 𝑝𝑜𝑡 | max(𝐷𝑉 𝑚 + 𝑉 𝑝𝑜𝑡 -|𝑥|, 0) 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

Visually, E 1 (x) has a trapezoidal shape. Noticeably for DV m =V pot , E 1 (x) become triangular with E 1 (x)=max(2DV m -|x|,0).

Knowing the neuron excitation in the one-dimensional case, it is easy to deduce the excitation for an N-dimensional input. For an input 𝑥 = (𝑥 𝑖 ) 𝑖∈⟦1;𝑁⟧ ∈ ℝ 𝑁 we have:

𝐸 𝑁 (𝑥) = ∑ 𝐸 1 (𝑥 𝑖 ) 𝑁 𝑖=1
The neuron fires when its potential reaches its threshold Th. Thus the receptive field D S of the neuron is defined by: 𝐷 𝑆 = {𝑥 ∈ ℝ 𝑁 /𝐸 𝑁 (𝑥) ≥ 𝑇ℎ}

When DV m =V pot , which is a condition that is easily reached as we will show later, D S has some nice properties. Indeed D S is bounded for Th>2DV m (N-1), and is then an L1-norm ball of diameter 2DV m N-Th. Indeed when Th>2DV m (N-1) for x in D S we have |x i |≤2DV m for all i, because otherwise E n (x)<Th. Thus D S is bounded and E(x) can be rewritten 𝐸(𝑥) = 2 * 𝐷𝑉 𝑚 * 𝑁 -‖𝑥‖ 1 and thus 𝑥 ∈ 𝐷 𝑆 ⇔ 𝑥 1 ≤ 2 * 𝐷𝑉 𝑚 * 𝑁 -𝑇ℎ. For lower values of Th, D S is not bounded, as all x such as 𝑥 1 = ⋯ = 𝑥 𝑁-1 = 0𝑎𝑛𝑑𝑥 𝑁 ∈ ℝ belong to D S , but we can show that for any 𝑘 ∈ ⟦1, 𝑁⟧, if Th>2(k-1)DV m , then for all 𝑥 ∈ 𝐷 𝑆 at least k components of x are inferior to 2DV m .

For the triangular encoding proposed in Section IV.A, the activation function has a triangular shape and can be written:

𝑎 0 (𝑥) = 𝐾 * 𝑚𝑎𝑥(1 - |𝑥| 𝐷𝑉 𝑚 , 0)
Where K is the maximum number of spikes emitted. In this case, and with the condition DV m =V pot , it can be shown, with a similar reasoning as previously, that D s is bounded for Th>KDV m (N-1), and that for Th>KDV m (N-0.5), D s is and L2-norm ball of diameter √2𝐷𝑉 𝑚 √𝑁 -𝑇ℎ 𝐾 * 𝐷𝑉 𝑚 .

Noticeably in both cases, if we denote E max the maximum possible excitation for one dimension, the bounded condition is given by: Les neurones utilisés dans les réseaux STDP peuvent avoir différents types de comportements tel que décharger de manière persistante pendant un stimulus ou seulement au début du stimulus, décharger par rebond après un stimulus inhibiteur, ou être sensibles seulement aux changements brusques dans un stimulus. On distingue deux grands types de modèles : les modèles où toute la dynamique du potentiel interne au neurone est mise en équation, ce qui peut être couteux en temps de calcul, et les modèles où les spikes sont modélisés comme des évènements discrets déclenchés par un dépassement de seuil et seule la dynamique en dessous de ce seuil est mise en équation. Le choix d'un modèle dépend du comportement que l'on souhaite reproduire, avec quel degré de réalisme et quel coût calculatoire.

Si les modèles de neurones sont variés, les lois de plasticité synaptique le sont aussi. La loi STDP la plus connue consiste en un changement persistent du poids de la synapse déclenché par l'occurrence d'un spike postsynaptique et d'un spike présynaptique. L'amplitude du changement dépend alors de l'intervalle de temps entre ces deux spikes. Les observations expérimentales montrent que le poids augmente pour un spike présynaptique précédant un spike postsynaptique et diminue pour un spike postsynaptique précédant un spike présynaptique. De plus l'amplitude du changement diminue avec la longueur de l'intervalle de temps. Cependant, d'autres types de dépendances ont été observés et utilisés dans des modèles. Les lois STDP basées sur des paires de spikes ne parviennent pas à expliquer toutes les observations. Aussi, d'autres modèles de plasticité ont été développés, prenant en compte plus de deux spikes. Il existe aussi des plasticités à court terme, pour lesquels les changements de poids ne sont pas persistants. Elles sont souvent induites par les spikes présynaptiques seulement, et se traduisent par une augmentation (facilitation) ou une diminution (dépression) des poids après plusieurs spikes présynaptiques consécutifs rapprochés. La plasticité à court terme permet une régulation du potentiel postsynaptique. Les lois STDP induisant un renforcement positif, il peut être difficile de les paramétrer pour être à la fois stables et discriminatives. Les mécanismes d'homéostasie, intervenant à des échelles de temps différentes et sur plusieurs synapses, permettent une meilleure stabilité. Ces mécanismes interviennent soit directement sur le poids des synapses, par exemple en assurant une normalisation de la somme des poids, soit sur l'excitabilité du neurone postsynaptique (on parle alors de plasticité intrinsèque), soit en modulant les paramètres de plasticité synaptique (on parle alors de méta-plasticité).

Dans un réseau, les neurones et les synapses obéissent à des règles simples mais leurs interactions sont complexes. Les études de réseaux aléatoires à poids synaptiques fixes montrent plusieurs dynamiques possibles, dont le régime asynchrone où chaque neurone décharge indépendamment selon une loi de Poisson. Ce régime ne peut être obtenu qu'avec un équilibre entre l'excitation et l'inhibition. D'autres études se focalisent sur l'évolution du poids des synapses selon différentes hypothèses de corrélation entre l'activité des neurones postsynaptiques et présynaptiques. Il a par exemple été montré que l'on peut obtenir un équilibre entre l'excitation et l'inhibition par une loi STDP symétrique, ce qui améliore les performances lors de tâches de reconnaissance de motifs.

Bien qu'il n'existe pas de méthode universelle pour entrainer un réseau STDP à effectuer une tâche de reconnaissance, il a été montré que ceux-ci ont des propriétés d'apprentissage non-supervisé. On trouve des exemples d'application à la reconnaissance de motifs visuels ou auditifs. La deuxième couche du réseau, appelée couche intermédiaire, a pour but de reconnaitre des motifs dans le train de spikes de la couche d'entrée, qui correspondent à différentes formes de signal. Les neurones intermédiaires implémentent un modèle « Leaky integrate and fire » (LIF), dont le potentiel augmente à chaque réception de spike et revient sinon à zéro selon une décroissance exponentielle. Un neurone LIF émet un spike (décharge) lorsque son potentiel atteint un seuil. Les poids des synapses provenant de la couche d'entrée sont initialisés aléatoirement. Avant apprentissage, chaque neurone reçoit donc une excitation similaire, insuffisante pour décharger. Les neurones reçoivent en plus les spikes du neurone d'attention, ce qui leur permet de décharger lorsque ce dernier décharge. Les synapses provenant de la couche d'entrée suivent une loi STDP. Leurs poids diminuent à chaque spike postsynaptique, mais augmentent si le spike postsynaptique coïncide avec un spike présynaptique. Grâce à cette loi les neurones intermédiaires deviennent sensibles à une forme spécifique du signal. La taille du champ récepteur de ces neurones, c'est-à-dire leur tolérance à la différence entre le signal en entrée et la forme apprise, peut être ajustée soit en modifiant leur seuil de décharge, soit en modifiant la largeur de la plage de sensibilité des neurones d'entrée. Afin d'éviter que plusieurs neurones déchargent simultanément et apprennent un même motif, un mécanisme dit « Winner-take-all » (WTA) est implémenté : lorsqu'un neurone décharge, les autres neurones de cette couche sont inhibés. Dans les dernières versions du réseau (LTSNet et PolyNet), une variante du WTA et de la loi STDP est implémentée, permettant à deux neurones de décharger simultanément, sans pour autant apprendre le même motif. Cela permet d'avoir des champs récepteurs se chevauchant partiellement, ce qui améliore la reconnaissance par la dernière couche du réseau. Lorsqu'un potentiel d'action est présent dans le signal, les neurones de la couche intermédiaire déchargent donc les uns à la suite des autres selon la forme du signal à différents moment du potentiel d'action.

Modèle de réseau développé dans la thèse

La dernière couche est la couche de sortie. Celle-ci doit émettre un spike pour chaque potentiel d'action dans le signal, reflétant ainsi l'activité neuronale enregistrée. Elle doit notamment être robuste Notre réseau a été comparé à deux autres logiciels de spike-sorting : Osort et Wave_clus, tous deux pouvant, comme le réseau STDP, s'utiliser de manière automatique. Chacune des méthodes testées a été exécutée sur différents jeux de données avec différentes caractéristiques, et la significativité des différences de performance a été évaluée sur chaque jeu à l'aide de tests statistiques (test de Welch). Cette dernière version a été adaptée pour traiter des signaux multi-électrodes, en dupliquant la structure initiale pour chaque électrode et en introduisant des connexions synaptiques supplémentaires au niveau de la couche de sortie. Cette structure, appelée PolyNet, permet de détecter chaque potentiel d'action au niveau de l'électrode présentant la plus forte amplitude, mais des améliorations restent à apporter pour plus de robustesse dans la classification.

Implémentations et résultats

Le réseau STDP développé a été testé sur ordinateur. Un travail préliminaire d'implémentation sur FPGA a été effectué à l'occasion d'un encadrement de stage. L'implémentation FPGA comprend pour le moment une couche d'entrée et un neurone d'attention fonctionnels. Ce travail pourra être poursuivi pour parvenir à une implémentation embarquée complète du réseau. Une implémentation sur une puce neuromorphique nécessitera plus de développements, mais on peut déjà estimer la consommation du réseau à moins d'un microwatt par électrode.

Figure I- 1 :

 1 Figure I-1: Emission and propagation of an action potential in neurons .............................................. Figure I-2: Examples of microelectrode arrays (MEA).. ......................................................................... Figure I-3: Difference between formal neurons and spiking neurons. ................................................. Figure I-4: Example of spike-timing-dependent plasticity, observed experimentally. ......................... Figure I-5: Example of neuromorphic device, including memristive synapses and Leaky-Integrate-and-Fire neurons. ......................................................................................................................................... Figure II-1: Spike-sorting principle......................................................................................................... Figure II-2: Decomposition of spike-sorting into three main steps. ...................................................... Figure II-3: Illustration of the expectation-maximization algorithm. .................................................... Figure II-4: Mean-shift algorithm illustration.. ...................................................................................... Figure II-5: Example of multiple electrodes recording. ......................................................................... Figure II-6: Two examples of bursts, during which the amplitude of the action potential decreases. Figure II-7: Example of non-stationary data during a long recording. .................................................. Figure II-8: Example of action potential overlap. .................................................................................. Figure III-1: Latency coding principle. ................................................................................................... Figure III-2: Illustration of polychrony. .................................................................................................. Figure III-3: Different neuronal response to different stimuli. .............................................................. Figure III-4: Experimentally observed STDP.. ........................................................................................ Figure III-5: Different forms of STDP rules. ........................................................................................... Figure III-6: STP observed experimentally. ............................................................................................ Figure III-7: Example of spike pattern recognition by (Masquelier et al. 2008). ................................... Figure III-8 : Example of visual pattern recognition by (Srinivasa et al. 2014).. .................................... Figure IV-1: Global structure of the STDP network. .............................................................................. Figure IV-2: Input layer implementation. .............................................................................................. Figure IV-3: Modeled and simulated EPSP for each input value. .......................................................... Figure IV-4: Effect of DV m and W min on the input value-EPSP relation, for a binary encoding. ............ Figure IV-5: Detection of an action potential by the attention neuron. ............................................... Figure IV-6: The attention neuron potential increases when an action potential is present in the input signal. ..................................................................................................................................................... Figure IV-7: STDP rule applied on the synapse connecting the input layer to the intermediate layer.Figure IV-8: Receptive field shape. ........................................................................................................ Figure IV-9: Effect of the WTA mechanism. .......................................................................................... Figure IV-10: Effect of the k-WTA mechanism on the receptive fields. ................................................ Figure IV-11: Comparison of the intermediate layer output quality for different k-WTA and different DV m , on the simulated single electrode dataset. .................................................................................. Figure IV-12: Two different implementation of the intrinsic plasticity rule. ........................................ Figure IV-13: Lateral STDP principle. ..................................................................................................... Figure IV-14: Improvement of the performance with lateral STDP implemented on the output layer of the LTSNet network. .............................................................................................................................. Figure IV-15: Spike pattern received by the output layer when using a structure with transmission delays on synapses from the intermediate layer and inhibition from the attention neuron ...............

Figure IV- 16 :

 16 Figure IV-16: Example of potential rebound of the DSSN neuron model, generating a spike after the end of the stimulus ................................................................................................................................ 78 Figure IV-17: Spiking properties of the LTS neuron after receiving a negative stimulus, with different models. .................................................................................................................................................. 80 Figure IV-18: Discrimination performance, assessed through an F-score, obtained with the LTS neuron on different conditions. ......................................................................................................................... 81 Figure IV-19: General principle of our adaptation of the network to the case of multiple electrodes 82 Figure IV-20: Two different polytrode structures for the attention neuron. ........................................ 83 Figure IV-21: Comparison of the ROC scores obtained with the two different structures for the attention neuron. .................................................................................................................................. 83 Figure IV-22: Different polytrode structures tested for the output layer. ............................................ 84 Figure IV-23: Recall of action potential from each different neural cell, on each output sublayer, on a simulated polytrode recording with four different neural cell and an SNR of 6. .................................. 85 Figure V-1: The two waveforms used in the preliminary dataset ......................................................... 89 Figure V-2: The three waveforms used in the simulated single electrode dataset .............................. 89 Figure V-3: Model of the transmembrane current through the neuron.. ............................................. 90 Figure V-4: Example of cells position relatively to the electrode line. The cells are placed along a line 20 µm away from the electrode line.. ................................................................................................... 91 Figure V-5: Example of action potential waveforms obtained for each simulated electrode and each simulated neural cell ............................................................................................................................. 92 Figure V-6: Sample of the tetrode recording d533101, with the four channels of the tetrode after filtering and the ground truth extracted from the intracellular recording. .......................................... 93 Figure V-7: Principle of the ROC curve .................................................................................................. 95 Figure VI-1: MiniNet structure ............................................................................................................ 100 Figure VI-2: Qualitative results of MiniNet on a recording from the preliminary dataset. ................. 103 Figure VI-3: Example of performance of MiniNet on one recording from the preliminary dataset. .. 104 Figure VI-4: ANNet structure ............................................................................................................... 105 Figure VI-5: Behavior of the intermediate layer of ANNet. ................................................................. 106 Figure VI-6: Examples of ANNet output and performance ................................................................. 107 Figure VI-7: ANNet performances compared to Osort and Wave_clus. ............................................. 108 Figure VI-8: ANNet performance on simulated recordings with different firing rate scenarios. ........ 109 Figure VI-9 : Two different structures adapted for tetrode recordings, for the ANNet implementation. ............................................................................................................................................................. 110 Figure VI-10 : F-scores obtained with the two tested structures, compared to the best single electrode performance. ....................................................................................................................................... 110 Figure VI-11: LTSNet structure ............................................................................................................ 111 Figure VI-12: LTSNet intermediate layer behavior .............................................................................. 114 Figure VI-13: Example of potential rebound of LTS neurons in the output layer. .............................. 115 Figure VI-14: Example of LTSNet output on a 5-s segment of simulated signal. ................................ 116 Figure VI-15: LTSNet results on the single electrode simulated dataset, compared to ANNet, Osort and Wave_clus. .......................................................................................................................................... 117 Figure VI-16: PolyNet structure for polytrode recordings (illustration for a line of 6 electrodes). .... 118 Figure VI-17: Example of output on a simulated polytrode recording with an SNR of 6. ................... 120 Figure VI-18: Action potentials stemming from different cells are recognized by a different output neurons................................................................................................................................................ 121

  experiments. Microelectrode arrays (MEA) offer a good solution to record the individual activity of numerous cells in vivo. They consist in micrometer-scale electrodes, often placed on sharp needles, arranged in an array, and implanted a few millimeters under the surface of the brain in the extracellular medium. When a neuron fires, the action potential induces transmembrane currents which modifies the potential of the extracellular medium. Thus an action potential can be recorded by an extracellular electrode placed close enough to the neuron (under about 100µm). Hence MEAs are able to record individual neural spikes. An example of widely used MEA for in vivo experiments is the Utah array (Figure I-2.a), which contains one hundred microelectrodes. MEAs are always improving, with arrays containing up to thousands of electrodes, spaced by tens of micrometers (Alivisatos

Figure I- 2 :

 2 Figure I-2: Examples of microelectrode arrays (MEA). (a) Utah array, adapted from (Hochberg et al. 2006). (b) Dense MEA, adapted from (Rios et al. 2016).

Figure I- 3 :

 3 Figure I-3: Difference between formal neurons and spiking neurons. (a) Formal neurons process real values through an activation function. (b) Spiking neurons process spikes through a dynamic integration.

Figure I- 4 :

 4 Figure I-4: Example of spike-timing-dependent plasticity, observed experimentally. Adapted from (Bi & Poo 1998)

Figure I- 5 :

 5 Figure I-5: Example of neuromorphic device, including memristive synapses and Leaky-Integrate-and-Fire neurons. Adapted from (Park et al. 2015)

Figure II- 1 :

 1 Figure II-1: Spike-sorting principle. An extracellular microelectrode records action potentials from several neural cells. These action potentials can be sorted as they have different shapes depending on which cell emits them.

Figure II- 2 :

 2 Figure II-2: Decomposition of spike-sorting into three main steps.

Figure II- 3 :

 3 Figure II-3: Illustration of the expectation-maximization algorithm. The expectation steps assign data points to a cluster. The maximization steps optimize the distribution's parameters to fit the data points' labels.

Figure II- 4 :

 4 Figure II-4: Mean-shift algorithm illustration. (a) Computation of local density maxima thanks to a window, shifted to its barycenter at each step. (b) Each point a space is assigned to the nearest local density maxima, with the method presented in (a), to form clusters.

Figure II- 5 :

 5 Figure II-5: Example of multiple electrodes recording, adapted from (Rossant et al. 2016). (a) Geometry of the electrode array. (b) Sample of recorded signal. Action potentials are highlighted with rectangles.

Figure II- 6 :

 6 Figure II-6: Two examples of bursts, during which the amplitude of the action potential decreases. Adapted from (Delescluse & Pouzat 2006)

Figure II- 7 :

 7 Figure II-7: Example of non-stationary data during a long recording, adapted from (Bar-hillel et al. 2006). The projection of detected action potentials' waveforms on two dimensions are shown at different times of the recording. The waveforms slowly evolve with time. When all action potentials are observed together (last picture), it is not possible to distinguish clusters.

Figure II- 8 :

 8 Figure II-8: Example of action potential overlap. The first two action potentials are not overlapping are clearly recognizable. The last two action potentials are overlapping and sum up into a new waveform that ca not be classified easily.

Figure III- 1 :

 1 Figure III-1: Latency coding principle. Adapted from (Thorpe & Gautrais 1997)

Figure III- 2 :

 2 Figure III-2: Illustration of polychrony. Adapted from (Izhikevich 2006). (a) Synaptic connections with different transmission delays. (b) Synchronous firing do not elicit postsynaptic spikes. (c) Example of spike pattern triggering a postsynaptic spike in neuron a. (d) Example of spike pattern triggering a postsynaptic spike in neuron.

  Figure III-3.a and b);  phasic spiking or bursting when the neuron fires or bursts only once at the beginning of the stimulation (Figure III-3.c and d);  frequency adaptation when the neuron adapts to the stimulus by decreasing its firing frequency during a maintained stimulus (Figure III-3.e);  accommodation when the neuron is not sensitive to slowly increasing stimulus but only sharp ramps (Figure III-3.f);  rebound spike or burst when the neuron can fire or burst after an inhibitory stimulus (Figure III-3.g and h);  threshold variability when the neuron's threshold can vary depending on its previous activity (Figure III-3.i).

Figure III- 3 :

 3 Figure III-3: Different neuronal response to different stimuli. Adapted from (Izhikevich 2004)

  ; D'amour & Froemke 2015; Abbott & Nelson 2000; Srinivasa et al. 2014) (see Figure III-5).

Figure III- 4 :

 4 Figure III-4: Experimentally observed STDP. Adapted from (Bi & Poo 1998). The weight's change is positive for a presynaptic spike preceding a postsynaptic spike and negative otherwise.

Figure III- 5 :

 5 Figure III-5: Different forms of STDP rules, observed experimentally (a,b,c), or used in models (d,e,f,g). Adapted from (Vogels et al. 2013)

  Figure III-6.a), or a facilitation when each presynaptic spike temporarily increases the synapse's weight (Figure III-6.b

Figure

  Figure III-6: STP observed experimentally. Adapted from (Markram et al. 1998). (a) Example of short term depression. (b) Example of short term facilitation. Bottom traces show the presynaptic spikes, top traces show the postsynaptic potential.

Figure III- 7 :

 7 Figure III-7: Example of spike pattern recognition by (Masquelier et al. 2008). One LIF neuron is trained to learn an arbitrary spike pattern through STDP. (a) Illustration of the pattern to learn (in red). Between patterns (in blue), the input activity consist in random Poisson spike trains. (b) Illustration of the neuron learning. Top: before learning the neuron fire independently of the pattern occurrences. Bottom: After learning, the neuron fire at the beginning of the pattern (pattern occurrence are shown in grey).

Figure III- 8 :

 8 Figure III-8 : Example of visual pattern recognition by (Srinivasa et al. 2014). (a) Structure of the network. (b) Patterns to learn. (c) Weights learnt by the reservoir excitatory neurons (Layer 2).

  . Each layer is designed to achieve a specific sub-processing task. The neuron model and the synaptic plasticity are both adjusted for each layer to obtain the desired behavior. The final version of the network consists in three main layers: an encoding layer that encodes the input signal into spikes, an intermediate layer that learns intermediate patterns, and an output layer that finalizes the process and fires once for each action potential in the signal. The network also features an attention mechanism. Each part of the network can be implemented with different solutions, and some mechanisms can be added to improve its performance. Through this PhD work three different versions of a complete network were implemented for processing single electrodes: MiniNet, ANNet and LTSNet, described in Section VI. The last version, LTSNet was also adapted to multi-electrode processing, in a version called PolyNet. Each of these versions uses different implementation solutions for each element of the network, some because they bring an obvious improvement, others because they combine well with other choices.

Figure IV- 1 :

 1 Figure IV-1: Global structure of the STDP network. The network is constituted of several layers, each having a different functional purpose, connected in a feedforward manner. An attention mechanism is implemented through an attention neuron. The network takes as input an electrode signal and outputs a spike train corresponding to the sorted action potentials.

Figure IV- 2 :

 2 Figure IV-2: Input layer implementation. (a) Sensitivity range for each input neuron. The sensitivity ranges overlap and cover all possible signal values. (b) Activation of each encoding neuron for a given signal value. Activated neurons are represented in red. (c) Activation of one encoding neuron thorough time. The neuron fires at regular time steps when the signal is within its sensitivity range. (d) Different activation functions: binary (left) and triangular (right). With the binary activation, encoding neurons fire one spike at each time step if the signal value is within their sensitivity range. With the triangular activation, encoding neurons can fire several spikes depending on the position of the signal value within their sensitivity range. (e) Several signal values, corresponding to different delays, are encoded at each time step.

  Thanks to this relation, we can choose a minimum weight W min , obtained for a neuron firing at each sampling step and adjust f d in consequence. Knowing these equilibrium weights, we can then compute the excitatory postsynaptic potential (EPSP) generated by each possible input signal value, which is the sum of the weighted spikes received by the postsynaptic neuron. We computed this approximation numerically and compared it to simulation results, as shown in FigureIV-3. We also observed the effects on the EPSP of the choices of both DV m and W min values (FigureIV-4). Input values close to zero generate a weak excitation whereas when the input value moves away from zero the postsynaptic excitation increases, which is the desired effect.

Figure IV- 3 :

 3 Figure IV-3: Modeled and simulated EPSP for each input value. Model is shown in red, values obtained in simulation, with a normal Gaussian noise as input signal, are shown in blue. This was obtained for DV m =1.75 and W min =0.13 and with a binary encoding

Figure IV- 5 :

 5 Figure IV-5: Detection of an action potential by the attention neuron. Top: input signal. Bottom: attention neuron's spike train.

Figure IV- 6 :

 6 Figure IV-6: The attention neuron potential increases when an action potential is present in the input signal. Top: input signal. Middle: EPSP generated by the synapses implementing an STP rule. Bottom: attention neuron's potential. In red are represented the attention neuron threshold, the self-excitation effect (red arrow), and the time intervals when the neuron is firing

  Figure IV-7.a). The coincidence time window was chosen to match the time resolution of the input spike train, which is about 0.05ms (see previous section). Two versions of this STDP rule were tested: an additive version and a multiplicative version, which both have simple properties (Figure IV-7.b).

Figure IV- 7 :

 7 Figure IV-7: STDP rule applied on the synapse connecting the input layer to the intermediate layer. (a) STDP rule. ∆t is the interval between the presynaptic spike and the postsynaptic spike, ∆w is the weight change. (b) Example of learnt weights for an additive STDP rule (left) and a multiplicative STDP rule (right). The two pictures represent the weight of the synapses connecting all input neurons to one particular intermediate neuron. Black is for a weight equal to zero and white for a weight equal to one.

  For the triangular encoding, when Th>KDV m (N-0.5) the receptive field is an L2-norm ball of diameter √2𝐷𝑉 𝑚 √𝑁 -𝑇ℎ 𝐾 * 𝐷𝑉 𝑚 . Figure IV-8.a illustrates the different possible shapes taken by the receptive field. These properties of the intermediate neuron receptive field have been confirmed by simulations (Figure IV-8.b). These simulation results, obtained for a simple case where the input values are lowdimensional and follows a Gaussian distribution, validate our theoretical results. They thus constitutes a first approximation of the real receptive field shape and allows us to choose the size of the receptive field to cover most of the occurrences of a learnt pattern, by adjusting either the size of the input neuron's sensitivity range or the intermediate neuron's threshold.

Figure IV- 8 :

 8 Figure IV-8: Receptive field shape. (a) Theoretical receptive fields obtained in a 3-dimentional case. The two at the top are obtained using the binary encoding, the two at the bottom for the triangular encoding. The two on the left are obtained on a bounded condition, and the two on the right for an unbounded condition. (b) Receptive fields obtained on simulation with Gaussian noise as input, in a 2-dimentional case. The two at the top are obtained using the binary encoding, the two at the bottom for the triangular encoding. The two on the left are obtained on a bounded condition, and the two on the right for an unbounded condition. The blues lines shows theoretical size of the receptive field, not taking into account the unbounded parts.

  05ms (Figure IV-9.a). The WTA mechanism ensures that the different neuron's receptive fields cover the possible values taken by the input without overlapping (Figure IV-9.b).

Figure IV- 9 :

 9 Figure IV-9: Effect of the WTA mechanism. (a) Intermediate neurons fire one after the other and not simultaneously. Top: input signal. Bottom: intermediate layer spike train, where the different colors stands for different neurons. (b) Receptive field of the intermediate neuron on data containing action potentials. The encoded input values have been represented in two dimension through a PCA. Each point represents an intermediate neuron firing. Different colors stands for different neurons (c) Learnt weights. Each square represent the weight of synapses projecting from all input neurons to one intermediate neuron. Black stands for 0 and white for 1.

Figure IV- 10 :

 10 Figure IV-10: Effect of the k-WTA mechanism on the receptive fields. (a) Receptive field obtained on simulation for a twodimensional case. Different colors are for different neurons (b) Outline of the receptive fields shown in (a). (c) Measure of the common surface between two different receptive fields. Top: results for 1-WTA. Bottom: results for 2-WTA. The 2-WTA implementation shows an overlap between receptive fields.

Figure IV- 11 :

 11 Figure IV-11: Comparison of the intermediate layer output quality for different k-WTA and different DV m , on the simulated single electrode dataset. On each graph a comparison is made between two methods. Each point represents the two different results for one recording. A point below the diagonal means the method plotted vertical is better the one plotted in horizontal. (a) Comparison of the entropy index between the 1-WTA and 2-WTA implementations, for different values of DV m . (b) Comparison of the entropy index between different values of DV m , for the same k-WTA implementation. (c) Comparison of the distance index between the 1-WTA and 2-WTA implementations, for different values of DV m . (d) Comparison of the distance index between different values of DV m , for the same k-WTA implementation.

Figure IV- 12 :

 12 Figure IV-12: Two different implementation of the intrinsic plasticity rule. (a) Input spikes arriving before an output spike decrease the threshold whereas input spikes arriving after an output spike increase the threshold. (b) Each output spike decreases the threshold whereas input spikes coinciding with an output spike increase the threshold.

Figure IV- 13 :

 13 Figure IV-13: Lateral STDP principle. (a) Elements involved in the lateral STDP: presynaptic spikes, postsynaptic spikes and inhibitions received by the postsynaptic neuron. (b) Weight changes induced by the lateral STDP. Left: classical STDP, postsynaptic spikes alone depress the synapse, pre and postsynaptic spike coincidences potentiate the synapse. Right: lateral STDP rule, presynaptic spikes alone potentiate the synapse, presynaptic spike and inhibition coincidences depress the synapse.

Figure IV- 14 :

 14 Figure IV-14: Improvement of the performance with lateral STDP implemented on the output layer of the LTSNet network. Each point represents the performance with and without lateral STDP for one recording from the simulated single electrode dataset.

Figure IV- 15 :

 15 Figure IV-15: Spike pattern received by the output layer when using a structure with transmission delays on synapses from the intermediate layer and inhibition from the attention neuron

Figure IV- 16 :

 16 Figure IV-16: Example of potential rebound of the DSSN neuron model, generating a spike after the end of the stimulus Our idea to use a neuron model that has a rebound property was initiated during a collaboration with Timothée Lévi and Takashi Kohno from the University of Tokyo. Following a 2-week stay at their facility, focused on FPGA implementation (see Section VI.F), we tested if the properties of their neuron model, called DSSN (Nanami & Kohno 2016), fit our problem. After adjusting the membrane time constant to our needs, we found that, after an inhibitory stimulus, the latency of the first spike emitted by the neuron is a decreasing function of the stimulus integral and that the number of spikes emitted is roughly proportional to the stimulus integral (Figure IV-17.a). Assuming that the total stimulus reflects how well the presented pattern matches the learnt pattern, this latency property is very interesting for our problem. After adjusting the model's parameters, we managed to obtain only one spike for a large range of stimulus, which was the desired effect (Figure IV-17.b).

Figure IV- 17 :

 17 Figure IV-17: Spiking properties of the LTS neuron after receiving a negative stimulus, with different models. (a) Original model from (Nanami & Kohno 2016). (b) Same model with modified parameters. (c) Simplified LTS model used in our network. Left: number of spikes emitted depending on the stimulus amplitude and duration. Middle: number of spikes emitted depending on the total stimulus (amplitude*duration). Different colors represent different stimulus amplitudes. Right: time of the first spike after the end of the stimulus, depending on the total stimulus (amplitude*duration). Different colors represent different stimulus amplitudes.

  . We also performed tests where the threshold was the sum of a fixed threshold of 5 and a variable threshold subject to IP. The parameters are set so that the variable threshold stabilizes to one quarter of the size of the learnt stimulus. With the IP, the output layer is even capable of distinguishing two patterns completely included one into the other (Figure IV-18.b). In our final single-electrode network implementation (LTSNet), we chose not to use a threshold adaptation, as it brings no obvious performance improvement for spike-sorting.

Figure IV- 18 :

 18 Figure IV-18: Discrimination performance, assessed through an F-score, obtained with the LTS neuron on different conditions. The testing dataset contains two different patterns. Between each testing dataset, the size of one of the two patterns varies, the size of the common parts between the two patterns varies, and the maximum variation between occurrences of the same pattern varies. (a) Results obtained without IP, (b) Results obtained with IP.

Figure IV- 19 :

 19 Figure IV-19: General principle of our adaptation of the network to the case of multiple electrodes

Figure IV- 20 :

 20 Figure IV-20: Two different polytrode structures for the attention neuron. (a) Baseline purely parallel structure. (b) Structure with lateral connections.

Figure IV- 21 :

 21 Figure IV-21: Comparison of the ROC scores obtained with the two different structures for the attention neuron. Each point correspond to a recording from the simulated polytrode dataset.

  It is thus necessary to extend the WTA mechanism to neighboring output sublayers. Overall, we tested different structures for the output layer, combining these two aspects. First, each output sublayer is connected to either one, three or five intermediate sublayers (see Figure IV-22.a), with a synaptic weight higher for connections corresponding to electrodes closer to each other. Second, the WTA mechanism is either unchanged or included lateral inhibition between three or five output layers. (see Figure IV-22.b).

Figure IV- 22 :

 22 Figure IV-22: Different polytrode structures tested for the output layer. (a) Different possible structures for the connection between the intermediate and the output layer. (b) Different possible structures for the WTA mechanism. The two structural aspects were tested with all possible combinations.

  Figure IV-23.a), which lowers the final score. When using lateral connections, this effect is reduced as each different action potential is detected mainly by one output sublayer, but the recall is overall lower (Figure IV-23.b). In our final version, we chose to use the three to one intermediate to output connections in combination with a five sublayer WTA mechanism, which is the structure that give the best results apart from the one with no lateral connections. Future work should focus on improving the structure to get a better recall.

Figure IV- 23 :

 23 Figure IV-23: Recall of action potential from each different neural cell, on each output sublayer, on a simulated polytrode recording with four different neural cell and an SNR of 6. (a) Results obtained with the purely parallel structure. (b) Results obtained with a structure with a WTA connecting 5 output sublayers, and each output sublayer connected to three intermediate sublayers.

Figure V- 1 :

 1 Figure V-1: The two waveforms used in the preliminary dataset

Figure V- 2 :

 2 Figure V-2: The three waveforms used in the simulated single electrode dataset

  our model the transmembrane currents follow a template waveform (Figure V-3.a) that propagates along one segment line, representing the neuron arborization. The waveform's amplitude decreases linearly along this line until being null. We also ensure that at any time the sum of the currents is null (Kirschoff's law), by subtracting the total current sum to the local currents. The final current waveforms obtained along the propagation segment are shown in Figure V-3.b. The potential recorded by an electrode is modeled as the potential at one point of the extracellular space, which is the sum of the local currents for all simulated neurons divided by the distances, to which we add some noise.

Figure V- 3 :

 3 Figure V-3: Model of the transmembrane current through the neuron. (a) Template waveform used to model the transmembrane current. (b) Transmembrane currents obtained at different points of the line representing the neuron, after applying a propagation, an amplitude decrease, and adjusting the currents to have a null sum.

Figure V- 4 :

 4 Figure V-4: Example of cells position relatively to the electrode line. The cells are placed along a line 20 µm away from the electrode line. (a) x-z view. (b) x-y view. Units are in meters. Red circles represents the electrodes, black circle represents the cells soma and black lines the propagation line.

Figure V- 5

 5 Figure V-5 shows an example of action potentials' waveforms generated with this method. Once the waveforms are computed, they are included in the recording following a Poisson process, with a firing rate of 10 Hz for each simulated cell. For each different number of cells and each different SNR, we generated 5 different recordings, each time with randomly placed neural cells, leading to a total of 60 simulated recordings, each of 100 s duration.

Figure V- 5 :

 5 Figure V-5: Example of action potential waveforms obtained for each simulated electrode and each simulated neural cell

Figure V- 6 :

 6 Figure V-6: Sample of the tetrode recording d533101, with the four channels of the tetrode after filtering and the ground truth extracted from the intracellular recording.

Figure V- 7 :

 7 Figure V-7: Principle of the ROC curve

Figure

  Figure VI-1: MiniNet structure

  Figure VI-2 shows the behavior of the network for one recording example. Figure VI-2.a shows the behavior of the intermediate layer. Thanks to the STP, intermediate neurons fire only when an action potential is present in the signal. During an action potential, different intermediate neurons fire in sequences, as they each recognize a different part of the action potential. Indeed, as shown in Figure VI-2.b, for each active intermediate neuron, the synapses stemming from the input layer have evolved to match a specific waveform, which the neuron is then able to recognize.Figure VI-2.c shows an example of the network's output. It can be seen that the output neuron 3 has learnt to recognize one action potential waveform, whereas the output neuron 1 has learnt to recognize the other one.

Figure VI- 2 :

 2 Figure VI-2: Qualitative results of MiniNet on a recording from the preliminary dataset. (a) Example of intermediate layer spike train during an action potential. (b) Weights learnt by the intermediate layer. Each square represents the weights of the synapses from all input neurons, organized into a grid, to one specific intermediate neuron. The weights go from 0 (black) to 1 (white). (c) Example of output layer spike train. Output spikes match the different action potential occurrences.

Figure VI- 3 :

 3 Figure VI-3: Example of performance of MiniNet on one recording from the preliminary dataset. (a) Evolution of the Fscores during the 200-s simulation. The two graphs are for the two ground truth cells and the different colors correspond to two different output neurons. (b) Mean F-scores on the last 100s of the same recording, for each pair of ground truth cell and output neuron.

  the MiniNet network with the introduction of an attention neuron, which gave its name to this new version. The attention neuron allows to isolate the network's detection function, so that the STP does not impair the intermediate layer's recognition function. The attention neuron then projects to the intermediate layer through fixed-weight synapses (see Figure, bringing an additional excitation to the intermediate layer, necessary for it to fire. Another important improvement was to set a shorter encoding step on the input layer, while keeping the same interval between two encoded signal values. This brings robustness against sampling time jitter without adding unnecessary synapses. The number of delays in the input layer was also reduced to 10, corresponding to a time window of 0.5 ms. This is more suitable for realistic action potentials, as the encoding time window should be slightly shorter than an action potential's duration. The structure of the output layer was also more complex. It is constituted of LIF neurons with IP, and it receives spikes from the intermediate layer trough synapses with different synaptic delays, presynaptically inhibited by the attention neuron, as described in Section IV.D.3. The synaptic weights can take positive as well as negative values. This structure forces the output layer to wait for the end of the spike sequence emitted by the intermediate layer before firing. This prevents the output layer from missing any important information for pattern separation. The delays also bring some information about the sequence timing. The characteristics of ANNet are summarized in TableVI-3 and Figure VI-4. The detailed implementation of ANNet and the results obtained can be found in an article (annex B), currently in review for publication.

Figure

  Figure VI-4: ANNet structure

  2). The behavior of the intermediate layer is similar to the one obtained with the first implementation. Thanks to the attention neuron, intermediate neurons fire only when an action potential is present in the signal. The spike sequence generated by the intermediate layer matches for similar action potentials and does not for different action potentials (Figure VI-5.a). Indeed, each active intermediate neuron learns to recognize a specific shape in the signal (Figure VI-5.b and c). The intermediate layer spike sequence can thus be seen as a signature of the action potential occurring in the input signal.

Figure VI- 5 :

 5 Figure VI-5: Behavior of the intermediate layer of ANNet. (a) Spike sequences emitted by the intermediate layer for each action potential occurrence. Top: signal shapes for all action potential occurrences on the last 100 s of simulation. Middle row: spike trains of the intermediate layer synchronized with 50 different action potential occurrences. Bottom row: distribution of intermediate spike latency relative to each action potential occurrence (the histograms are cumulated). The different colors stand for different intermediate neurons. (b) Weight evolution of the synapses projecting on one specific intermediate neuron. Each square represents the weights of the synapses from all input neurons at a specific time. The weights go from 0 (black) to 1 (white). (c) Weights learnt by each intermediate neuron after a 200-s simulation. Each square represents the weights of the synapses from all input neurons, organized into a grid, to one specific intermediate neuron. The weights go from 0 (black) to 1 (white).

Figure VI- 6 :

 6 Figure VI-6: Examples of ANNet output and performance (a) Output and scored on a simulated recording. Left: comparison of the output spike train, composed of ten neurons, with the truth spike train, composed of three neurons, on a 10-s segment of input signal. Matching spike trains have been highlighted in red. Right: evolution of the performance of the network over the 200-s simulation. (b) Same as (a) for a real recording (dataset d553101). Here the truth is only known for one cell thus only the performance relative to this cell can be computed.

  Figure VI-7.a). The performances on real recordings confirmed these results, except that Wave_clus obtained quite low scores on data with high SNR (Figure VI-7.b). An insight on the detailed scores for the simulated recordings with an SNR of 4.5, where ANNet perform significantly better than the two other methods, shows that the STDP network has a better recall and a better clustering score (Figure VI-7.c).

Figure VI- 7 :

 7 Figure VI-7: ANNet performances compared to Osort and Wave_clus. (a) Performance on the single electrode simulated dataset. (b) Performance on real recordings. Error bars show the standard deviation. ANNet performance was compared to Wave_clus and Osort through statistical tests (see Section V.C). ns stands for non-significant, * for p<0.05, ** for p<0.01, *** for p<0.001. (c) Detailed mean scores on the 10 simulated recordings with an SNR of 4.5.

  Figure VI-8.a). In the second, each cell became silent one after the other during a 50 s time interval (Figure VI-8.b). The results show qualitatively that the network is robust to such variations.

Figure VI- 8 :

 8 Figure VI-8: ANNet performance on simulated recordings with different firing rate scenarios.

  Figure VI-9.a). In a second architecture, the input layer, the attention neuron and the intermediate layer were all duplicated and connected to a common output layer (Figure VI-9.b). We compared the performances of these two methods to the performance obtained on the best single channel for each tetrode, with the single-electrode version of the network (Figure VI-10).

Figure VI- 9 :

 9 Figure VI-9 : Two different structures adapted for tetrode recordings, for the ANNet implementation.

Figure

  Figure VI-10 : F-scores obtained with the two tested structures, compared to the best single electrode performance.

Figure

  Figure VI-11: LTSNet structure

  on our single electrode simulated dataset (see Section V.A.1). The behavior of the intermediate layer is qualitatively similar to the previous version, except that two intermediate neurons fire each time simultaneously, due to the 2-WTA mechanism (Figure VI-12). Concerning the output layer, thanks to their rebound properties, LTS neurons fire after the inhibitory stimulus received from the intermediate layer. The neuron receiving the strongest stimulus fires first (Figure VI-13).

Figure VI- 12 :

 12 Figure VI-12: LTSNet intermediate layer behavior. (a) Spikes emitted by the intermediate layer in response to action potential in the signal. The signal is shown in black. Spikes are shows by the colored vertical bars, different colors standing for different intermediate neurons. (b) Weights learnt by each intermediate neuron. Each square represents the weights of the synapses from all input neurons, organized into a grid, to one specific intermediate neuron. The weights go from 0 (black) to 1 (white).

Figure VI- 13 :

 13 Figure VI-13: Example of potential rebound of LTS neurons in the output layer. Top: Spike train generated by the intermediate layer. Different colors stands for different intermediate neurons. Middle: Potential evolution of the different output neurons in different colors. Bottom: output neurons spikes. Colors match the middle panel.

Figure

  Figure VI-14 shows an example of LTSNet output on a 5-s sample of a simulated recording. Over the 15 output neurons, three neurons are active, and reproduce the true activity. LTSNet performance was assessed on the single electrode simulated dataset and compare to ANNet, as well as Osort (Rutishauser et al. 2006) and Wave_clus (Rutishauser et al. 2006). LTSNet has similar scores as ANNet and Wave_clus, and perform significantly better than Osort on recordings with a low SNR (Figure VI-15.a). Looking at the detailed score on recordings with an SNR of 4.5, it can be seen that LTSNet has a better recall than ANNet, but the clustering performance is degraded (Figure VI-15.b).

Figure VI- 14 :

 14 Figure VI-14: Example of LTSNet output on a 5-s segment of simulated signal. Top: input signal Middle: spike train corresponding to true action potentials in the signal. Bottom: output spike train, composed of 15 neurons. Spike trains matching the truth have been highlighted in red.

Figure VI- 15 :

 15 Figure VI-15: LTSNet results on the single electrode simulated dataset, compared to ANNet, Osort and Wave_clus. (a) Mean F-scores on recordings with different SNR. Error bars show the standard deviation. A Welch test with a Bonferroni correction was used to compare LTSNet to each of the three other methods shown. ns stands for non-significant, * for p<0.05, ** for p<0.01, *** for p<0.001. (b) Detailed mean scores on the 10 simulated recordings with an SNR of 4.5

Figure VI- 16 :

 16 Figure VI-16: PolyNet structure for polytrode recordings (illustration for a line of 6 electrodes).

Figure VI- 17 :

 17 Figure VI-17: Example of output on a simulated polytrode recording with an SNR of 6. Truth is shown at the top. Each panel shows the recorded signal for each electrode (black line), and the spikes from the corresponding output sublayer (colored bars). Different colors correspond to different neurons within the same sublayer.

Figure VI- 18 :

 18 Figure VI-18: Action potentials stemming from different cells are recognized by a different output neurons. (a) Evolution of the paired F-scores during the simulation. Different colors stand for different output neurons within the same output sublayer. (b) Number of action potentials detected by each output neurons and number of false positives and false negatives, within the last 50 seconds of the simulation.

Figure VI- 19 :

 19 Figure VI-19: Mean scores on the polytrode simulated recordings with different SNR, for each type of error.

  implementing neural network on FPGA[START_REF] Li | An FPGA-Based Silicon Neuronal Network with Selectable Excitability Silicon Neurons[END_REF]. I stayed for 2 weeks in Tokyo for this purpose and the work started there was continued by Joel Fournier during a 4-month internship under my supervision. This collaboration and this internship resulted in the implementation of an input layer, a short term-plasticity and an attention neuron, which were tested all together. The implementation was done on a Kintex-7 device, on a Genesys 2 development board from Digilent (Figure. The results were qualitatively good, as this attention neuron was able to detect, in real time, action potentials in a testing signal generated on the fly (Figure. The parameters still need to be adjusted to optimize the detection and to match the characteristics of a realistic signal. This work should be continued with Takashi Kohno and Timothée Levi to add an STDP rule and LTS neurons to obtain a complete network. During his internship, Joel Fournier also implemented a protocol for receiving data from a computer via a USB-interface included in the development card we used. This interface will be very useful in the future to send electrode recordings in real-time to the FPGA for a real time spike-sorting.

Figure VI- 20 :

 20 Figure VI-20: Parts of the network implemented on FPGA. The parts that were not implemented are grayed.

Figure

  Figure VI-21: Genesys 2 board used for FPGA implementation

  𝑇ℎ > 𝐸 𝑚𝑎𝑥 * (𝑁 -1)IntroductionEnregistrer le cerveau est important à la fois pour en étudier le fonctionnement, mais également pour des applications telles que les interfaces cerveau-machine. Les microélectrodes extracellulaires permettent l'enregistrement de l'activité individuelle des cellules neurales. Avec les évolutions techniques, les matrices d'électrodes comportent de plus en plus de sites, permettant d'enregistrer de nombreux neurones simultanément. Cela ouvre beaucoup d'opportunités mais nécessite des algorithmes adaptés, notamment pour les interfaces cerveau machine qui nécessitent un traitement en temps réel des données. En particulier, pour les enregistrements extracellulaires, il est préférable de trier les potentiels d'action émis par différents neurones et enregistrés sur une même électrode, une opération appelée spike-sorting. De nombreuses méthodes existent, mais peu permettent un traitement en temps réel des données. Ce travail de thèse est focalisé sur le développement d'une méthode radicalement nouvelle, utilisant un réseau de neurones artificiel « spike-timing-dependent plasticity » (STDP). Ce type de réseau, encore peu utilisé pour des tâches de reconnaissance, a des propriétés d'apprentissage non-supervisé intéressantes pour le spike-sorting. Un tel algorithme pourrait être implémenté dans des puces neuromorphiques très basse consommation, qui connaissent aujourd'hui d'importants développements.Etat de l'art des méthodes de spike-sortingUne microélectrode extracellulaire enregistre les potentiels d'action émis par des neurones proches. La forme de ces potentiels d'action diffère selon la position du neurone qui l'émet, ce qui permet de les trier par une méthode de spike-sorting.La plupart des méthodes se décomposent en trois étapes principales : la détection des potentiels d'action, l'extraction de traits caractéristiques de leur forme, puis leur classification en groupes qui correspondent alors aux différents neurones enregistrés. La détection se fait par un seuillage, soit directement sur le signal, soit après un prétraitement basé par exemple sur un calcul d'énergie, l'application d'ondelettes ou l'utilisation d'un gabarit de potentiel d'action. La forme du potentiel d'action détecté peut alors être directement utilisée pour la classification, mais la plupart des méthodes en extraient quelques caractéristiques, afin de réduire le nombre de dimensions à traiter. Ces caractéristiques peuvent être prédéfinies, telles que l'amplitude ou la largeur du potentiel d'action ou des coefficients d'ondelette, ou bien déterminées automatiquement par un algorithme de réduction de dimension tel que l'analyse en composantes principales (ACP). La dernière étape consiste à classer les vecteurs de caractéristiques obtenus dans différents groupes. Parmi les algorithmes de regroupement utilisés on peut citer l' « expectation-maximization », le « K-mean », le « mean-shift », le « superparamagnetic clustering ». Une étude a notamment utilisé un réseau STDP monocouche pour cette étape de classification. D'autres méthodes utilisent une approche plus globale, en modélisant le signal enregistré comme la somme de potentiel d'actions. C'est le cas des méthodes dites de « template matching ». Différents gabarits de potentiels d'action sont déterminés lors d'une étape de prétraitement semblable aux

  Le réseau STDP qui a été développé est un réseau « feedforward », constitué de trois couches de neurones, ayant chacune un rôle spécifique, ainsi que d'un mécanisme d'attention. Cette structure, construite pour traiter un signal mono-électrode, a ensuite été adaptée pour traiter un signal provenant d'électrodes multiples.La première couche, dite couche d'entrée, a pour but de transformer le signal d'entrée en train de spikes. Chaque neurone de cette couche agit comme un neurone sensoriel sensible à une certaine plage de valeur. Il décharge à intervalles réguliers lorsque le signal est dans sa plage de sensibilité. Les neurones d'entrée ont différentes plages de sensibilité et prennent en compte la valeur du signal avec différents délais, ce qui permet d'encoder à chaque instant la forme du signal dans une fenêtre de temps de 0,5 ms.Les potentiels d'action ne sont présents que de manière ponctuelle dans le signal. Un mécanisme d'attention a été implémenté afin que le réseau ne traite que les parties du signal comportant un potentiel d'action. Ce mécanisme d'attention est implémenté grâce à une plasticité à court terme, qui permet de diminuer le poids des neurones d'entrée déchargeant fréquemment, c'est-à-dire ceux codant pour des valeurs proches de zéros. Dans la première version du réseau (MiniNet) cette plasticité a été implémentée sur les synapses reliant la couche d'entrée à la couche intermédiaire. Par la suite, un neurone d'attention a été implémenté pour isoler cette fonction de détection. Celui-ci reçoit les spikes émis par la couche d'entrée via des synapses implémentant une plasticité à court terme et émets une série de spikes pendant la durée d'un potentiel d'action présent dans le signal.

  Différentes versions du réseau STDP ont été implémentées suivant les modèles décrits au paragraphe D, chacune présentant des améliorations. La première version implémentée, MiniNet, est aussi la plus simple. Il n'y a pas de neurone d'attention, la plasticité à court terme étant implémentée sur les synapses reliant les couches d'entrée et intermédiaire. Ce réseau donne déjà de bons résultats sur des données simulées simples. Des tests additionnels sur la couche de sortie montrent que la STDP latérale améliore les résultats et que la plasticité intrinsèque permet de mieux discriminer des potentiels d'action de durées différentes.Le réseau a ensuite été amélioré par l'augmentation de la fréquence d'encodage par la couche d'entrée et l'introduction d'un neurone d'attention. Cette version, nommée ANNet, comprend également une couche de sortie inhibée par le neurone d'attention, recevant les spikes de la couche intermédiaire par plusieurs délais synaptiques et implémentant une plasticité intrinsèque. Ce réseau a été comparé à Osort et Wave_clus sur des jeux de données réelles et simulées et donne de meilleurs résultats sur les signaux avec un rapport signal-sur-bruit inférieur à 4, typique des enregistrements réels. Des tests préliminaires ont également été menés sur les données tétrodes, avec une structure de réseau en entonnoir, montrant l'intérêt de croiser les informations provenant de plusieurs électrodes voisines.Enfin la dernière version mono-électrode du réseau, LTSNet, a pour principale amélioration l'introduction d'un modèle de neurone LTS sur la couche de sortie, qui permet de s'affranchir de la structure complexe d'ANNet sur cette couche (délais synaptiques, inhibition par le neurone d'attention, et plasticité intrinsèque). Ce réseau implémente également un 2-WTA sur la couche intermédiaire et une STDP latérale sur la couche de sortie. Les performances obtenues sont similaires avec celles d'ANNet, avec une structure plus simple.

  Table II-1: List of existing spike-sorting software. ................................................................................. Table IV-1: Parameters used for the simplified LTS model ................................................................... TableIV-2: Mean F-scores obtained with the different structures tested on the simulated polytrode dataset ................................................................................................................................................... Table V-1: Parameters used to generate the three waveforms of the simulated single electrode dataset ............................................................................................................................................................... Table V-2: Noise level used and corresponding SNR for the simulated single electrode dataset ........ Table V-3: Amplitude of the each neural cell's action potential for the different SNR ........................ Table VI-1: Main features of MiniNet .................................................................................................. Table VI-2: Detailed parameters of MiniNet ....................................................................................... Table VI-3: Main features of ANNet ....................................................................................................

	LIST OF ABBREVIATIONS
	ANN	Artificial Neural Network
	BCI	Brain Computer Interface
	FPGA	Field-Programmable Gate-Array
	IP	Intrinsic Plasticity
	LIF	Leaky-Integrate-and-Fire
	LTS	Low Threshold Spiking
	MEA	Micro-Electrode Array
	ROC	Receiver Operating Characteristic
	SNN	Spiking Neural Network
	SNR	Signal-to-Noise Ratio
	STDP	Spike-Timing-Dependent Plasticity
	STP	Short-Term Plasticity

Table VI-4: Main features of LTSNet ................................................................................................... Table VI-5: Detailed parameters of LTSNet ......................................................................................... Table VI-6: Resources used for MiniNet .............................................................................................. Table VI-7: Resources used for ANNet ................................................................................................ Table VI-8: Resources used for LTSNet ................................................................................................

Table II -1: List of existing spike-sorting software. Adapted from http://simonster.github.io/SpikeSortingSoftware/

 II

	OSort	MATLAB	local	energy	N/A	template matching	Yes	No	No	(Rutishauser et al.
			threshold	with							2006)
	Software	Language	Detection alignment		Feature Extraction	Clustering		Drift Overlap Large-	Publications	Comments
	pebble	C++,	?		PCA	ISO-SPLIT		No	No	scale MEA No	(Magland & Barnett
		MATLAB									2016)
	BinaryPursuitSpik phy (previously	MATLAB Python	binary pursuit raw signal threshold	N/A PCA	binary pursuit EM GMM, masked EM	No No	Yes No	No Yes	(Pillow et al. 2013) (Kadir et al. 2014;	GUI for inspecting
	eSorting klustakwik)		with alignment		GMM					Rossant et al. 2016)	sorting results. Large,
	bpsort	MATLAB	binary pursuit, raw	PCA (initialization)	t-distribution	MM	Yes	Yes	Yes		active community
	spikesort	MATLAB	signal raw signal threshold threshold	PCA, factor analysis,	(initialization) 1D GMM, 1D k-means No	No	No		Claims	"99.5%
			with (height + width) alignment	sparse PCA, t-SNE							accuracy."	Only
			(initialization)								supports clustering into
	CBPSpikesortDem	MATLAB	continuous	basis	continuous basis pursuit	continuous	basis	No	Yes	Yes?	(Ekanadham et al.	two units.
	o SpikeSorter.jl	Julia	pursuit hidden	markov	spike width, trough to	pursuit hidden markov model No	No	No	2014)	Algorithm	described
	ClusterLizard	C++	raw signal threshold wavelets + Lillifors test model valley ratio	Euclidean distance	Yes	No	No	(Knieling et al. 2016)	in Spike sorting with
	Combinato SpikeSorting.jl EToS spyke FMMSpikeSorter SpyKING Circus gpu_python KFMM trisdesclous KiloSort Wave_clus	Python Julia C++ Python MATLAB Python Python MATLAB Python MATLAB, CUDA C MATLAB	raw signal threshold with alignment raw signal threshold, power, raw signal threshold multimodality-weighted Wavelets PCA PCA, multimodality pick-up algorithm, Graph Laplacian features, PCA nonlinear energy, alignment raw signal threshold PCA, ICA with alignment raw signal threshold focused mixture model raw signal threshold PCA N/A N/A + iterative template matching N/A N/A raw signal threshold PCA with alignment Spatiotemporal SVD with alignment raw signal threshold wavelets + Lillifors test	superparamagnetic clustering + template OSort-style template matching matching Variational Bayes and EM t-distribution and Gaussian gradient ascent mixture (mean-shift variant) model focused mixture local density model Generalized Polya urn clustering (Rodriguez & Laio) + template dependent Dirichlet matching process MM Kalman filter EM GMM EM GMM, k-means Template matching via stochastic batch superparamagnetic	Yes Yes? No No No No No No Yes? No No Yes Yes No Yes No No No No Yes No No	No No No Yes Yes? Yes No No No Yes No	(Niediek et al. 2016) (Takekawa et al. 2010; Takekawa et al. 2012) (Spacek et al. 2009; Swindale & Spacek 2014) (Carlson et al. 2014) (Yger et al. 2016) (Gasthaus et al. 2009) (Calabrese & Paninski 2011) (Pachitariu et al. 2016) (Quiroga et al. 2004)	hidden Markov models GUI for inspecting Many options planned; clusters see documentation GUI
						optimization
	MoDT	MATLAB,	N/A		PCA	Mixture of drifting t-	Yes	No	No	(Shan et al. 2017)
		CUDA C				distributions
	moksm	MATLAB	N/A		N/A	Mixture of drifting t-	Yes	No	No	(Shan 2014)
						distributions
	opass	MATLAB	gamma	process	PCA	gamma	process	Yes	Yes	Yes?	(Carlson et al. 2013)
			model			model
	OpenElectrophy	Python	raw	signal	PCA, ICA, wavelets	EM GMM, K-means,	No	No	No		GUI	for	manual
			threshold, MTEO		mean-shift						clustering	and
												inspecting results

Table IV -1: Parameters used for the simplified LTS model

 IV

	Parameter	Value
	ε	0.03
	α n	-200
	α p	-10
	Th	480
	g	100

Table IV -2: Mean F-scores obtained with the different structures tested on the simulated polytrode dataset

 IV

	WTA connections	1	3	5
	Intermediate to output connections
	1	0.476	0.448	0.448
	3	0.332	0.421	0.456
	5	0.177	0.24	0.448

Table V -2: Noise level used and corresponding SNR for the simulated single electrode dataset

 V

	σ noise	0.5	0.75	1	1.25	1.5	1.75	2
	SNR	13.33	8.89	6.67	5.33	4.44	3.81	3.33

Table V -3: Amplitude of the each neural cell's action potential for the different SNR

 V

	SNR		3			4		5		6
	Number of cells	2	3	4	2	3	4 2 3 4	2	3	4
	Action potential amplitude	2.4 2.4 2.1 3.2 3.2 2.8 4 4 3.5 4.8 4.8 4.2
	for each neural cell	3.6 3 2.7 4.8 4 3.6 6 5 4.5 7.2 6 5.4
			3.6 3.3		4.8 4.4	6 5.5		7.2 6.6
				3.9			5.2	6.5			7.8

Table VI -2: Detailed parameters of MiniNet Input layer parameters

 VI

	Parameter	Description	Value
	ΔV m	Sensitivity margin (half-size of the sensitivity range)	1.5σ noise
	N overlap	Number of neuron active at the same time within one column	8
	Δts	Input layer sampling period	0.05 ms
	Δt c	Time interval between two encoding delays	0.05 ms
	N c	Number of encoding delays	24
	Intermediate layer parameters
	Parameter	Description	Value
	N neur	Number of neurons	30

Table VI -3: Main features of ANNet Input layer

 VI Encoding frequency of 80Hz. At each encoding step, 10 signal values, separated by 0.05ms each are encoded (0.5ms time window). The sensitivity margin DV m is 1.75σ noise , overlap is 10. Attention mechanism Attention neuron. Input layer projects to the attention neuron through synapses implementing an STP rule. Intermediate layer 60 LIF neurons. Synapses stemming from the input layer implement and STDP rule.The intermediate layer receives the attention neuron spike-train through fixed weight synapses. The threshold is set so that this layer cannot fire if the attention neuron does not fire. The output layer is connected to the intermediate layer through both excitatory and inhibitory synapses, duplicated with different transmission delays. All these synapses implement an STDP rule. The spike transmission is inhibited by the attention neuron, which thus also prevent STDP.

	Output layer	10 LIF neurons, implementing an IP rule.

Table VI -4: Main features of LTSNet Input layer

 VI Encoding frequency of 80Hz. At each encoding step, 10 signal values, separated by 0.05ms each are encoded (0.5ms time window). The sensitivity margin DV m is 2σ noise , overlap is 10. Attention mechanism Attention neuron. Input layer projects to the attention neuron through synapses implementing an STP rule. Intermediate layer 100 LIF neurons. Synapses stemming from the input layer implement and STDP rule. A 2-WTA mechanism is used. The intermediate layer receives the attention neuron spike train through fixed weight synapses. The threshold is set so that this layer cannot fire if the attention neuron does not fire.

	Output layer	15 simplified LTS neuron.
		Synapses stemming from the intermediate layer are inhibitory and
		implement a lateral STDP rule.

Table VI -5: Detailed parameters of LTSNet Input layer parameters

 VI

	ΔV m τ stdp+ τ stdp-	Sensitivity margin (half-size of the sensitivity Positive STDP rule time window range) Negative STDP rule time window	Δt c *1.01 = 0.0505 ms 2σ noise 0
	N overlap Δw pair	Number of neuron active at the same time within one column Weight change for a presynaptic spike coinciding with a postsynaptic spike	10 0.005
	Δts Δw post	Input layer sampling period Weight change for each postsynaptic spike	0.0125 ms -0.55* Δw pair
	Δt c τ res	Time interval between two encoding delays STDP resource time constant	0.05 ms 3.125 µs
	N c f res	Number of encoding delays STDP resource consumption factor	10 0.5
	Attention neuron parameters Output layer parameters
	Parameter Parameter	Description Description	Value Value
	τ m N neur	Membrane time constant Number of neurons	2* Δt s = 0.025 ms 15
	τ refrac τ m	Refractory period Membrane time constant	0 2 ms
	Th τ refrac	Neurons threshold Refractory period	0.947*N overlap *N c = 94.7 0
	w self V reset	Weight of the self-excitatory synapse Reset potential	0.077*N overlap *N c = 7.7 0
	τ stp	Short term plasticity time constant	20 ms
	W min	Equilibrium weight for the input neuron coding for 0	0.13
	Intermediate layer parameters
	Parameter	Description	Value
	N neur	Number of neurons	100
	τ m	Membrane time constant	2* Δt s = 0.025 ms
	τ refrac	Refractory period	Δt c = 0.05 ms
	V reset	Reset potential	0
	V inhib	Post-inhibition potential	0
	w AN	Weight of the synapses coming from the attention neuron	0.45 * N c *N overlap = 45
	w 0	Average weight of the feedforward synapses at initialization.	0.7
			(w AN + w 0 *N c *N overlap ) *
	Th	Neurons threshold	(1-exp(-Δt c /τ m ))/(1-exp(-Δt s / τ m ))
			= 252.7
	Parameter k WTA	Description Maximum number of neuron that can fire simultaneously	Value 2

Table VI -6: Resources used for MiniNet NEURONS SYNAPSES Layer Number of neurons Number of spike generated Connection Number of synapses Number of spike transmitted

 VI

	Input layer	53	160e3 /s	Input to	38 160	115e6 /s
				intermediate
	Intermediate	30	10 /AP	Intermediate to	300	100 /AP
	layer			output
	Output layer	10	1 /AP

Table VI -7: Resources used for ANNet NEURONS SYNAPSES

 VI

	Layer	Number of	Number of	Connection	Number of	Number of
		neurons	spike		synapses	spike
			generated			transmitted
	Input layer	57	800e3 /s	Input to	34 200	480e6 /s
				intermediate
	Attention	1	40 /AP	Input to	570	8e6 /s
	neuron			attention
				neuron
	Intermediate	60	10 /AP	Intermediate	60 000	100 000 /AP
	layer			to output
	Output layer	10	1 /AP

Table VI -8: Resources used for LTSNet NEURONS SYNAPSES

 VI

	Layer	Number of	Number of	Connection	Number of	Number of
		neurons	spike		synapses	spike
			generated			transmitted
	Input layer	50	800e3 /s	Input to	50 000	800e6 /s
				intermediate
	Attention	1	40 /AP	Input to	500	8e6 /s
	neuron			attention
				neuron
	Intermediate	100	20 /AP	Intermediate	1500	300 /AP
	layer			to output
	Output layer	15	1 /AP

on a field-programmable gate-array (FPGA), which is much more adapted than a computer for parallel processing.

Knowing the receptive field of the neuron and the distribution of the input values, we can then deduce the synapse weight value according to the STDP rule, and thus the value of V pot . The STDP rule state that for each input value 𝑥 ∈ ℝ 𝑁 , the weight change for the synapse corresponding to the i th delay and a sensory neuron with a center value c is:

We note p(x) the probability of x to be an input of the network. The mean variation of the synapse weight when the network receive an input is:

For x following a Gaussian distribution, 〈Δ𝑤 𝑖 (𝑐)〉 is maximum for c=0 and decrease when |c| increase. This works more generally if p(x) is symmetric relatively to each of the dimensions and p(x) decreases when |x i | increases, as D S is also symmetric and its cross-section with the hyperplane define by x i =c decreases when |c| increase. We suppose that for all 𝑥 ∈ 𝐷 𝑠 such that‖𝑥‖ ∞ > 2𝐷𝑉 𝑚 , p(x)=0, which is in particular true if for all 𝑥 ∈ 𝐷 𝑠 ‖𝑥‖ ∞ ≤ 2𝐷𝑉 𝑚 . Then 〈Δ𝑤 𝑖 (𝑐)〉 is positive for c=0 and negative when |c| is high enough. Thus there exist a unique value V pot for which 〈Δ𝑤 𝑖 (𝑐)〉 is positive if |c|<V pot , and negative if |c|>V pot , which correspond to the initial definition of V pot . Additionally, if Δ𝑤 + = 2Δ𝑤 -, we can verify that V pot =DV m , thanks to the symmetry properties of D S and p.

Article about ANNet

The following pages are an article, recently accepted for publication in the International Journal of Neural Systems. This article sums up the implementation and results obtained with ANNet (see Section VI.B).

Conclusion et perspectives

Le travail effectué a permis de montrer la faisabilité d'appliquer un réseau STDP au problème de spikesorting. Les performances obtenues sont en effet comparables, voire supérieures, à celle de l'état de l'art pour des signaux mono-électrodes. Les mécanismes développés pour répondre aux différents aspects du problème, à savoir reconnaitre des motifs temporels de tailles variables parmi des stimuli non pertinents, pourraient être appliqués à des problèmes similaires. Des améliorations peuvent encore être apportées à ce réseau, notamment pour le traitement de signaux multi-électrodes, par des modifications de structure ou l'ajout de nouveaux mécanismes. Si l'implémentation sur FPGA marque un premier pas vers une application embarquée, l'utilisation de puces neuromorphiques pour un tel algorithme marquerait une grande avancée pour le spike-sorting et pour le traitement de signaux neuronaux en général. Il y a en effet, pour ce type de traitement, un besoin important d'implémentations embarquées et peu consommatrices, étant données les contraintes sur les implants d'enregistrement.