Diemo Schwarz
email: schwarz@ircam.fr

Dominique Fourer

Methods and Datasets for DJ-Mix Reverse Engineering

HAL is

Introduction

Understanding DJ practices remains a challenging important part of popular music culture [START_REF] Barchiesi | Reverse engineering of a mix[END_REF][START_REF] Brecht De Man | An analysis and evaluation of audio features for multitrack music mixtures[END_REF]. The outcomes from such an understanding are numerous for musicological research in popular music, cultural studies on DJ practices and reception, music technology for computer support of DJing, automation of DJ mixing for entertainment or commercial purposes, and others. In order to automatically annotate recorded mixes, several components are required:

Identification of the contained tracks (e.g. fingerprinting) to obtain the playlist, Alignment to determine where in the mix each track starts and stops, Time-scaling to determine what speed changes were applied by the DJ to achieve beat-synchronicity, Unmixing to estimate the cue regions where the cross-fades between tracks happen, the curves for volume, bass and treble, and the parameters of other effects (compression, echo, etc.), Content and metadata analysis to derive the genre and social tags attached to the music to inform about the choices a DJ makes when creating a mix.

The ABC DJ project has received funding from the European Unions Horizon 2020 research and innovation programme under grant agreement No 688122.

Most of these components have been addressed by recent MIR research except the alignment, time-scaling, and unmixing part for which we propose a method based on multi-scale correlation, dynamic time warping, and time-frequency gain curve estimation to increase its robustness to interferences with other tracks. To come closer to actual DJ practices, we can retrieve the alignment and volume curves from example DJ mixes, and then combine them with content and genre information to investigate the content-dependent aspects of DJ mix methods.

As a working definition, we can roughly distinguish three levels of mixing:

Level 1, broadcast mixing, is a simple volume cross fade without paying attention to changing content (as performed by consumer audio players such as iTunes, or in a broadcast context). Level 2, lounge mixing, is beat-synchronous mixing with adaptation of the speed of the tracks and possibly additional EQ fades, while playing the tracks mostly unchanged. Level 3, performative mixing, is using the DJ deck as a performance instrument by creative use of effects, loops, and mashups with other tracks.

This paper addresses the level 1 and 2 cases, while level 3 can blur the identifiability of the source tracks.

Related Work

There is much more existing work in the field of studio mixing where a stereo track is produced from individual multi-track recordings and software instruments by means of a mixing desk or DAW [START_REF] Brecht De Man | An analysis and evaluation of audio features for multitrack music mixtures[END_REF][START_REF] Jacob A Maddams | An autonomous method for multi-track dynamic range compression[END_REF][START_REF] Mansbridge | An autonomous system for multitrack stereo pan positioning[END_REF][START_REF] Perez-Gonzalez | Automatic gain and fader control for live mixing[END_REF]. They produced ground truth databases [START_REF] De Man | The open multitrack testbed[END_REF] and crowd-sourced knowledge generation [START_REF] De | Crowd-sourced learning of music production practices through large-scale perceptual evaluation of mixes[END_REF] with some overlap with DJ mixing. However, when seeing the latter as the mixing of only two source tracks, the studied parameters and influencing factors differ too much from what is needed for DJ mixing. There is quite some existing work on methods to help DJs to produce mixes [START_REF] Aspillaga | Mixme: A recommendation system for DJs[END_REF][START_REF] Bittner | Automatic playlist sequencing and transitions[END_REF][START_REF] Cliff | Hang the DJ: Automatic sequencing and seamless mixing of dancemusic tracks[END_REF][START_REF] Fujio | A system of mixing songs for automatic DJ performance using genetic programming[END_REF][START_REF] Ishizaki | Full-automatic DJ mixing system with optimal tempo adjustment based on measurement function of user discomfort[END_REF][START_REF] Kim | Automatic DJ mix generation using highlight detection[END_REF][START_REF] Molina | Beatjockey: A new tool for enhancing DJ skills[END_REF], but much less regarding information retrieval from recorded mixes, with the exception of content-based analysis of playlist choices [START_REF] Kell | Empirical analysis of track selection and ordering in electronic dance music using audio feature extraction[END_REF], track boundaries estimation in mixes [START_REF] Glazyrin | Towards automatic content-based separation of DJ mixes into single tracks[END_REF][START_REF] Scarfe | Segmentation of electronic dance music[END_REF], and the identification of the tracks within the mix by fingerprinting [START_REF] Sonnleitner | Landmark-based audio fingerprinting for DJ mix monitoring[END_REF]. To this end, Sonnleitner et. al. provide an open dataset3 of 10 dance music mixes with a total duration of 11 hours and 23 minutes made of 118 source tracks. The included playlists contain hand-annotated time points with relevant information for fingerprinting, namely the approximate instant when the next track is present in the mix. Unfortunately, this information is not accurate enough for estimating the start point of the track in the mix. As a result, it cannot be used for our aims of DJ mix analysis and let alone reverse engineering.

Barchiesi and Reiss [START_REF] Barchiesi | Reverse engineering of a mix[END_REF] first used the term mix reverse engineering (in the context of multi-track studio mixing) for their method to invert linear processing (gains and delays, including short FIR filters typical for EQ) and some dynamic processing parameters (compression). Ramona and Richard [START_REF] Mathieu | A simple and efficient fader estimator for broadcast radio unmixing[END_REF] tackle the unmixing problem for radio broadcast mixes, i.e. retrieving the fader positions of the mixing desk for several known input signals (music tracks, jingles, reports), and one unknown source (the host and guests' microphones in the broadcast studio). They model the fader curves as a sigmoid function and assume no time-varying filters, and no speed change of the sources that is only correct in the context of radio broadcast. These two latter references both assume having sample-aligned source signals at their disposal, with no time-scaling applied, unlike our use-case where each source track only covers part of the mix, can appear only partially, and can be time-scaled for beat-matched mixing. There is rare work related to analysis [START_REF] Fourer | Objective characterization of audio signal quality: Application to music collection description[END_REF] and inversion of non-linear processing applied to the signal such as dynamic-range compression [START_REF] Gorlow | Model-based inversion of dynamic range compression[END_REF] which remains challenging and full of interest for unmixing and source separation.

Hence, this work realizes our idea first presented in [START_REF] Schwarz | Towards Extraction of Ground Truth Data from DJ Mixes[END_REF], by applying it to a large dataset of generated DJ mixes [START_REF] Schwarz | Unmixdb: A dataset for DJ-mix information retrieval[END_REF]. It already inspired work on a variant of our unmixing method based on convex optimization, and a hand-crafted database [START_REF] Werthen-Brabants | Ground truth extraction & transition analysis of DJ mixes[END_REF].

DJ Mix Reverse Engineering

The input of our method is the result of the previous stage of identification and retrieval on existing DJ mixes or specially contrived databases for the study of DJ practices. We assume a recorded DJ mix, a playlist (the list of tracks played in the correct order), and the audio files of the original tracks. Our method proceeds in five steps, from a rough alignment of the concatenated tracks with the mix by DTW (section 3.1), that is refined to close in to sample precision (section 3.2), then verified by subtracting the track out of the mix (section 3.3), to the estimation of gain curves (section 3.4) and cue regions (section 3.5).

Step 1: Rough Alignment

Rough alignment uses the Mel Frequency Cepstral Coefficients (MFCC) of the mix X(k, c) (k being the mix frame index and c ∈ {1, 2, ..., 13} the Mel frequency index) and the concatenated ones of the I tracks S(l, c) = (S 1 . . . S I ) as input (window size 0.05 s, hop size 0.0125 s), l being the frame index of the concatenated matrix S. The motivation is that the MFCC representation is more robust in practice than discrete Fourier-based representation against possible pitch changes from time-scaling of the source tracks in the DJ mix. Since the tracks are almost unchanged in level 2 mixes, Dynamic Time Warping (DTW) [START_REF] Robert | Ground-truth transcriptions of real music from force-aligned midi syntheses[END_REF] can latch on to large valleys of low distance, although the fade regions in the mix are dissimilar to either track, and occur separately in S(l, c). To ease catching up with the shorter time of the mix, we provide a neighborhood allowing the estimated alignment path to perform larger vertical and horizontal jumps, shown in Fig. 1 (right). The DTW alignment path not only gives us the relative positioning of the tracks in the mix, but also their possible speed change, applied by the DJ to achieve beat-synchronous mixing, see Fig. 1 (left): First, we estimate the speed factor, assuming that it is constant for each track, by calculating the mean slope of the alignment path in a window of half the track length, centred around the middle of the track. Then, the intersections of the slope lines with the track boundaries in S(l, c) provide an estimate of the frame start of the tracks in the mix. The start position expresses the offset of the start of the full source track with respect to the mix, and not the point from where the track is present in the mix. Since the source tracks are mixed with non-zero volume only between the cue-in and cue-out regions, the track start point can be negative.

Step 2: Sample Alignment

Given the rough alignment and the speed estimation provided by DTW, we then search for the best sample alignment of the source tracks. To this end, we first time-scale the source track's signal according to the estimated speed factor. We then shift a window of the size of an MFCC frame, taken from the middle of the time-scaled track, around its predicted rough frame position in the mix. The best time shift is simply provided by the maximum of the cross-correlation computed between the mix and the track. Please note that this process is not directly applied during the step 1 due to the high computational cost. The sample alignment considers a maximal delay equal to the size of a window and can be computed in a reasonable time.

Step 3: Track Removal

The success of the sample alignment can be verified by subtracting the aligned and time-scaled track signal from the mix for which a resulting drop in the rootmean-square (RMS) energy is expected. This method remains valid when the ground truth is unknown or inexact. Fig. 2 illustrates the result of track removal applied on a mix in our dataset. We can observe that the resulting instantaneous RMS energy of the mix (computed on the size of a sliding window) shows a drop of about 10 dB. A short increase is also observed during the fades where the suppression gradually takes effect.

Step 4: Volume Curve Estimation

We introduce a novel method based on the time-frequency representation of the signal to estimate the volume curve applied to each track to obtain the mix. Given the discrete-time mix signal denoted x(n) and the constituent samplealigned and time-scaled tracks s i (n), we aim at estimating the mixing function a i (n) as:

x(n) = I i=1 a i (n)s i (n) + b(n) , ∀n ∈ Z (1)
where b(n) corresponds to an additive noise signal. From a "correctly" aligned track s i , its corresponding volume curve âi is estimated using the following steps:

1. we compute the short-time Fourier transforms (STFT) of x and s i denoted S i (n, m) and X(n, m) (n and m being respectively the time and frequency indices)

2. we estimate the volume curve at each instant n by computing the median of the mix/track ratio computed along the frequencies m ∈ M, where M is the set of frequency indices where |S i (n, m )| 2 > 0, such as:

âi (n) =    median |X(n,m )| |Si(n,m )| ∀m ∈M if ∃m s. t. |S i (n, m )| 2 > 0 0 otherwise (2)
3. we optionally post-process âi (n) to obtain a smooth curve by removing outliers using a second median filter for which a kernel size equal to 20 provides good results in practice.

The resulting volume curve can then be used to estimate the cue points (the time instants when a fading effect begins or stops) at the next step. An illustration of the resulting process is presented in Fig. 3.

Step 5: Cue Point Estimation

In order to estimate the DJ cue points, we apply a linear regression of âi at the time instants located at the beginning and at the end of the resulting volume curve (when âi (n) < Γ , Γ being a threshold defined arbitrarily as Γ = 0.7 max(â)). Assuming that a linear fading effect was applied, the cue points can easily be deduced from the two affine equations resulting from the linear regression. The four estimated cue points correspond respectively to:

1. n 1 , the time instant when the fade-in curve is equal to 0 2. n 2 , the time instant when the fade-in curve is equal to max(â i ) 3. n 3 , the time instant when the fade-out curve is equal to max(â i ) 4. n 4 , the time instant when the fade-out curve is equal to 0.

In order to illustrate the efficiency of the entire method (steps 4 and 5), we present in Fig. 3 the results obtained on a real-world DJ-mix extracted from our proposed dataset.

The UnmixDB Dataset

In order to evaluate the DJ mix analysis and reverse engineering methods described above, we created a dataset containing excerpts of open licensed dance tracks and their corresponding automatically generated mixes [START_REF] Schwarz | Unmixdb: A dataset for DJ-mix information retrieval[END_REF], available at https://zenodo.org/record/1422385. We use track excerpts of c.a. 40 seconds due to the high runtime and memory requirements, especially for the DTW that is of quadratic memory complexity.

Each mix is based on a playlist made of 3 track excerpts such that the middle track is embedded in a realistic context of beat-aligned linear cross fading to the other tracks. The first track's BPM is used as the seed tempo onto which the other tracks are adapted.

Each playlist of 3 tracks is mixed 12 times with combinations of 4 variants of effects and 3 variants of time scaling using the treatments of the sox open source command-line program.The 4 effects are: none: no effect bass: +6 dB bass boost using a low-shelving biquad filter below 100 Hz compressor: heavy dynamics compression (ratio of 3:1 above -60 dB, -5 dB makeup gain) distortion: heavy saturation with +20 dB gain These effects were chosen to cover treatments likely to be applied to a DJ set (EQ, compression), and also to introduce non-linear treatments (distortion) to test the limits of re-engineering and unmixing methods.

The 3 timescale methods are:

none: no time scaling, ie. the tracks are only aligned on the first beat in the cue region and then drift apart resample: linked time and pitch scaling by resampling (sox speed effect) stretch: time stretching while keeping the pitch (sox tempo effect using WSOLA) These 3 variants allow to test simple alignment methods not taking into account time scaling, and allow to evaluate the influence of different algorithms and implementations of time scaling.

The UnmixDB dataset contains the complete ground truth for the source tracks and mixes. For each mix, the start, end, and cue points of the constituent tracks are given with their BPM and speed factors. Additionally, the song excerpts are accompanied by their cue region and tempo information.

Table 1 shows the size and basic statistics of the dataset. We also publish the Python source code to generate the mixes, such that other researchers can create test data from other track collections or other variants.

Our DJ mix dataset is based on the curatorial work of Sonnleitner et. al. [START_REF] Sonnleitner | Landmark-based audio fingerprinting for DJ mix monitoring[END_REF], who collected Creative-Commons licensed source tracks of 10 free dance music mixes from the Mixotic net label. We used their collected tracks to produce our track excerpts, but regenerated artificial mixes with perfectly accurate ground truth.

Evaluation

We applied the DJ mix reverse engineering method on our UnmixDB collection of mixes and compared the results to the ground truth annotations. To evaluate the success of our method we defined the following error metrics: frame error: absolute error in seconds between the frame start time found by the DTW rough alignment (step 1, section 3.1) and the ground truth (virtual) track start time relative to the mix sample error: absolute error in seconds between the track start time found by the sample alignment (step 2, section 3.2) and the ground truth track start time relative to the mix speed ratio: ratio between the speed estimated by DTW alignment (step 1, section 3.1) and the ground truth speed factor (ideal value is 1) suppression ratio: ratio of time where more than 15 dB of signal energy could be removed by subtracting the aligned track from the mix, relative to the time where the track is fully present in the mix, i.e. between fade-in end and fade-out start (step 3, section 3.3, bigger is better) fade error: the total difference between the estimated fade curves (steps 4 and 5, sections 3.4 and 3.5) and the ground truth fades. This can be seen as the surface between the 2 linear curves over their maximum time extent. The value has been expressed in dB s, i.e. for one second of maximal difference (one curve full on, the other curve silent), the difference would be 96 dB. The sample alignment results given in Fig. 6 and Table 2 show that the ground truth labels can be retrieved with high accuracy: the median error is 25 milliseconds, except for the mixes with distortion applied, where it is around 100 ms. These errors can already be traced back to the rough alignment (section 3.1):

Fig. 4 shows that it is not robust to heavy non-linear distortion, presumably because the spectral shape changes too much to be matchable via MFCC distances. This error percolates to the speed estimation (Fig. 5), and sample alignment.

The track removal time results in Fig. 8 show sensitivity to the bass and distortion effect (because both of these introduce a strong additional signal component in the mix that is left as a residual when subtracting a track), and also perform less well for time-scaled mixes.

The fade curve volume error in Fig. 10 shows a median of around 5 dB s, which corresponds to a very good average dB distance of 0.3 dB, considering that the fades typically last for 16 seconds. While developing our method, we noticed the high sensitivity of the sample alignment and subsequent track removal (steps 2 and 3, sections 3.2 and 3.3) on the accuracy of the speed estimation. This is due to the resampling of the source track to match the track in the mix prior to track removal. Even an estimation error of a tenth of a percent results in desynchronisation after some time. To judge the influence of this accuracy, we produced a second set of the sample error and suppression ratio metrics based on a run of steps 2 and 3 with the ground truth speed re-injected into the processing. The rationale is that the speed estimation method could be improved in future work, if the resulting reductions of error metrics are worthwhile. Also note that the tempo estimation is inherently inaccurate due to it being based on DTW's discretization into MFCC frames. In mixes with full tracks, the slope can be estimated more accurately than with our track excerpts simply because more frames are available. Figures 7 and9 show the quartile statistics of the sample error and suppression ratio with re-injected ground truth speed. We can see how most variants are improved in error spread for the former, and 4 variants are greatly improved for the latter, confirming the sensitivity of the track removal step 3 on the speed estimation.

Conclusions and Future Work

The presented work is a first step towards providing the missing link in a chain of methods that allow the retrieval of rich data from existing DJ mixes and their source tracks. An important result is the validation using track removal in section 3.3 to compute a new metric for the accuracy of sample alignment. This metric can also be computed even without ground truth. A massive amount of training data extracted from the vast number of collections of existing mixes could thus be made amenable to research in DJ practices, cultural studies, and automatic mixing methods. With some refinements, our method could become robust and precise enough to allow the inversion of fading, EQ and other processing [START_REF] Barchiesi | Reverse engineering of a mix[END_REF][START_REF] Mathieu | A simple and efficient fader estimator for broadcast radio unmixing[END_REF]. First, the obtained tempo slope could be refined by searching for sample alignment at several points in one source track. This would also extend the applicability of our method to mixes with non-constant tempo curves. Second, a sub-sample search for the best alignment should achieve the neutralisation of phase shifts incurred in the mix production chain. We could also check whether a DTW with relaxed endpoint condition [START_REF] Silva | On the effect of endpoints on dynamic time warping[END_REF] for the beginning and end of a mix could be advantageous. Furthermore, the close link between alignment, time-scaling, and unmixing hints at the possibility of a joint and possibly iterative estimation algorithm, maximising the match in the three search spaces simultaneously.

Fig. 1 :

 1 Fig. 1: Left: DTW distance matrix, alignment path (red), track boundaries (vertical lines), found slope lines anchored on track mid-point (circles), and estimated track start (blue marks) on an artificial DJ mix of 3 tracks from our dataset. Right: extended DTW neighbourhood.

Fig. 2 :

 2 Fig. 2: Resulting RMS energy (in dB) after the subtraction of each track (3) from a mix including fades. Each source track signal is filled with zeros to obtain the same duration.

Fig. 3 :

 3 Fig. 3: Estimated volume curve (black), linear fades (blue), ground truth fades (red)

Figures 4 -

 4  show the quartile statistics of these metrics, broken down by the 12 mix variants (all combinations of the 3 time-scaling methods and 4 mix effects). The sample alignment results given in Fig.6and Table2show that the ground truth labels can be retrieved with high accuracy: the median error is 25 milliseconds, except for the mixes with distortion applied, where it is around 100 ms. These errors can already be traced back to the rough alignment (section 3.1):

Fig. 5 :

 5 Fig. 5: Box plot of ratio between estimated and ground truth speed per variant.

Fig. 7 :

 7 Fig. 7: Box plot of absolute error in track start time found by sample alignment when re-injecting ground truth speed.

Fig. 10 :

 10 Fig. 10: Box plot of volume difference of fades per variant per mix variant.

Table 1 :

 1 Basic statistics of the UnmixDB dataset.

	Number of tracks	37
	Number of playlists	37
	Number of tracks per playlist	3
	Number of variants per playlist 12
	Number of mixes	444
	Average duration of tracks [s]	46
	Average duration of mixes [s]	107
	Total duration of tracks [min]	1016
	Total duration of mixes [min]	2743
	Median tempo of tracks [bpm]	128
	Minimum tempo of tracks [bpm] 67
	Maximum tempo of tracks [bpm] 140

Table 2 :

 2 Statistics of absolute error in track start time found by sample alignment.

		mean std	min median max
	none none	0.0604 0.2469 0.0010 0.0251 2.1876
	none bass	0.1431 0.7929 0.0005 0.0254 7.7191
	none compressor	0.0806 0.4424 0.0010 0.0251 4.4995
	none distortion	1.3376 3.3627 0.0011 0.1042 23.7610
	resample none	1.1671 7.0025 0.0002 0.0270 71.0080
	resample bass	1.3337 7.2079 0.0005 0.0277 73.1192
	resample compressor 6.8024 17.0154 0.0010 0.0372 134.2811
	resample distortion 1.8371 3.8551 0.0013 0.1483 23.8355
	stretch none	0.2502 1.1926 0.0002 0.0251 10.0048
	stretch bass	0.3300 1.4249 0.0005 0.0264 9.6626
	stretch compressor 0.1520 1.0025 0.0008 0.0251 10.1076
	stretch distortion	1.0629 2.2129 0.0014 0.0911 10.3353
	all	1.2131 6.2028 0.0002 0.0282 134.2811

http://www.cp.jku.at/datasets/fingerprinting


Mahyar Shahsavari
email: mahyar.shahsavari@gmail.com

Philippe Devienne

Pierre Boulet

Spiking Neural Computing in Memristive Neuromorphic Platforms

Neuromorphic computation using Spiking Neural Networks (SNN) is proposed as an alternative solution for future of computation to conquer the memory bottelneck issue in recent computer architecture. Different spike codings have been AQ1 discussed to improve data transferring and data processing in neuro-inspired computation paradigms. Choosing the appropriate neural network topology could result in better performance of computation, recognition and classification. The model of the neuron is another important factor to design and implement SNN systems. The speed of simulation and implementation, ability of integration to the other elements of the network, and suitability for scalable networks are the factors to select a neuron model. The learning algorithms are significant consideration to train the neural network for weight modification. Improving learning in neuromorphic architecture is feasible by improving the quality of artificial synapse as well as learning algorithm such as STDP. In this chapter we proposed a new synapse box that can remember and forget. Furthermore, as the most frequent used unsupervised method for network training in SNN is STDP, we analyze and review the various methods of STDP. The sequential order of pre-or postsynaptic spikes occurring across a synapse in an interval of time leads to defining different STDP methods. Based on the importance of stability as well as Hebbian competition or anti-Hebbian competition the method will be used in weight modification. We survey the most significant projects that cause making neuromorphic platform. The advantages and disadvantages of each neuromorphic platform are introduced in this chapter.

Introduction

The mammalian nervous system is a network of extreme complexity which is able to perform cognitive computation in a parallel and power-efficient manner. Understanding the principles of the brain processing for computational modeling is one of the biggest challenges of the 21st century that led to the new branch of research e.g., neuromorphic computing. Neuromorphic engineering represents one of the promising fields for developing new computing paradigms complementing or even replacing current Von Neumann architecture [1].

The main remarkable difference between conventional Von Neumann architecture and neuromorphic systems is in their use of memory structures. The way of communication between memory and central processing unit (CPU) in conventional computing is not efficient. The memory and CPU communication suffers from what is called Von Neumann memory bottelneck. The CPUs access both data and program in memory using the same shared resources. CPUs spend most of their time idle because the speed of CPU is much more than memory due to the quality of materials applied to manufacturing the transistors in CPU and different memories.

If we want to apply better quality of memory such as SRAM, regarding to the high demands of memory usages the machine would be more expensive. To improve the efficiency of nowadays computation platforms, the applicable solution is what commonly known as the cache hierarchy; in other words, a limited amount of fast but costly memory sit closer to the processing unit, while most of the data would be stored in the cheaper but larger memory as it is shown in Fig. 1a. To execute computational tasks, instruction codes and data stored in the memory are fetched to the processor, and after execution, pushed back to the memory unit, via a memory bus.

Subsequently, it would be operating system (OS) duty to manage the data around these different levels of memory to optimize the system speed by consisting frequentlyused data to the closer memory with better quality and speed rate. On the other hand, the multi-core platforms are commonly used in the new hardwares and the memory hierarchy management would be more significant and difficult too. By proposing computing unit next to the local memory, neuromorphic brain-inspired computing paradigms offer an attractive solution for implementing alternative non von Neumann architectures, using advanced and emerging technologies such as memristor [2].

Neuromorphic systems are electronic implementations inspired from neural systems that is known as neuro-inspired computation system. The idea of creating circuit model for a neuron system refers back at least to 1907, where a neuron is modeled by a resistor and a capacitor [3]. However, the first neuromorphic term was coined by Carver Mead [START_REF] Mead | Proc. IEEE[END_REF] using Very Large Scale Integration (VLSI) technology to propose an implementation of neural system hardware. Mahowald and Mead implemented the first silicon retina model with considering of adaptivity and energy efficiency by simulating retina functionalities [START_REF] Mahowald | [END_REF]. Tobi Delbruck built on the idea of adaptive photoreceptor circuits developed in [START_REF] Delbruck | Proceedings of IEEE International Symposium on Circuits and Systems-ISCAS '94[END_REF] and presented approaches for enhancing retinomorphic sensors consist of 128×128 pixel Dynamic Vision Sensor (DVS). DVS established a benchmark in neuromorphic vision domain with introducing Address Event Representation (AER) sensory data in which each individual pixel processed the normalized time derivative of the sensed light and provided an output in the form of spikes of the pixel addresses. In addition, vision sensory neuromorphic research, there are several neuromorphic studies using auditory and olfactory sensors [START_REF] Sarpeshkar | Neuromorphic Systems Engineering: Neural Networks in Silicon[END_REF][START_REF] Chiu | [END_REF][9] for review study in neuromorphic research using different sensory inputs, we refer the readers to [10].

(a) (b)

Fig. 1 Computational architecture a Von Neumman architecture, fast and costly memory are closer to cores in multiprocessor platforms as cashes and local memory as well as inexpensive and slower memory are in other layers close to magnetic memory to save the cost of CPU (memory hierarchy). b Neuromorphic architecture inspired from neural networks in the biological brain, capable to conquer Von neumann bottelneck issue, performing parallel and cognitive computing, as well as considering that the synapses are local memories connected to each neurons as computational cores More close to our research, in 2014 two distinguished articles were published that increased the scientists attentions to the general neuromorphic platforms as novel computing architectures. Merolla et al. [11] in an IBM research was sponsored by DARPA, have demonstrated a computing hardware consist of the compact modular core for large-scale neuromorphic system architecture. The cores combine digital neurons with the large synaptic array. This general purpose neuromorphic processor was built using thousands of neurosynaptic cores are involved one million neurons and 256 million of reconfigurable synapses. The second notable work published in 2014 was Spiking Neural Network Architecture (SpiNNaker) project [12].

The SpiNNaker project is a decade old, comprehensive description of the project is announced in [12]. SpiNNaker project aims to deliver a massively parallel million core architectures whose interconnections are inspired by the connectivity properties of the mammalian brain. The hardware platform is suitable to model the large-scale spiking neural networks in biological real time. Neuromorphic and neuro-inspired computing is now being adapted by an increasing number of academic and industrial different research teams. In recent few years, there have been many valuable publications explaining the use of novel materials such as memristors are able to emulate some of the properties observed in biological synapses [2,[13][START_REF] Shi | 2015 IEEE International Electron Devices Meeting (IEDM)[END_REF][START_REF] O'connor | [END_REF][16][17]. Our work focuses on an alternative approach aimed at high performance computation to realize the compact, parallel, cognitive and energy-efficient architecture structure that emulate the style of computation of the biological brain, using the Spiking Neural Network (SNN) structure, modeling the neurons as computational cores next to memristive artificial synapses as local memories to skip memory delay bottelneck similar to what is shown in Fig. 1b. Therefore, it is necessary to define, analyze and verify the efficient models of network topology, neuron and synapse models based on state-of-the-art technologies besides choosing the optimized learning model adapted to our platform and devices. The structure of Chapter is followed by reviewing SNN and more significantly the functionality of various spike information codings. In the same section, we discuss different neural network topologies. Furthermore in the Sect. 3, different models of neuron is presented. Synapse and learning are explained in the Sect. 4 which various methods of spike-timing-dependent plasticity (STDP) [18,19] are studied comprehensively. The state-of-the-art of the most important neuromorphic platforms and projects in the world is presented in Sect. 5.

Lateral inhibition and Homeostasis have been discussed at the discussion part of this chapter.

Spiking Neural Networks

Artificial neural networks (ANN) can generally be categorized into three generations.

The first generation of neural network consisted of McCulloch and Pitts neurons [20] that the output signals are limited to discrete '0' or ' In this model, the precise firing times of neurons are used for information coding.

Spiking neural networks belong to the third generation of neural networks.

Indeed, artificial neural network in the first and second generation is a mathematical model of mammalian brain though, SNN is an electronic hardware neuromorphic model of the biological brain. Networks composed of spiking neurons are able to process significant amount of data using a relatively small number of spikes [21].

Due to the similarity between the biological neurons and spiking models functional-

Spike Information Coding

Spike is the language that neurons communicate to each other in SNN architectures.

One of the key unresolved questions in neuroscience is how information processed in the brain. The nature of the neural code is an unresolved topic of research in neuroscience. However, based on what is known from biology, a number of neural information encoding have been proposed:

1.

Rate coding

The rate of spikes in a time-window is counted for the information transmission.

It is also called as frequency coding (Fig. 2a). As the intensity of a stimulus increases more, the firing rate of spikes increases more too. Rate encoding is motivated by the observation that biological neurons eager to fire more often for stronger stimuli. There are two types of rate coding namely spike-count rate and time-dependent firing rate. In spike-count rating by counting the number of spikes that are generated during a trial and dividing by the duration of the trial, we calculate the temporal average of rating. In independent firing rate, the average number of spikes over trial happens during a short interval between times t and t + t, divided by the duration of the interval. Brette [22] has compared these two approaches in rate information coding in more details.

Latency coding

In this model, information is supposed to be contained in the exact timing of a set of spikes relative to each other as it is shown in Fig. 2b. It is already proved that precisely timed patterns of spikes have been postulated to play a significant role in the networks of neuron in different functions [23]. Precise spike timing is one of the important parameters that control variety forms of synaptic plasticity.

Latency coding by using sequences of spikes are mainly observed in feed-forward networks since noise and dynamics of recurrent networks can disrupt spike timing precision, some attempts to harvest precise spiking timing in recurrent networks have been done for example by exploring the idea of reservoir computation [24].

Phase coding

This model generates the times of emitted spikes based on the time point in a periodic signal. In this (Fig. 2c) method the spike trains can encode information in the phase of a pulse respecting to the background oscillations. Phase coding method has been used both in models and experimentally. Phase coding has been suggested for the hippocampus as well [25]. Spiking networks exploring the phase coding strategy have recently been applied in tasks as olfactory systems or robot navigation [26].

Rank-coding (spike-order coding)

In this method of spike coding, information is encoded by the order of spikes in the activity of a group of neurons as it is depicted in Fig. 2d. Rank-coding approach has been suggested to describe ultra-fast categorization observed in the visual system. This model assumes that each neuron emits only a single spike during a presentation of the image. This method can be implemented in a feedforward network with inhibitory feedback connections. developed a spiking neural model that was able to classify static images with a processing speed comparable to that observed in humans one.

Population coding

This coding model is a method to introduce stimuli by applying the joint activities of the group of neurons. In population coding, each neuron has a distribution of responses to the certain set of inputs, and the responses of group of neurons will be combined to present a value for the inputs (Fig. 2e). During the last two decades, the theory has focused on analyzing the methods in which different parameters that characterize neuronal responses to external stimuli affect the information content of these responses. Recent challenge in population coding is to develop a theory that can generate predictions for specific readout mechanisms for example for visual target information [28].

Sparse coding

This model of coding generally refers to a representation where a few number of neurons are active, with the majority of the neurons inactive or showing low activity see Fig. 2f. Sparse coding has been suggested as a guiding principle in neural representations of sensory input, specially in the visual sensory system.

It is also discussed that sparse coding offers a useful solution to the problem of representing natural data because such a scheme allows the system to take advantage of the sparse structure of the sensory environment. It is believed that the natural environment is inherently sparse and codes that using this structure can be both metabolically efficient and useful for learning. Sparseness can be defined over a population of neurons at a specific point in time (population sparseness)

or it can be measured for a single neuron over a certain time-window [29].

Network Topology

The interconnection structure of neurons in a network of neurons is called topology, architecture or graph of an artificial neural network. The manner in which the interconnection is structured intimately is linked to the learning algorithms applied to train the neural networks. Indeed, the interconnection can be structured in numerous ways results in numerous possible topologies that are divided into two basic classes namely: Feed-Forward Neural Networks (FFNN) and Recurrent (or feedback) Neural Networks (RNN) depicted in Fig. 3.

Feed-Forward Neural Networks (FFNN)

The FFNN is divided into two different structure called single-layer FFNN and multilayer FFNN. The single-layer is structured as an input and output layer which is strictly a feed-forward or acyclic graph. We do not count the input layer because no calculation is performed in input nodes (neurons). The multilayer FFNN has one or more hidden layers between input and output layers similar to Fig. 3a which has one hidden layer. By adding one or more hidden layers, the neural network can extract the higher-order statistics which is particularly valuable when the size of the input layer is large [START_REF] Haykin | Neural Networks: A Comprehensive Foundation[END_REF]. Among the known types of neural networks (NN), the feed-forward neural networks are the mostly used because of their simplicity, flexible structure, good qualities of representation, and their capability of universal approximation. Respecting to the way of interconnectivity of the nodes (neurons) there are two kinds of feed-forward architecture:

• fully connected

In this configuration, every node in each layer of the network is connected to every other node in the next layer. In fact, we can call them globally connected networks. The Restricted Boltzmann Machine (RBM) could be an example of fully connected FFNN.

• partially connected

In this configuration, some communication links are missing. The convolutional neural networks is a good example for the partially connected FFNN. Partially connected topologies present a suitable alternative with a reduced degree of redundancy and thus a potential for increased efficiency of neural networks.

Recurrent Neural Networks (RNN)

The RNN is distinguished from FFNN in that it has at least one feedback loop connection. Recurrent neural networks can be single-layer or multilayer as well. Unlike feed-forward neural networks, recurrent networks retain a state that can represent information from an arbitrarily long context window. Although recurrent neural networks have traditionally been difficult to train, and often contain thousands of parameters, recent studies in network architectures, optimization techniques, and parallel computation have enabled successful large-scale learning to use RNN [START_REF] Lipton | [END_REF]. Hopfield [START_REF] Hopfield | Proc. Natl. Acad. Sci[END_REF] network is an example of the recurrent artificial neural network that is used to store one or more stable vectors. The stable vectors can be considered as memories that the network recalls them when provided with similar vectors that operate as a queue to the network memory. Other example of RNN is Elman network [START_REF] Elman | [END_REF] that refers as a simple Recurrent Network is the special case of recurrent artificial neural networks. This type of artificial neural network has the memory that allows it to both detect and generate time-varying patterns.

Modern Neural Networks

Here, we discuss recent feed-forward promising neural network which has been applied in different sensory computation applications.

U N C O R R E C T E D P R O O F

from multiple neurons in the previous layer and produce an output which is a threshold or sigmoidal function of the weighted sum of its inputs. The connectivity pattern between the nodes of one layer and the node of the subsequent layer, responsible for the weighted sum operation forms the convolution kernel. Each layer mainly has one or few number of convolution kernels that link the activity of a set of neurons from one layer to the target neuron of the next layer [34].

Convolutional neural networks which have been explored intensively within the neuromorphic community for visual processing tasks [35]. They are normally implemented on CPUs and GPUs which consume a significant amount of power.

In recent years, System-On-Chip (SOC) solutions and FPGA platforms have been used to implement these networks for increasing their performance while decreasing their power consumption.

• Deep Belief Networks (DBN) Deep learning is currently an extremely active research area in machine learning and cognitive computing society. It has obtained many successes in a wide area of applications such as speech recognition, computer vision, and natural language processing. Deep Belief Networks (DBNs), introduced by Hinton and his colleagues as a special type of deep neural networks with generative model properties [36]. This network is structured as interconnected pairs of Restricted Boltzmann Machines. An adaptation of the neural model to allow transfer of parameters to a 784-500-500-10 layer spiking DBN was described in [START_REF] O'connor | [END_REF] with good performance on the MNIST digit database. DBN architecture has been implemented on a Xilinx Spartan-6 LX150 FPGA [37] with very promising classification performance results (92%) on the same MNIST database.

This FPGA implementation of the DBN (also called Minitaur) contains 32 parallel cores and 128 MB of DDR2 as main memory.

Spiking Neuron Model

The neuron is a dynamic element and processing unit that emits output pulses whenever the excitation exceeds some threshold. The resulting sequence of pulses or "spikes" contains all the information that is transmitted from one neuron to the other one. In this section, we compare the biological, artificial and spiking neuron and furthermore, we explain various model of spiking neuron models.

Biological, Artificial and Spiking Neuron

A biological neuron is an electrically excitable cell that processes and transmits infor-

u j = i 1 w ji x i (1)
y j = f (u j + b j ) (2)
It is obvious in Fig. 4b that x 1 , x 2 , x 3 , …, x i are neuron inputs, w ji is the synaptic weights between neuron j and neuron i, b j is bias, f is known as activation function or transfer function and y j is output of the neuron. Based on the model and application of neural networks, there are several types of activation functions such as threshold or step function, linear function, and Non-linear (Sigmoid) function.

Here to be able to Understand how neural network works we explain the functionality of neuron using threshold function. Respecting to the input connections in Fig. 4b, we can define a threshold for transfer function f by defining threshold θ . Here, we choose θ = 0 in the way we could perform a binary classification.

y j = 1 if u j ≥ 0 0 if u j < 0 (3)
where u j is the induced local field of the neuron; which is,

u j = i 1 w ji x i + b j (4)
Such a model of neuron is referred to McCulloch and Pitts [20].

Spiking Neuron

The Spiking neural model is an electrical model of physiological neuron that can be implemented on the circuit using traditional devices or state-of-the-art technologies e.g., CMOS transistors or on hardware platforms e.g., FPGAs. In Spiking model the neurons communicate using spikes and the input spikes make an action potential firing if inside a neuron reaches to the desired threshold (can be compared to threshold activation function in the artificial model of the neuron). Different models of the spiking neuron are proposed that here we study the main models.

Hodgkin-Huxley Model

The first electrical model and in other words the first spiking model of neuron is

Hodgkin-Huxley neuron model [38] The equivalent circuit of Hodgkin-Hulxey model is shown in the left side of Fig. 5 that by representing the Krichhoffs law and using this circuit we can write following equations:

I L (t) = V C (t) -E rest R L (5) I syn (t) = C dV C (t) dt + V C (t) -E rest R L (6)
Solving the Eq. 6 leads to an exponential answer (Eq. 7) that can model the behavior of membrane potential.

V C (t) = v ∞ (1 -exp(- t τ ) + E rest ( 7

U N C O R R E C T E D P R O O F

Respecting to the synaptic current charging if there is enough input current to membrane the neuron will fire. We note that τ = RC in Eq. 7 is the time constant for charging and discharging the membrane.

Integrate-and-Fire (I&F) Neurons

Integrate-and-Fire (I&F) neuron model are derived from the Hodgkin-Huxley neuron model. There is an important type of I&F neuron model which is named Leaky-Integrate-and-Fire (LIF). There are other types of I&F models such as Quadratic-Integrate-and-Fire (QIF). The Leaky-Integrate-and-Fire (LIF) neuron model is a wellstudied model of the neuron. There are three reasons for using LIF in our platform.

• The fabricated model with recent CMOS technology is available [39,40].

• LIF works effectively in spiking and event-based networks [START_REF]Pulsed Neural Networks[END_REF].

• LIF models are quite fast to simulate, and particularly attractive for large-scale network simulations [START_REF] Brette | [END_REF].

Neurons integrate the spike inputs from other neurons they are connected to. These input spikes change the internal potential of the neuron, it is known as neuron's membrane potential or state variable. When this membrane potential passes a threshold voltage due to integrated inputs, the action potential occurs, in other words, the neuron fires. The model is described by the neuron membrane potential:

τ n dv dt = -v(t) + R I syn (t) ( 8
)
I syn (t) = j g i j n α(t -t (n) j ) (9)
where, v(t) represents the membrane potential at time t, τ n = RC is the membrane time constant and R is the membrane resistance. Equation 8 describes a simple parallel resistor-capacitor (RC) circuit where the leakage term is due to the resistor and the integration of I syn (t) is due to the capacitor. The total input current, I syn (t), is generated by the activity of pre-synaptic neurons. In fact, each pre-synaptic spike generates a post-synaptic current pulse. The total input current injected to a neuron is the sum over all current pulses which is calculated in Eq. 9. Time t (n) j represents the time of the n th spike of post-synaptic neuron j, and g i j is the conductance of synaptic efficacy between neuron i and neuron j. Function α(t) = qδ(t), where q is the injected charge to the artificial synapse and δ(t) is the Dirac pulse function. If input voltage passes the threshold, the neuron fires and resets to resting state. The membrane potential stays for a definite period, which is called the refractory period, below the reset value.

Izhikevich Neuron Model

Izhikevich neuron model [43] combines the biological plausibility of Hodgkin-Huxley model and the computational efficiency of integrate-and-fire neurons. Using this model, we can simulate tens of thousands of spiking cortical neurons in real time. The model has two main characteristics it is computationally simple as well as capable of producing rich firing patterns that physiological neuron could produce.

dV (t) dt = 0.04V (t) 2 + 5V (t) + 140 -u(t) + I (t) (10) du(t) dt = a.(b.V (t) -u(t)) (11) if V (t) ≥ 30 mV, then V (t) ← c u(t) ← u(t) + d ( 12
)
where V (

Synapse and Learning

Synapse is a specialized structure with highly plastic characteristics enabling two neurons to exchange spike signals between themselves in other words, adjusting the connection strength between neurons. Thanks to the plasticity property of synapse, we can basically say the synapse is where the learning happens in neural network system. A physiological synapse connects the axon of a presynaptic neuron (the neuron before the synapse) to the dendrite of a postsynaptic neuron (the neuron after the synapse). Two behavioral types of biological synapses are defined:chemical and electrical.

The chemical synapse is the primary definition of neurotransmitters between

Synapse Model

Emerging devices in nano-scale have demonstrated novel properties for making new memories and unconventional processing units. One of those is the memristor that was hypothetically presented by Leon Chua in 1971 [45] and after a few decades, HP was the first to announce the successful memristor fabrication [46]. The unique properties in memristor nano-devices such as, extreme scalability, flexibility because of analog behavior, and ability to remember the last state make the memristor a very promising candidate to apply it as a synapse in Spiking Neural Network (SNN) [47].

In the recent years, there have been several research works using non-volatile resistive nanodevice as a synapse to build a SNN hardware [11,47,48]. Forgetting in the biological brain is an important key of adaptive computation, as without forgetting the biological memory soon becomes overwhelmed by the details of every piece of information ever experienced. Consequently, some studies have been done using volatile memory as a synapse in brain-like computing [49][50][51].

We combine both volatile and non-volatile types of artificial synapses. It leads to make a synapse which can forget if the information is not important as well as remember if it is significant data. Due to the demonstrated potential of NOMFET (Nanoparticle Organic Memory Field-Effect Transistor) [49,50] to play the role of a synapse, we use it as a volatile synapse in the synapse box. The non-volatile device could be any solid-state memristor. We have chose here the resistive memory presented in [52] as non-volatile memory. Resistive RAM is modeled in our previous work [53] and is used here as a nonvolatile memristor in the synapse box. As it is shown in Fig. 8b by changing the doped-undoped regions of device, the conductance will be changed. Bigger doped region leads to more conductivity. Therefore by controlling this boundary between two regions, the conductivity is controlled. The behavior of memristor can be modeled as follows [46]:

v(t) = R m i(t) (13) R m = R ON w(t) D + R OFF 1 - w(t) D ( 14
)
where R m is the variable resistance of memristor, w(t) is the width of the doped region, D is the overall thickness of device, R ON and R OFF are device resistances while the active region is completely doped (w = D) and mostly undoped (w → 0) respectively (Fig. 8b). To model the changing of the conductance, we use the model extracted from Eq. 14 and introduced in [START_REF] Querlioz | IEEE/ACM International Symposium on Nanoscale Architectures[END_REF] by considering g max = 1 R ON and g min = 1 R OFF as the maximum and minimum device conductance respectively.

NOMFET is designed particularly for neuro-inspired computing architectures i ds (t) = g(q np (t), v ds (t), t)v ds [START_REF] O'connor | [END_REF] where g is the conductance of the device, v ds (t) is the applied voltage and q np is the charges trapped in the NP. For more details of physical structure and behavior of NOMFET refer to [50,51].

Figure 8c is the synapse box schematic that we apply in our simulation platform to take the advantages of both nonvolatile and volatile artificial synapses. The equivalent circuit of transistor is depicted in Fig. 8d. Actually, weight modification follows the Short-term potentiation (STP) rule until reaching the Long-term pote potentiation (LTP) threshold in NOMFET. The modification of nonvolatile device is based on STDP learning. Indeed the NOMFET reacts similar to a high-pass filter (HPF). The stimuli spikes with low frequency are not qualified to pass in forgetting Phase. In LTP , stimuli spikes which have more frequency pass to interfere in learning phase (Fig. 6). This synapse box is an approach to improve the quality of synapse for better learning in SNN that have demonstrated better learning performance in SNN rather than nonvolatile memristive synapse [START_REF] Shahsavari | 12th ACM/IEEE International Symposium on Nanoscale Architectures (Nanoarch 2016)[END_REF].

Learning and Plasticity

To be able to model a proper synapse to contribute in learning process in an efficient way in neural system, we need to analyze how learning happens in synapse. Neurons and synapses are the two basic computational units in the brain. The human brain consists of 10 11 neurons and an extremely large number of synapses, 10 15 , which act as a highly complex interconnection network among the neurons.

Subsequently, each neuron is connected to 1000-10000 synapses [START_REF] Drachman | [END_REF]. Neuron computation is performed by integrating the inputs coming from other neurons and producing spikes as based on variety of the connections. The synapses contribute to the computation by modifying their connection strength as a result of neuronal activity, which is known as the synaptic plasticity. This synaptic plasticity is believed as the basis of adaptation and learning, even in traditional neural network models where several synaptic weight updating rules are based on Hebb's law [57,58].

Classes of Learning Algorithms

The primary significance of any type of neural networks is the property of learning from the environment to improve the performance of neural network. There are several types of learning algorithms. Although interconnection configuration of neural network is important in learning however, learning algorithms generally differ from each other in the way in which they adjust synapse weights. Simon Haykin, mentioned five different basic algorithms for learning in his book [START_REF] Haykin | Neural Networks: A Comprehensive Foundation[END_REF] namely memorybased, Hebbian, error-correction, competitive, and Boltzmann learning. Memorybased learning functionality is based on memorizing the training data explicitly.

Hebbian and competitive learning are inspired by neurobiology. Error-correction is working using optimum filtering rule and Boltzmann learning is based on ideas borrowed from statistical mechanics. In general, learning algorithms can be divided into supervised or with teacher learning, semi-supervised learning, and unsupervised or without teacher learning algorithms.

• Supervised algorithms

Teacher has the knowledge of environment and this knowledge will be shared with the network as some examples of inputs and their corresponding outputs. • Semi-supervised algorithms Semi-supervised learning falls between supervised learning and unsupervised learning. Labeled data are often difficult, expensive, and time consuming to obtain, as they require the efforts of experienced human annotators. Meanwhile unlabeled data may be relatively easy to collect. Semi-supervised uses large amount of unlabeled data, together with the labeled data, to build better classifiers. Intuitively, in semi-supervised learning we can consider the learning problem as an exam and labeled data as the few example problems that the teacher solved in the course.

The teacher also provides a set of unsolved problems. Semi-supervised learning requires less human effort and gives higher accuracy, therefore it is of great interest both in theory and in practical application.

• Unsupervised algorithms

There is no teacher and environment is unknown for the network too. There is no labeled data output in unsupervised learning. Unsupervised learning can be thought of as finding patterns in the data above and beyond what is considered as pure unstructured noise. One very simple classic example of unsupervised learning is clustering. Hebbian plasticity is a form of unsupervised learning, which is useful for clustering input data but less appropriate when a desired outcome for the network is known in advance.

Short-Term and Long-Term Plasticity

Physiological synapses have an inherent dynamics, that controls how the pattern of amplitudes of postsynaptic responses depends on the temporal pattern of the incoming spike train. Indeed, each effective spike evokes a spike response in the postsynaptic neuron that is fewer (depression) or bigger (facilitation or potentiation)

than the previous one. The strength of synaptic connections or weights are caused by memorizing events, underling the ability of the brain to memorize. In the biological brain, short-term plasticity refers to a number of phenomena that affect the probability that a presynaptic action potential opens postsynaptic channels and that takes from milliseconds to tens of seconds. Short-term plasticity is achieved through the temporal enhancement of a synaptic connection, which then quickly decays to its initial state. Short-term plasticity depends on the sequence of presynaptic spikes Fig. 9.

In local learning process, iteration of stimulation leads to a more stable change in the connection to achieve long-term plasticity. Long-term plasticity is sensitive to the presynaptic firing rate over a time scale of tens or hundreds of seconds [59].

In general, synapses can exhibit potentiation and depression over a variety of time scales, and multiple components of short-or long-term plasticity. Thus, four combination are possible from short and long term plasticity: Short-term potentiation (STP), short-term depression (STD), Long-term potantiation (LTP) and long-term depression (LTD) [START_REF] Dayan | Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems[END_REF]. where W(x) is called a STDP learning function. Based on Zhang et al. [62] in their experimental work presented W(x) as:

W (x) = A + e ( -x τ + ) if x ≥ 0 -A -e ( x τ - ) if x < 0 ( 17
)
where the parameters A + and A -depend on the current value of the synaptic weight w i j . The time constants τ + and τ -are on the order of 10 ms.

Different Models for STDP Learning

Multiple pre-or postsynaptic spikes occurring across a synapse in an interval of time, the plasticity modification depends on their timing in a more complex manner. For instance, pair-based STDP models present "pre-post-pre" and "post-pre-pos" triplets of spikes with the same pairwise intervals should induce the same plasticity, however experimental studies demonstrated that these two triplet patterns have different effects [63,64].

• Pair-based STDP

In this model of spike counting in the STDP interpret the biological evidence in terms of a pair-based update rule, i.e. the modification of a synaptic weight depends on the temporal difference between pairs of pre-and postsynaptic spikes:

W inc (x) = F inc (w).e (-| t| τ + ) if t > 0 W dec (x) = -F dec (w).e (-| t| τ - ) if t < 0 ( 18
)
In Eq. 18, t = t

dx j (t) dt = - x j (t) τ pre + t pre j δ(t -t pre j ) ( 19
)
where t pre j represents the history of the firing times of the presynaptic neuron. In particular, the variable is increased by an amount of one at the arrival time of a presynaptic spike and reduces exponentially with time constant τ pre afterwards.

Similarly, each spike from postsynaptic neuron i contributes to a trace x i (t):

dx i (t) dt = - x i (t) τ post + t post i δ(t -t post i ) ( 20
)
where t The update of the weight W ji at the moment of a postsynaptic spike is proportional to the momentary value of the trace x j (t) (filled circles). This gives the amount of potentiation due to pre-before-post pairings.

Analogously, the update of W ji on the occurrence of a presynaptic spike is proportional to the momentary value of the trace x i (t) (unfilled circles), which gives the amount of depression due to post-before-pre pairings steady-state distribution of synaptic weights converges to this value regardless of its initial value. The stability of the mean is not a sufficient condition for the steady-state distribution of synaptic strengths to be fully stable, each synapse must also have a stable deviation from the mean. The connection strength of a particular synapse can be presented as w = w + δw, where δw is the deviation of the synapse from the mean. If the deviation is going to grow over time, the synapses will drift away from the mean and the distribution will be partially stable. If the deviation tends to decrease, the synapses will cluster around the mean and the distribution will be stable.

• The triplet model

The standard pair-based STDP models predict that if the repetition frequency is increased, the strength of the depressing interaction becomes greater, leading to less network potentiation. The frequency-dependence of STDP experiments can be accounted for if one assumes that the basic building block of potentiation during STDP experiments is not only a pair-wise interaction but also could be a triplet interaction between two postsynaptic spikes and one presynaptic spike. Pfister and Similarly to pair-based rules, each spike from presynaptic neuron j contributes to a trace x j (t) at the synapse:

dx j (t) dt = - x j (t) τ pre + t pre j δ(t -t pre j ) (21)
where t pre j

presents the firing times of the presynaptic neuron. In contrast with pair-based STDP, each spike from postsynaptic neuron i contributes to a fast trace

x i (t) and a slow trace x i (t) at the synapse:

dx i (t) dt = - x i (t) τ 1 post + t post i δ(t -t post i ) ( 22
)
dx i (t) dt = - x i (t) τ 2 post + t post i δ(t -t post i ) (23)
where τ 1 post < τ 2 post , how the triplet model works is depicted in Fig. 12. In this model, LTD is induced as in the standard STDP pair model in Eq. 18, i.e. the weight change is proportional to the value of the fast postsynaptic trace x i (t) evaluated at the arrival of a presynaptic spike. The new feature of the rule is that LTP is pursued by a triplet effect: the weight change is proportional to the value of the presynaptic trace x j (t) evaluated at the arrival time of a postsynaptic spike as well as to the slow postsynaptic trace x i (t) from previous postsynaptic spike.

The main functional advantage of a triplet STDP rule is that it can be mapped to a Bienenstock-Cooper-Munro learning rule [66]. It means if we assume that the pre-and postsynaptic spike trains are managed by Poisson statistics, the triplet rule presents depression for low postsynaptic firing rates and potentiation for high postsynaptic firing rates.

• Suppression model

Plasticity experiments using triplets of spikes demonstrated different effects than the hippocampal results. In the synapses of the visual cortex of rats, pre-post-pre triplets induce potentiation while post-pre-post triplets induce depression. These results led Froemke et al. [67] to develop the suppression model, in which STDP is induced by nearest neighbor pre-and postsynaptic spikes. In this model of STDP, the plasticity is computed from the standard pair-based STDP curve, however the impact of the presynaptic spike in each pair is suppressed by previous presynaptic spikes and, similarly, the plasticity induced by the postsynaptic spike in each pair is suppressed by previous postsynaptic spikes as it is shown in Fig. 13. The suppression is maximal after each pre-or postsynaptic spike, and it decreases exponentially as the interval between consecutive pre-or postsynaptic spike increases. In a post-pre-post sequence of spikes, the timing of the first post-pre pairing was the best predictor for the synaptic weight modification. Moreover, in a pre-post-pre sequence of spikes, the first pre-post pair induces potentiation, nevertheless the amount of depression induced by the second post-pre pair is suppressed by the first presynaptic spike. In the suppression STDP model, synaptic weight modification is presented by

w = (1 -e - t pre τ pre )(1 -e - t post τ post ) × A inc .e (-t τ inc ) if t ≥ 0 -A dec .e ( t τ dec ) if t < 0 ( 24
)
where t pre is the interval between the presynaptic spike in the pair and its preceding presynaptic spike, and t post is the interval between the postsynaptic spike and its preceding spike. This model introduces a proper fit to triplet and quadruplet protocols particularly in the visual cortex, and also represents a much better prediction for synaptic changing due to natural spike trains [67]. Nonetheless, it does not predict the increase of LTP with the repetition frequency. This model introduce two second messengers called "up" and "down" messengers, which cause to potentiation and depression, respectively which can be in active or inactive states. The arrival of presynaptic spike causes to a fraction of the inactive down messengers a transition to the active state. Similarly, when a postsynaptic spike arrives in the synapse, it shifts a portion of the inactive up messengers into their active state. The messengers go back to their inactive states when there is no spike. Subsequently, when a presynaptic spike arrives, the synapse is depressed proportionally to the value of active down messenger, provided that this is greater than a threshold θ dn . Similarly, each postsynaptic spike leads synapse to potentiate proportionally to the amount of active up messenger provided that it is greater than a threshold θ up . Therefore, the presynaptic spike has three roles in this model: it transmits resting NMDARs into the up state, it activates the down messenger, and it induces depression. The postsynaptic spike also plays three roles: it movement resting NMDARs into the down state, it activates the up messenger, as well as it induces potentiation see Fig. 14.

Shortly, the specific property of the NMDAR-based learning model compared to the pair-based model is the possibility of a stable synaptic distribution and anti-Hebbian competition when the maximum depression is significantly larger than the maximum potentiation.

• Other methods

In addition to the reviewed methods above, there are other types of STDP models for learning such as supervised [70] and reinforcement learning [71]. However, due to the unsupervised nature of STDP learning that is interesting for neuro- Such systems should be power-efficient. Thus researchers have developed braininspired architectures such as spiking neural networks. For large scale brain-like computing on neuromorphic hardware, there are four approaches:

1. Microprocessor based approaches where the system can read the codes to execute and model the behavior of neural systems and cognitive computation such as the SpiNNaker machine [12].

2. Fully digital custom circuits where the neural system components are modeled in circuit using state-of-the-art CMOS technology e.g., IBM TrueNorth machine [11].

3. Analog/digital mixed-signal systems that model the behavior of biological neural systems, e.g. the NeuroGrid [17] and BrainScales [72] projects.

4. Memristor crossbar array based systems where the analog behavior of the memristors emulate the synapses of a spiking neural network.

In the following, we give some details about these approaches and compare their performance.

SpiNNaker is a massively parallel and processor-based (ARM processor) system with the purpose of building large scale spiking neural networks simulations. It is highly scalable and capable to simulate a network from thousands to millions of neurons with varying degree of connectivity. It proposes to integrate 57,600 custom VLSI chips based on the AER (Address Event Representation) communication protocol [73]. Each chip contains 18 fixed-point advanced RISC ARM968 process- The BrainScales project (Brain-inspired multiscale computation in neuromorphic hybrid systems) is the successor of FACETS [START_REF] Wendt | Proceedings of the 2nd WSEAS International Conference on Computer Engineering and Applications[END_REF] project. This project proposes the design and implementation of a custom analog/digital mixed-signal simulation engine that is able to implement the differential equations with an acceptable accuracy. This computational neuroscience model is provided by neuro-scientists, and reproduces the results obtained from numerical simulations executed on conventional computers. The Heidelberg University BrainScales project (HICANN chip) aims to produce a wafer-scale neural simulation platform, in which each 8 inch silicon wafer integrates 50 × 106 plastic synapses and 200,000 biologically realistic neuron circuits (see Fig. 15c). In order to have a scalable size with maximum number of processors on the wafer, relatively small capacitors have been applied for model- An alternative to these architectures, that has been proposed by several authors [47-49, 75, 76], is to use memristive devices as synapses in neuromorphic circuits. This has the potential to lower the energy consumption by a large proportion. It has also been showed that the memristors can emulate the STDP learning rule, and thus lead to unsupervised learning circuits. We have thus chosen to study this kind of architecture and, in particular, to check how some parameters of the architecture or of the devices influence the learning capabilities of the circuit.

Discussion

Still for a network simulation and implementation of neuromorphic spiking system, we need more techniques such as homeostasis and lateral inhibition to support learning process for an optimized system. Homeostasis is used in the SNN to adapt the threshold level of neurons to learning in SNN. Another consideration is lateral inhibition while we are using unsupervised learning methods such as STDP. Here we discuss Winner Take-All (WTA) method.

Homeostasis

Homeostasis addresses a general principle that safeguards the stability of natural and AQ3 artificial neural systems, where stability is understood in its more classical sense of robustness against external perturbations. Homeostasis is a fundamental concept in neuropsychology, psychophysiology and neuroscience. Homeostasis will be defined as negative feedback control. In physiological neural systems, the synaptic input of a neuron is changing over time due to the external neural drive and learning results of synaptic plasticity. From a perspective of metabolic cost, a restricted activity range of a neuron is really meaningful.

In STDP learning, the synaptic input of a neuron may strongly increase or decrease for a long time and as a result the neural activity will be drifting to an extremely high or low level. Homeostasis is a neuron property that regulates the firing threshold to prevent a neuron to be hyperactive [START_REF] Marder | [END_REF] dV th dx = γ ( f r meanf r target ) (25) where f r mean is the mean activity (or firing rate) of a neuron, f r target is the target activity, and γ is a multiplicative positive constant. Consequently, the activity of the neuron is bounded in a homeostatic range to encode the synaptic input more effectively to improve the STDP learning [78].

Winner-Take-All

In a winner-take-all (WTA) network, in output layer or partially output layers, neurons compete with each other based on their output activities, which leads to an adaptation only of the weights of the neuron with the highest output activity [79].

In unsupervised learning using spike coding and plasticity learning. Without competition, all the neurons would behave alike and no specialization takes place in the neurons. The theoretical analysis shows that winner-take-all is a surprisingly powerful computational method compared with threshold gate (McCulloch-Pitts neuron) and sigmoidal gate [80]. There have been many implementations of winner take all (WTA) computations in recurrent networks in the literature [81,[START_REF] Oster | Proceedings of the 2004 11th IEEE International Conference on Electronics, Circuits and Systems[END_REF]. Also there have been many analog VLSI implementations of these circuit [START_REF] Oster | Proceedings of the 2004 11th IEEE International Conference on Electronics, Circuits and Systems[END_REF][START_REF] Hafliger | [END_REF]. In WTA, after the competition, only one neuron will be the most active for some inputs and the rest of the neurons will eventually become inactive for those inputs. Physiologically plausible learning methods can be mainly classified as dense, local, or sparse. Competitive learning such as WTA is a local learning rule as it activates only the unit that fits the input pattern best and suppresses the others through fixed inhibitory connections.

The simplest competitive computational model is a hard WTA that computes a function f W T A :R n → {0, 1} n whose output b 1 , . . . , b n = f W T A (x 1 , . . . , x n ) satisfies b i = 1 if x i > x j for all j = i 0 if x i < x j for some j = i The sequential order of pre-or postsynaptic spikes occurring across a synapse in an interval of time leads to defining different STDP methods. Based on the importance of stability as well as Hebbian competition or anti-Hebbian competition the method will be used in weight modification. We surveyed the most significant projects that cause making neuromorphic platform. The advantages and disadvantages of each neuromorphic platform have been introduced in this chapter.

U N C O R R E C T E D P R O O F

  :28/2/2019 Pages: 38 Layout: T1-Standard

  ity, SNNs provide powerful tools to emulate data processing in the brain, including neural information processing, plasticity and learning. Consequently, spiking networks offer solutions to a broad range of specific problems in applied engineering image detection, event detection, classification, speech recognition and many cognitive computation domain applications. 435574_1_En_25_Chapter TYPESET DISK LE CP Disp.:28/2/2019 Pages: 38 Layout: T1-Standard U N C O R R E C T E D P R O O F

Fig. 2 Fig. 3

 23 Fig. 2 Spike information coding strategies a Rate coding. b Latency coding. c Phase coding. d Rank-coding (spike-order coding). e Population coding. f Sparse coding (a) (b)

Fig. 4

 4 Fig.[START_REF] Mead | Proc. IEEE[END_REF] The structure of a neuron a Physiological neuron. b Artificial neuron model

Fig. 5

 5 Fig. 5 Electrical circuit represents Hodgkin-Haxley model of the neuron. a Details circuit model of the neuron with sodium and potassium channels effects and leakage current. b Equivalent circuit for more simplicity in solving equations

  ) 435574_1_En_25_Chapter TYPESET DISK LE CP Disp.:28/2/2019 Pages: 38 Layout: T1-Standard

Fig. 6

 6 Fig. 6 Simulation of a single LIF neuron in Matlab, the input spikes are applied in t = [10, 30, 40, 50] ms. Between 10 and 30 there is more decrease than between 30 and 40

  t) and u(t) are variables without any dimension, and a, b, c, and d are parameters without dimension. V (t) represents the membrane potential of the neuron and u(t) represents a membrane recovery variable, which accounts for the activation of K + ionic currents and inactivation of N a + ionic currents, and it provides negative feedback to V (t). Synaptic currents or injected dc-currents are delivered via the variable I (t). The parameter a describes the time scale of the recovery variable 435574_1_En_25_Chapter TYPESET DISK LE CP Disp.:28/2/2019 Pages: 38 Layout: T1-Standard U N C O R R E C T E D P R O O F u(t). Smaller value results in slower recovery. The parameter b presents the sensitivity of the recovery variable u(t) to the subthreshold fluctuations of the membrane potential V (t). Greater values couple V (t) and u(t) more strongly resulting in possible subthreshold oscillations and low-threshold spiking dynamics. The parameter c represents the after-spike reset value of the membrane potential V (t) caused by the fast high-threshold K + conductances. Finally, the parameter d describes afterspike reset of the recovery variable u(t) caused by slow high-threshold N a + and K + conductance. Different firing behaviors can occur in biological spiking neurons and Izhikevich model can produce them is shown in Fig. 7.

Fig. 7

 7 Fig. 7 Different Known types of neurons correspond to different values of the parameters a, b, c, and d could be reproduced by Izhikevich model From[43]

[ 50 ]Fig. 8

 508 Fig. 8 Artificial synapse: a schematic view of the NOMFET as a volatile memory, b TiO 2 based nonvolatile memory, c synapse box schematic, d equivalent circuit with simple elements

  :28/2/2019 Pages: 38 Layout: T1-Standard

The supervision is continued

  letting a modification rule adjust the synapses until the desired computation emerges as a consequence of the training process. Then the supervision process is stopped and network must have the similar outputs with the specific inputs while the supervision was working. Error-correction algorithms which include the back-propagation using gradient descent is an example of supervised algorithms, other well-known supervised algorithms are support vector machines (SVM) and Bayesian type of learning algorithms. In fact, we put label on the data in training and check those labels in testing. This type of algorithms are used for regression and classifications. 435574_1_En_25_Chapter TYPESET DISK LE CP Disp.:28/2/2019 Pages: 38 Layout: T1-Standard

Fig. 9 Fig. 10

 910 Fig.9 Implementation of plasticity by local variables which each spike contributes to a trace x(t). The update of the trace depends on the sequence of presynaptic spikes

  the temporal difference between the post-and the presynaptic spikes, and F inc (w)/F dec (w) presents the dependence of the update on the current synaptic weight. A pair-based model is fully specified by defining the form of F inc (w)/F dec (w) as well as determining which pairs are taken into account to perform a new modification. A pair-based weight modification rule can be implemented using two local variables: one for a low-pass filtered version of the presynaptic spike train and another one for the postsynaptic spike train as it is shown in Fig. 11. Let us suppose that each spike from presynaptic neuron j contributes to a trace x j (t) at the synapse weight then we can write:

Fig. 11

 11 Fig.11Pair-based STDP using local variables. The spikes of presynaptic neuron j leave a trace x j (t) and the spikes of the postsynaptic neuron i leave a trace x i (t). The update of the weight W ji at the moment of a postsynaptic spike is proportional to the momentary value of the trace x j (t) (filled circles). This gives the amount of potentiation due to pre-before-post pairings. Analogously, the update of W ji on the occurrence of a presynaptic spike is proportional to the momentary value of the trace x i (t) (unfilled circles), which gives the amount of depression due to post-before-pre pairings

Gerstner [ 65 ]

 65 to propose the triplet model, which takes into account interactions of spikes beyond pre-post pairings. This model is based on sets of three spikes, one presynaptic and two postsynaptic. For a pre-post-pre triplet, the first presynaptic spike enforces extra depression on the synapse, additionally for a post-pre-post 435574_1_En_25_Chapter TYPESET DISK LE CP Disp.:28/2/2019 Pages: 38 Layout: T1-Standard U N C O R R E C T E D P R O O F triplet the first postsynaptic spike enforces extra potentiation. The triplet model sums the contributions of all previous pre-and postsynaptic spikes as well as all pre-post pairings. Pfister and Gerstner [65] also provided a version of the triplet model based only on nearest neighboring spikes, but the qualitative behavior of both all to all and nearest neighboring versions is similar.

Fig. 13 •

 13 Fig.13 The suppression STDP model. a Spike interactions in the suppression model, in which the impact of the presynaptic spike in a pair is suppressed by a previous presynaptic spike (top), and the impact of the postsynaptic spike is suppressed by a previous postsynaptic spike (bottom). b Plasticity in the suppression model induced by triplets of spikes: pre-post-pre triplets induce potentiation (top left), and post-pre-post triplets induce depression (bottom right), From[64]

Fig. 14

 14 scheme and delay partition. Choosing each category should be made consciously and take into account the relevant available experimental findings. The recent available evidences shows that both potentiation and depression are dependent on the weight. Accordingly it is recommended to begin with very simplified models.Moreover, we know that STDP models which assume some weight dependence generate different behavior from the additive model. The pair-based and triplet models are partially stable and use Hebbian competition. The Suppression and NMDAR-based have more stability but they use anti-Hebbian competition. The main challenge in this domain is to perform analytical and simulation studies that are able to identify and characterize their composite effects, and investigate their functional consequences.

Fig. 15

 15 Fig. 15 Large scale spiking neural network systems, a Principal architectural parts of a SpiNNaker processing node. b In TrueNorth, conceptual blueprint of an architecture like the brain, tightly integrates memory, computation, and communication in distributed modules that operate in parallel and communicate via an event-driven platform. c Schematic of HICANN board in BrainScales project. d The chip comprises a 256 × 256 array of neuron elements, an asynchronous digital transmitter for sending the events generated by the neurons, a receiver block for accepting events from other sources, a router block for communicating packets among chips, and a memory blocks for supporting different network configurations

  . The idea is to use an adaptive threshold for the membrane potential. If the neuron is too much active in a short time window the 435574_1_En_25_Chapter TYPESET DISK LE CP Disp.:28/2/2019 Pages: 38 Layout: T1-StandardU N C O R R E C T E D P R O O Fthreshold grows gradually; likewise, when a neuron is not active in a certain time window the threshold is reduced slightly.

( 26 ) 7 Conclusion

 267 Therefore in the case of inputs x 1 , . . . , x n a single output b i has values 1 that marks the position of the biggest input x i . Wolfgang Maass[80] introduced two types of WTA namely k-WTA and soft-WTA. In k-WTA, b i has value 1 if and only if x i is among the k largest inputs. In soft-WTA the i th output is an analog variable r i whose value reflects the rank of x i among the input variables. We use WTA in our research that will be presented in the next sections. 435574_1_En_25_Chapter TYPESET DISK LE CP Disp.:28/2/2019 Pages: 38 Layout: T1-Standard Neuromorphic computation using Spiking Neural Networks (SNN) is proposed as an alternative solution for future of computation to conquer the memory bottelneck issue in recent computer architecture. Different spike codings have been discussed to improve data transferring and data processing in neuro-inspired computation paradigms. Choosing the appropriate neural network topology could result in better performance of computation, recognition and classification. The model of the neuron is another important factor to design and implement SNN systems. The speed of simulation and implementation, ability of integration to the other elements of the network, and suitability for scalable networks are the factors to select a neuron model. The learning algorithms are significant consideration to train the neural network for weight modification. Improving learning in neuromorphic architecture is feasible by improving the quality of artificial synapse as well as learning algorithm such as STDP. In this chapter we proposed a new synapse box that can remember and forget. Furthermore, as the most frequent used unsupervised method for network training in SNN is STDP, we analyzed and reviewed the various methods of STDP.

• Convolutional Neural Networks (CNN) Convolutional

network is a multi-layer feed-forward network architecture in which neurons in one layer receive inputs 435574_1_En_25_Chapter TYPESET DISK LE CP Disp.:28/2/2019 Pages: 38 Layout: T1-Standard

  Standarding cores next to the custom routing infrastructure circuits which is dedicated 96 kB of local memory besides 128 MB of shared Dynamic Random Access Memory (DRAM) as it is depicted in Fig.15a. The router memory consists of a three-state 1024 × 32 bits Content Addressable Memory (CAM) and a 1024 × 24 bits Random Access Memory (RAM). Going more to the details, each ARM core has a local 32 kB

	instruction memory and 64 kB data memory. Regarding to the architecture and design
	properties, SpiNNaker offers very fast simulation of large scale neural networks. It
	has a remarkable flexibility for arbitrary connectivity for network architecture and
	various neurons, synapses and learning algorithms. However, the system still uses
	von Neumann architecture with a large extent of memory hierarchies found in con-
	ventional computers with memory wall bottleneck issues. Although using low-power
	ARM processors dedicated to power-efficient platforms used in training and robotic
	applications with four to 48 nodes, SpiNNaker consumes a relatively small amount
	of power. However, the largest machine with the ability to simulate of one percent of
	a human brain and incorporating over a million ARM processor cores, still requires
	up to 75 kW of electrical power.

435574_1_En_25_Chapter TYPESET DISK LE CP Disp.:28/2/2019 Pages: 38 Layout: T1-IBM designed a scalable, flexible and non-von Neumann full custom spiking neural network named "TrueNorth". Although TrueNorth uses transistors as digital gates, they use event-driven method to communicate in fully asynchronous manner. The structure of TrueNorth consists of 5.4 billion transistors to build 4096 neurosynaptic cores. Each core includes 256 digital LIF neurons, 256 × 256 binary programmable synapses, and asynchronous encoding/decoding and routing circuits. Each synapse has a binary behavior that can be individually turned on or off and can be assigned to model one type of inhibitory and two types of excitatory synapse with different weights. Neuron dynamics has a global 1 kHz clock and so is discretized into 1 ms time steps. Regarding to the synaptic matrix, each neuron can be connected to one up to 256 neurons of a destination core. The routing in TrueNorth is less flexible than in SpiNNaker, however TrueNorth can distribute the system memory includes core synaptic matrix and routing table entries (Fig. 15b) The architecture thus supports dynamics of connectivity that includes feed-forward, recurrent, and lateral connections. The power consumption is 20 mW/cm 2 , though the traditional central processing unit (CPU) is 50-100 W/cm 2 . In this platform the synapses do not implement any plasticity mechanism, therefore they are not able to perform on-line learning.

Author Proof

U N C O R R E C T E D P R O O F