D Querlioz
email: damien.querlioz@u-psud.fr

W S Zhao

P Dollfus

J.-O Klein

O Bichler

C Gamrat

Bioinspired Networks with Nanoscale Memristive Devices that Combine the Unsupervised and Supervised Learning Approaches

Keywords: memristive devices, unsupervised learning, supervised learning, learning architecture, device variations, synapses, neuroinspired systems I

This work proposes two learning architectures based on memristive nanodevices. First, we present an unsupervised architecture that is capable of discerning characteristic features in unlabeled inputs. The memristive nanodevices are used as synapses and learn thanks to simple voltage pulses which implement a simplified "Spike Timing Dependent Plasticity" rule. With system simulation, the efficiency of this scheme is evidenced in terms of recognition rate on the textbook case of character recognition. Simulations also show its extreme robustness to device variations. Second, we present a supervised architecture that can learn if the classification of every input is given. Simulations prove its efficiency. A good robustness to device variation is seen, but not to the level of the unsupervised approach. Finally, we show that both approaches can be combined, with variation robustness higher than in the supervised case. This opens important prospects, like the possibility to first train the system in an unsupervised way with unlabeled data, while still benefiting of the simplicity to program a supervised system.

INTRODUCTION

In recent years, the implementation of neuroinspired systems that exploit nanoelectronics has become extremely popular and has led to the emergence of major projects [START_REF] Versace | The brain of a new machine[END_REF], [START_REF] Modha | Cognitive computing[END_REF]. Indeed, different nanoscale devices (like "memristive" devices [START_REF] Strukov | The missing memristor found[END_REF][START_REF] Agnus | Two-Terminal Carbon Nanotube Programmable Devices for Adaptive Architectures[END_REF][START_REF] Jo | Nanoscale Memristor Device as Synapse in Neuromorphic Systems[END_REF]) could implement functions analogous to the biological synapses (the connections between neurons in the brain). Since the lack of a compact synapse has largely hindered the development of neuroinspired electronic systems until now [START_REF] Arthur | Learning in silicon: Timing is everything[END_REF], this raises the hope of a groundbreaking, smarter and efficient electronics. This is all the more expected, since neuroinspired approaches may not suffer from the variation issue of most nanoscale technologies [START_REF] Agnus | Two-Terminal Carbon Nanotube Programmable Devices for Adaptive Architectures[END_REF], [START_REF] Jo | CMOS Compatible Nanoscale Nonvolatile Resistance Switching Memory[END_REF], [START_REF] Borghetti | Memristive' switches enable `stateful' logic operations via material implication[END_REF].

In all this research, for using nanodevices as synapses, two competitive paradigms have emerged: the ones based on the unsupervised and supervised approaches, respectively. Currently, the prevalent approach consists in the imitation of a learning rule found in Biological synapses, Spike Timing Dependent Plasticity (STDP) [START_REF] Bi | Synaptic modification by correlated activity: Hebb's Postulate Revisited[END_REF], either faithfully [START_REF] Snider | Spike-timing-dependent learning in memristive nanodevices[END_REF], [START_REF] Jo | Nanoscale Memristor Device as Synapse in Neuromorphic Systems[END_REF], [START_REF] Kuzum | Nanoelectronic Programmable Synapses Based on Phase Change Materials for Brain-Inspired Computing[END_REF], [START_REF] Yu | An Electronic Synapse Device Based on Metal Oxide Resistive Switching Memory for Neuromorphic Computation[END_REF] or under a simplified form [START_REF] Querlioz | Simulation of a memristorbased spiking neural network immune to device variations[END_REF], [START_REF] Suri | Phase change memory as synapse for ultra-dense neuromorphic systems: Application to complex visual pattern extraction[END_REF]. These approaches work under the unsupervised learning paradigm: some data used for training are presented to the network, which learns naturally to identify correlations or specific features among them, without any need for a supervisor. Because of the biological inspiration, this raises the hope of providing braintype intelligence and efficiency to electronic system. However, how to actually unleash this potential and program such networks is still an open question, although some simulations have already suggested impressive real life applications [START_REF] Bichler | Extraction of temporally correlated features from dynamic vision sensors with spike-timing-dependent plasticity[END_REF].

On the other hand, schemes imitating the classical supervised neural networks have been proposed [START_REF] Lee | Defect-tolerant nanoelectronic pattern classifiers[END_REF][START_REF] Liao | Design and Modeling of a Neuro-Inspired Learning Circuit Using Nanotube-Based Memory Devices[END_REF][START_REF] Chabi | Robust neural logick block (NLB) based on memristor crossbar array[END_REF][START_REF] Gorchetchnikov | Review and unification of learning framework in Cog Ex Machina platform for memristive neuromorphic hardware[END_REF]. Such networks offer the benefit of being straightforward and natural to program. Their capabilities are also well understood thanks to several decades of research in the neural network field [START_REF] Haykin | Neural Networks: A Comprehensive Foundation[END_REF]. However, they are not as flexible and adapted to process natural and unexpected data as bioinspired unsupervised networks could be. They indeed require well identified (or labeled) data for training.

These considerations suggest how it may be useful to combine both approaches to benefit from the best of both worlds. In this work, we exploit our special purpose simulator ("Xnet") [START_REF] Querlioz | Simulation of a memristorbased spiking neural network immune to device variations[END_REF], [START_REF] Bichler | Extraction of temporally correlated features from dynamic vision sensors with spike-timing-dependent plasticity[END_REF], to propose and validate an architecture based on nanoscale synapses that use both supervised and unsupervised paradigms and possess the advantages of both approaches.

A serious issue hindering the development of nanoelectronics is the device variation (or device mismatch) issue, which affects all memristive nanodevices particularly. Though progress towards technological maturity will improve the situation, it is expected to be intrinsic to ultra-small sizes that make the devices particularly sensitive to any variation in the fabrication process. This raises a challenge to the design of circuits and systems able to exploit their new functionalities. In this work, special attention is given to this issue, and we evaluate the robustness of our proposed strategies to device variations in detail.

The article is organized as follows. First, we introduce the unsupervised scheme. Then we assess its performance on a textbook case (character recognition) and its robustness to variation. A similar work is then performed on the supervised scheme. Finally, the combination of both schemes into a single system is studied.

II. UNSUPERVISED LEARNING SCHEME

In an unsupervised learning scheme, training data is presented to the system that must learn to establish correlations and to "interpret" the data, without any information on what is being presented. To demonstrate such functionality, we build on the simplified STDP scheme introduced in [START_REF] Querlioz | Simulation of a memristorbased spiking neural network immune to device variations[END_REF] that can be implemented with simple voltage pulses. For implementation, it requires bipolar memristive nanodevices with intermediate resistive states like in [START_REF] Jo | Nanoscale Memristor Device as Synapse in Neuromorphic Systems[END_REF], which are used as synapses. As a replacement, it may also exploit phase change memories associated in "2-PCM" circuits as evidenced experimentally in [START_REF] Suri | Phase change memory as synapse for ultra-dense neuromorphic systems: Application to complex visual pattern extraction[END_REF] (which has the advantage of good technological maturity).

A. Basic Principle

In this architecture, CMOS input and output "neurons" are connected by the memristive nanodevices (that act as "synapses"). Their conductance plays the role of a synaptic "weight". It is natural to lay out the nanodevices in the classical crossbar as illustrated on Figure 1 (CMOS silicon neurons and their associated synaptic driving circuitry are the dots, the squares being the nanodevices). The synapses indeed act as adaptive resistors. With the crossbar layout, if several synapses are active at the same time (i.e. receiving voltage spikes), the output receives directly the sum of the currents flowing through the synapses. In a more futuristic design, the system could also be laid out in a CMOL architecture where nanodevices crossbar is fabricated on top of the CMOS neurons and driving circuits [START_REF] Lee | Defect-tolerant nanoelectronic pattern classifiers[END_REF].

This kind of connectivity corresponds to a "feed-forward" architecture in the neural networks community. Of particular interest, in the case of our network, it avoids the "sneak path" issue because both programming and reading are performed in parallel. This issue usually limits the competitiveness of crossbars [START_REF] Liang | Cross-Point Memory Array Without Cell Selectors-Device Characteristics and Data Storage Pattern Dependencies[END_REF] and requires complex counter-measures like complementary resistive switches [START_REF] Linn | Complementary resistive switches for passive nanocrossbar memories[END_REF].

The input neurons present the input (or stimuli in neural terms) as asynchronous voltage spikes using several possible coding schemes described in Figure 2 (spiking rate is proportional to stimulus intensity). All stimuli encoding schemes lead to identical performance of the final system in terms of recognition rate. These stimuli may originate for example directly from a spiking retina [START_REF] Lichtsteiner | A 128× 128 120 dB 15 μs Latency Asynchronous Temporal Contrast Vision Sensor[END_REF] or cochlea [START_REF] Chan | AER EAR: A Matched Silicon Cochlea Pair With Address Event Representation Interface[END_REF] commonly designed in the neuromorphic community that naturally present data as asynchronous spikes, similarly to biological retina or cochlea.

As a result of learning, the output neurons will become selective to the different stimuli classes that are presented in a fully unsupervised manner: the output neurons will develop selectivity to specific features contained in the input patterns.

The CMOS output neurons are Leaky Integrate and Fire type [START_REF] Querlioz | Simulation of a memristorbased spiking neural network immune to device variations[END_REF]. They may be implemented with CMOS analog "neuromorphic" circuits [START_REF] Arthur | Learning in silicon: Timing is everything[END_REF], or digital circuitry [START_REF] Merolla | A digital neurosynaptic core using embedded crossbar memory with 45pJ per spike in 45nm[END_REF]. Additionally, the system features "lateral inhibition" and "homeostatis", which were described in [START_REF] Querlioz | Simulation of a memristorbased spiking neural network immune to device variations[END_REF].

B. Simplified STDP Rule

The learning rule of the memristive nanodevices needs to be fully local, so that it can be implementable in the crossbar architecture. It consists in a "simplified Spike Timing Dependent Plasticity" rule, illustrated in Figure 3, and which works as follows. When an output neuron spikes:  If input neurons spiked within a given time window preceding the output's spike, the conductance of the synapses connecting these input neurons and the output neuron are increased by a step G+  In any other case, the conductance of the synapses connecting the input neurons to the output neuron are decreased by a step G-As seen in Figure 3, this constitutes an extreme simplification of the biological STDP rule, which is believed to be a foundation of learning by the brain [START_REF] Bi | Synaptic modification by correlated activity: Hebb's Postulate Revisited[END_REF]. When an input neuron spikes, it applies an input pulse to the nanodevices to which it is connected. When an output neuron spikes it applies an output pulse to the nanodevices to which it is connected. When the voltage applied on the device (difference between the voltages applied at the two ends) reaches a memristive device threshold (VT+ or VT-), its conductance is increased by a quantity G+, or decreased by G-, respectively Figure 4 shows a set of simple voltage pulses which may implement this learning rule. They have the advantage of extreme simplicity. When an input neuron spikes, it applies a long voltage pulse that is sufficient to drive current into the memristive devices, but not enough to change their conductance. When an output neuron spikes, it applies a specific waveform. If an input neuron connected to the output neuron is not active, this waveform causes the conductance of the memristive device connecting the two neurons to decrease. If the input is still active, the difference of the two voltages applied on the memristive device causes its conductance to increase.

The disadvantage of the pulses of Figure 4 is the long input pulse which can be power hungry with some device technology. Figure 5 shows an alternative to these pulses, which is demanding lower power [START_REF] Suri | Phase change memory as synapse for ultra-dense neuromorphic systems: Application to complex visual pattern extraction[END_REF] for the cost of limited added complexity. The input neuron does not apply a voltage pulse during whole time it is active, but only at the beginning to drive current into the output neuron, and when a signal is sent back to it when one of the output neuron becomes active.

…if Output spikes while Input is still active For device modeling, we use the model introduced and experimentally verified in [START_REF] Querlioz | Learning with memristive devices: how should we model their behavior?[END_REF] for memristive devices used as analog memories. An increase in the conductance is modeled by the equation:

V T+ Voltage Time Input Neuron active
min max min GG GG Ge         (1)
Similarly, a decrease is modeled by: max max min

GG GG Ge         (2)
Gmin and Gmax are the minimum and maximum conductances of the nanodevices. + and _ characterize the conductance step induced by each voltage pulse. + and are fitting parameters. The choice of all these parameters is inspired by the experimental measurements of [START_REF] Jo | Nanoscale Memristor Device as Synapse in Neuromorphic Systems[END_REF] and is listed in Table I. For demonstration of the concept, in this paper we use the widely studied case of handwritten digits recognition. The MNIST database is used, which consists in 60,000 handwritten digits of 28x28 pixels by 250 writers [START_REF] Lecun | Gradient-based learning applied to document recognition[END_REF]. Although this may not be the final application for our system, this constitutes the most studied problem in machine learning and is a mandatory step to evaluate our approach. In this paper, all simulations are system-level and are based on a C++ special purpose code ("Xnet") [START_REF] Querlioz | Simulation of a memristorbased spiking neural network immune to device variations[END_REF], [START_REF] Bichler | Extraction of temporally correlated features from dynamic vision sensors with spike-timing-dependent plasticity[END_REF]. The code is eventbased for simulation performance and runs on conventional CPUs. It simulates the equations of the network, using the voltage pulses shown in Figures 4 or 5 In order to achieve learning, we present the full MNIST training database (60,000 digits) three times to the circuit. Each input neuron is connected with one pixel of the image. It emits spikes with a jittered rate that is proportional to the pixel intensity (Figure 2(b)). The initial phase is random. No kind of preprocessing on the digits is used and the set is not augmented with distortions. The network is then tested on the MNIST test database (10,000 digits), which consists in digits that have not been presented during training.

C. Results

Figure 6 plots the conductances learned by the system in a configuration with only 10 output neurons. It is remarkable that without any supervision and using only our local custom STDP rule, the system has identified 9 (out of 10) different numbers, the real features of the input. Moreover it has learned the distinctive features of the digits (and not just the most likely handwriting): it has learnt the loop of the digit two, the bottom of the six, or the horizontal parts of three and eight.

To evaluate the performance of the system, we can define a recognition rate. For that purpose, we associate output neurons with the digit for which they spike the most frequently a posteriori. In hardware this association could be performed with complex digital circuitry, and constitutes a difficulty. A satisfactory approach is presented in section IV of this paper, with the association of a supervised neural network.

A running average of the recognition rate during learning is shown in Figure 7(a). Whichever the number of output neurons, the network learns rapidly during the first 10,000 digit presentation and then keeps improving its performance gradually. The final performance is given as the mean recognition rate on the test database. It is actually higher than the performance on the training set seen in Figure 7(a), and is plotted in Figure 7(b). With ten output neurons the recognition rate reaches 60 %. For better performance, more output neurons are needed. With 50 output neurons the recognition rate reaches 81% and with 300 output neurons 93.5 %. A traditional artificial neural network with back-propagation with 300 hidden neurons (that has the same number of adjustable parameters) reaches 95 % [START_REF] Lecun | Gradient-based learning applied to document recognition[END_REF], which compares to our rate of 93.5 %. In the literature, the best algorithm has a largely superior 99.7 % recognition rate, but has 12 million adjustable parameters (vs. 235,200 here) and uses a largely augmented training set [START_REF] Ciresan | Deep Big Simple Neural Nets Excel on Handwritten Digit Recognition[END_REF]. Though our numbers are more modest, the interest here is that the network is fully unsupervised with extremely simple local learning rules.

The two variations of voltages pulses (Figures 4 and5) gave identical performance in terms of recognition rate. All the results presented in the paper use the pulses of Figure 4.

Important, thanks to the unsupervised nature of learning, the complicated "labeling" step does not need to be performed right away. Lots of unknown data may be presented for training, and the labeling can be performed from a limited subset of well identified data. This is a strong advantage: for many problems involving natural data. Often, we have a lot of data, but that were not classified, and the system can here perform this analysis by itself.

D. Extreme robustness to device variations

One of the most important features of the unsupervised learning scheme is its extreme robustness to device variation. For every memristive devices, we considered variations in the parameters   ,   of equations ( 1) and ( 2), the minimum conductance Gmin, and the maximum conductance Gmax. To this purpose, we performed Monte Carlo simulations of our system, with all the parameters of all devices computed from a Gaussian probability law. Every simulation was repeated five times, and the average recognition rate is given. Figure 8 plots the degradation of the recognition rate, when device variation is increased. We see that the network is immune to variations up to 25% of the mean value of all these parameters, which constitutes an extremely high level of variation for an electron device (but typical for research nanodevices [START_REF] Jo | CMOS Compatible Nanoscale Nonvolatile Resistance Switching Memory[END_REF], [START_REF] Borghetti | Memristive' switches enable `stateful' logic operations via material implication[END_REF]). With 50% of variation, there is a small decrease of the recognition rate. With a dramatic variability of 100% on the synaptic parameters, the recognition rate decreases significantly, but interestingly the functionality of the network is not challenged. We should note that with a variability of 50%, 4% of the nanodevices cannot be programmed in at least one direction (i.e. they have an  value of 0). The latter figure becomes 30% when parameter variability is 100%. This shows the overall tolerance to defects. Figure 8 was computed for a system with 50 outputs. The results were similar for a system with 300 outputs (which appeared slightly more robust). This degree of robustness to device variation is exceptional in electronic system and constitutes one of the strongest points of the approach. It takes root in the unsupervised nature of learning: output neurons learn features for which they are naturally "fit". The variability is thus not a problem but a germ for learning. The extreme simplicity of the voltages pulses is also fundamental to ensure the robustness. Relative std deviation of device parameters (%) Figure 8. Robustness to devcie variation for the three schemes (unsupervised, supervised, combined). Degradation of the system's recognition rate as a function of device variability. The variable parameters are   ,   , Gmin, Gmax.

III. SUPERVISED LEARNING SCHEME

A. Basic Principle

Using the same kind of nanodevices, a similar scheme may be developed to implement supervised learning, while retaining some variation insensitivity.

It is comparable to the unsupervised scheme and also uses the general architecture of Figure 1, but there is no voltage applied on the nanodevices when an output neuron spikes. The system works as follow. The input neurons present a digit during a given time, using one of the coding scheme of Figure 2. After this time period, input and output neurons apply a programming pulse on the nanodevices as illustrated on Figure 9. The input neurons apply a pulse only if they spiked during the period. The output neurons apply a voltage pulse depending on their situation, which can only program the nanodevices in conjunction with a waveform on the input neuron end.

If the output neuron was expected to spike (it corresponds to the digit actually presented), the connections to the input neurons that spiked should be strengthened. The output neuron applies a negative spike that will strengthen the positive part of pre-synaptic pulses, and increase the conductances of nanodevices that present such pulses. If the output neuron was not expected to spike, inversely it should apply a positive pulse to weaken the synapses connected to input neurons that spiked. If the output neuron actually spiked (i.e. made a mistake) a pulse is applied to significantly reduce the conductances. If it did not spike (and thus was right)which is the most typical situationthe pulse is lowered in order to only weakly weaken the conductances (and not cause the conductances to decrease too rapidly).

B. Results

A first test is to apply the scheme directly to the MNIST database. The parameters of the simulation are listed in Table I. The network performance on the testing set is 87.5%. The perceptron, a standard supervised neural network with similar complexity, has a performance of 88% [START_REF] Lecun | Gradient-based learning applied to document recognition[END_REF].

If the input neuron spiked it applies….

If the output neuron was supposed to spike… -but spiked -and did not spike If the output neuron was not supposed to spike… Thanks to the pass/fail approach a good variation to variability is achieved, although it is not as good as in the unsupervised scheme of section 3. With 10% variability on   ,   , Gmin, Gmax performance (averaged on 5 runs) is reduced to 86.2%. With 25% variability it is reduced to 83.2%, and with 50% variability to 75.7%. This constitites impressive robustness, but is actually not to the level of the unsupervised approach, as shown in Figure 8. This scheme is not as robust as the previous one, because the system cannot decide which output neuron will learn which feature (this is imposed by the supervisor). Some memristive devices (connecting important pixels of the input to an output neuron) can thus play an important role. For relative standard variation of the characteristics above 50%, some devices become effectively nonfunctional, and in particular some of these important devices, causing a substantial decrease of the recognition rate.

V input Time Time V output Time V output Time V output V T+ V T-
If we compare with the unsupervised approach, the supervised one has the clear advantage of requiring less neurons and synapses, and to be simple to use. By contrast, it does not have its level of robustness to device variation. More important, it requires having every data used for training labeled since the system does not learn the distinguishable features itself as in unsupervised scheme, but based on the expected output.

IV. COMBINATION

We have proposed two schemes for designing learningcapable architectures with nanodevices. They are in fact compatible. To illustrate this we propose a two-crossbar architecture, illustrated in Figure 10. Every output of the unsupervised crossbar is fed as an input spike to the supervised crossbar. The role of the supervised layer is to label neurons trained by the unsupervised first layer of section II. It associates spikes on the output of the first layer with the actual digits presented. As illustrated on Figure 11, this works very well in practice. Until iteration 60,000 the supervised layer's performance remains low, and then it matches the performance of the unsupervised layer, as calculated in section II. It can even become slightly become higher because the supervised layer is more adaptable than the majority rule used in section II.

Of particular interest, the problem solved by the supervised crossbar is simpler than by that of section III. This makes it less sensitive to device variations, as seen in Figure 8. Even with variations of   ,   , Gmin, Gmax of 25% the supervised layer performance is matching perfectly the unsupervised layer very well. Starting with variations of 50%, the degradation of the recognition rate is slightly worse, but still better than in purely supervised problem of section III.

Globally, the supervision does not intrinsically improve the recognition rate, but provides a practical and flexible approach for the labeling operation. This opens up many opportunities. One ideal way to exploit the system is to first train the unlabeled layer on natural data, and then to perform a "labeling" on a small training set by activating the supervised layer. This works very well in the case of our MNIST dataset and identical recognition rate is obtained as with training the supervised layer with all the data when the supervised layer is trained with as little as 2,000 digits. This could be a useful way to process all kinds of natural data. We may collect tons of video, auditory and olfactory data and feed it to our adaptable electronic system. It would learn distinguishable features thanks to the unsupervised layer. Then a quick training with well identified data of the supervised layer can provide the system's functionality. With conductances in normalized units (Gmax=1):

  = 10 -2 ,   = 5.10 -3 , min w = 10 -4 , max w = 1, + = 3.0, -= 3.0, the input pulses are 25 ms long, the output pulses 0.5 ms (as inspired by [START_REF] Jo | Nanoscale Memristor Device as Synapse in Neuromorphic Systems[END_REF]). Neurons parameters are taken from [START_REF] Querlioz | Simulation of a memristorbased spiking neural network immune to device variations[END_REF]. When a variability of X%, is applied on a parameter, it means that the standard deviation of the parameter is X% of its mean value. Device variability is always applied to Gmax, Gmin,   ,  

Supervised Crossbar

Same as unsupervised. For the weak conductance reduction   = 5.10 -4

V. CONCLUSION AND PERSPECTIVES

We have presented two different learning approaches to exploit memristive nanodevices as synapses in neuroinspired systems. The STDP-based unsupervised approach can learn based on unlabeled data and has extreme tolerance to device variation (relative standard deviation of 50% may be tolerated on all significant device parameters). The supervised approach is simple to program but needs labeled data, and is not as tolerant. Both approaches can be combined, which opens the way of first training the system on a large dataset, and then of training the supervised layer on a small labeled subset to identify what has been learnt. In this situation, an excellent robustness to device variation is achieved.

Such works highlight the credibility of the new computing paradigm associating neuroinspired systems and nanoelectronics, to provider "smarter" electronics, especially for embedded applications where the ability to process natural data efficiently and in real time can be particularly useful. Future works will focus on experimental demonstration beyond single device that we have already achieved [START_REF] Suri | Phase change memory as synapse for ultra-dense neuromorphic systems: Application to complex visual pattern extraction[END_REF] and new applications like video [START_REF] Bichler | Extraction of temporally correlated features from dynamic vision sensors with spike-timing-dependent plasticity[END_REF], auditory or olfactory data processing or the inference in complex databases. More complex networks, with several unsupervised layers will also be considered.

Figure 1 .Figure 2 .

 12 Figure 1. General architecture for the unsupervised case

Figure 4 .

 4 Figure 4. Pulses for simplified STDP (voltage pulses as a function of time).When an input neuron spikes, it applies an input pulse to the nanodevices to which it is connected. When an output neuron spikes it applies an output pulse to the nanodevices to which it is connected. When the voltage applied on the device (difference between the voltages applied at the two ends) reaches a memristive device threshold (VT+ or VT-), its conductance is increased by a

Figure 5 .

 5 Figure 5. Low power variationof the pulses of Figure 4.

Figure 6 .

 6 Figure 6. Conductances learned in a simulation with 10 output neurons. Red is maximum cinductance (Gmax), blue is minimum conductance (Gmin).

  .

Figure 7 .

 7 Figure 7. (a) Recognition rate during learning for simulations with different numbers of output neurons (from bottom to top: 10, 50, 100, 300). The recognition rates are running averages on 10,000 iterations. (b) Recognition rate on the test dataset as a function of the numer of output neurons

  to the rate with no variation(%)

Figure 9 .

 9 Figure 9. Voltage pulses used for supervised learning (voltage pulses as a of time).

Figure 10 .

 10 Figure 10. Architecture combining unsupervised and supervised crossbar

Figure 11 .

 11 Figure 11. Two layer learning. Learning rate of the second (labeling) layer trained by variability compatible supervision (red) compared with learning of the first layer (black), as computed in the previous section. Variability of 10% on all parameters (initial conductances,   ,   ,Gmin, Gmax)

TABLE I

 I

	.	PARAMETERS USED IN THE SIMULATIONS
	Unsupervised
	Crossbar

ACKNOWLEDGMENT

The authors would like to thank S. J. Thorpe, P. Dollfus, M. Suri and B. DeSalvo for fruitful collaboration.

This work was supported by the European Union through the FP7 Project NABAB (Contract FP7-216777) and CNRS/INSIS (PEPS Synapse).


C Ghfiri

A Durier

C Marot

A Boyer

S Ben Dhia

Modeling the internal activity of an FPGA for conducted emission prediction purpose

Keywords: Integrated circuits, Internal Activity, Conducted emission, ICEM, FPGA, Dynamic power

End-users of integrated circuits need models to anticipate and solve conducted emission issues at board level in a short time. This is the purpose of the standard IEC62433-2 ICEM-CE. Although it proposes methods to extract circuit models from measurements, they cannot provide activity dependent models and may lead to inaccurate results for large and complex circuits. This paper describes a new methodology of construction of the internal activity block of an ICEM-CE model adapted to large digital integrated circuits and validated on a FPGA. The method is based on a predictive approach using the estimation tools of the dynamic power and the data path delays proposed by the manufacturer of the integrated circuit.

I. INTRODUCTION

The evolution of integrated circuits and their complexity through the increase of the number of transistors and the miniaturization of electronic components has a huge effect on the current density and the increase of parasitic emissions. Thus, there is a crucial need to construct EMC models to predict the noise generated by integrated circuits (IC). Existing standards propose EMC models for prediction of conducted emissions of IC, such as ICEM-CE (Integrated Circuit Emission Model -Conducted Emissions) known as IEC 62433-2 standard [START_REF]EMC IC modelling -Part 2: Models of integrated circuits for EMI behavioural simulation -Conducted emissions modelling (ICEM-CE) -Edition 1.0[END_REF]. It is particularly adapted to IC end-user's needs about prediction of conducted emission (CE) at PCB level due to the switching of internal logic blocks (core) and IOs of ICs. ICEM-CE model is made of two main parts: the first one describes the switching or internal activity (IA) of the circuit, the second one the filtering effect brought by the die and package interconnects. In ICEM vocabulary, it is known as passive distribution network (PDN). The IA block mainly consists of a set of equivalent current sources that reproduces core and IO switching activity, although IO noise can also be accurately predicted by adding IBIS model to PDN block [START_REF] Ghfiri | Construction d'un modèke ICEM pour prédire l'émission électromagnétique d'un FPGA[END_REF]. The construction of the IA block dedicated for core noise prediction remains a challenging task for IC end-users.

Several methods have been proposed for the construction of the IA block of the core. As suggested by the standard, it can be extracted from an inverse method based on the measurement of the external current flowing in the ground or power supply pins and the modelling of IC, package and PCB interconnects (PDN) [START_REF] Ghfiri | Construction of an Integrated Circuit Emission Model of a FPGA[END_REF]. The main drawback of this measurement-based approach is that the IA extraction result depends on the PDN measurement and is extremely sensitive to any measurement errors. The IA Core block can also be constructed from the simulation of the transistors netlist, the transistor model library and the standard cell models [START_REF] Vrignon | Characterization and Modeling of Parasitic Emission in Deep Submicron CMOS[END_REF][5] [START_REF] Leca | EMI Measurements, Modeling, and Reduction of 32-Bit High-Performance Microcontrollers[END_REF]. However, this approach requires considerable simulation time and relies on unavailable data for IC end-users.

The IA block may be estimated from basic technological information about the technology and IC characteristics (e.g. CMOS technological node, number of logic gates, die surface…), for example with tool like ICEM expert module in IC-EMC software [START_REF] Sicard | Using ICEM model Expert for TC1796 IC-EMC application note[END_REF]. The IA is estimated roughly from the floorplanning definition of the circuit or without exact information about the circuit, but the main drawbacks of this approach are the precision and the lack of consideration of the actual IC configuration made by the end-user (hardware or software).

For some IC families such as programmable hardware device (e.g. Field Programmable Gate Array (FPGA)), design tools provide to the end-users accurate estimation of the average dynamic power estimation from post-placement and routing simulations of a given configuration. Moreover, some advanced FPGA design tools are able to determine the instantaneous dynamic power consumption from a vectorbased simulation, which allows the calculation of the timedomain profile of the dynamic current consumption and thus a precise extraction of IA core block [START_REF] Liehui Ren | Prediction of Power Supply Noise From Switching Activity in an FPGA[END_REF]. However, this type of approach requires long simulation times, whose results are dependent on the input test vectors. To overcome this issue, we propose to estimate the IA block of an FPGA from vectorless power estimation and static timing analysis applied on post-placed and routed design. Compared to the vectorbased approach, this solution is able to provide a good estimation of the IA block in a very short time, without additional steps in the FPGA design flow, and from a reduced set of data. Moreover, the proposed approach is able to take into account the random activity of the circuit. The proposed method can be applied by any FPGA end-user. Furthermore, it could be adapted to any circuit by extracting the IA parameters by measurements.

In this paper, the methodology of construction of the IA block of an FPGA (XC6SLX16-FT256 Spartan 6 from Xilinx) is presented, and validated through several case studies. Its accuracy is evaluated through comparison between measurements and simulations of the CE. The first chapter is devoted to the presentation of the methodology of construction of the IA core block. Secondly, the case studies are exposed. Finally, the last chapter is dedicated to the validation of the methodology through the different case studies.

II. INTERNAL ACTIVITY CONSTRUCTION PRINCIPLE

The CE of a synchronous digital circuit is related to the instantaneous current consumption, i.e. the charge transfer from the power supply to internal logical gates synchronized by a clock signal. A straightforward approach to model the source of IC CE relies on one or several current sources connected to an equivalent model of the power distribution network of the circuit that filters the noise produced by the circuit switching activity. This is the approach proposed by ICEM standard to model the CE produced by an IC: the equivalent current sources belong to a block called Internal Activity (IA) while the power delivery network forms the block Passive Distribution Network (PDN). The relevance of the model depends on:

• The accuracy of the PDN block. This block is supposed independent of the instantaneous activity of the circuit. Numerous publications present reliable methods to extract a model of a circuit PDN so that we consider that the PDN is known in the following paragraphs. • The accuracy of the IA block. Multiple variable parameters affect this block, such as the clock frequency, the power supply voltage or the internal activity of the circuit.

Before presenting our methodology of extraction of the IA block for FPGA circuit, a preliminary question is: is it necessary to determine precisely the time-domain waveform of the instantaneous current consumption to obtain an acceptable CE prediction? In the next paragraphs, we will show that it is not necessary and only two criteria must be respected to obtain a reliable prediction of CE.

A. Influence of the instantaneus current consumption waveform on emission spectrum

In digital synchronous circuits, current consumption appears mainly during transitions from logical state 0 to 1. Thus, the time domain profile of the instantaneous current is a periodic series of short pulses; its period is equal to the clock period Tc. As explained in [START_REF] Liehui Ren | Prediction of Power Supply Noise From Switching Activity in an FPGA[END_REF], the power supply noise can be predicted if the charge consumed at each clock transition is determined correctly. The waveform of the current i(t) delivered by the IA block must reproduce this charge quantity, given by:

( ) ∫ = τ 0 dt t i q (1)
Where q is the transferred charge during a clock cycle and τ is the duration of the charge transfer or the current pulse. Moreover, to reproduce the CE spectrum, the duration τ of the charge transfer must be also known. In order to illustrate this principle, let consider the three basic periodic pulsed waveforms presented in Fig. 1, which model the dynamic current consumption of a digital circuit. The clock frequency is 10 MHz. The pulse durations are identical; their amplitudes are chosen such as the transferred charge is identical. These current waveforms are generated by a current source and form an IA block which is connected to the PDN block representative of a large digital IC. The voltage across a 1 Ω resistor connected to the V ss pins of the circuit is simulated for each waveform. Fig. 2 presents a comparison of the simulated voltages using the different waveforms in frequency domain. In spite of some differences in high frequency (around the nulls of the spectral envelop), the envelopes of the spectra are very close, providing nearly identical estimation of the CE noise. Whatever the waveform of the instantaneous current consumption, if the transferred charge and transfer duration are respected, an acceptable estimation of the CE can be obtained. This is the purpose of our methodology.

B. Presentation of the IA construction methodology for an FPGA circuit

The charge quantity transferred at each clock cycle can be determined from measurements or the evaluation of the average dynamic power consumption P dyn_avg by a design tool. It is given by (2):

DD L N i C avg dyn V i q i F P ⋅ = ∑ = ) ( ). ( 1 _ α (2)
where N is the number of internal logical nodes or signals in the circuit, α(i) is the average number of transitions on signal i per clock cycle (toggle rate), q L (i) is the quantity of the charge associated to signal i, V DD is the power supply voltage and F C is the clock frequency of the circuit. For the Spartan 6 family of Xilinx, the manufacturer puts in the ISE software the module Xilinx Power Analyzer (XPA) which gives a detailed estimation of the power consumption in the clock tree, the signals and logic blocks, and the IOs after a post-placement and routing simulation. XPA also gives a vectorless estimation of the toggle rates and average dynamic power of all the internal logic signals. From these estimated parameters, the average quantity of charge transferred during a clock cycle Q L can be calculated. For each clock period T C , the internal activity produces the same quantity of the charge as given by the formula [START_REF] Ghfiri | Construction of an Integrated Circuit Emission Model of a FPGA[END_REF], where I avg is the average dynamic current consumption.

( ) ( )

C avg N i L L T I i q i Q ⋅ = = ∑ =1 α (3)
ISE Xilinx tool performs a static timing analysis which gives a detailed report about the slack and the data path delays at the fast and slow PVT (Process/Voltage/Temperature) corners. For the construction of the IA, the pulse duration τ is considered to be the median value of the interval whose boundaries are the maximum data path τ max and the minimum data path τ min given by the static timing report [START_REF] Vrignon | Characterization and Modeling of Parasitic Emission in Deep Submicron CMOS[END_REF]. The average dynamic current associated to a periodic pulsed waveform i(t) is calculated using (5).

dt t i T I C avg ⋅ ⋅ = ∫ τ 0 ) ( 1 (5)
In the next paragraphs, we consider the dynamic current consumption waveform is a symmetrical triangular pulse. The peak amplitude of the pulse I max is given by [START_REF] Leca | EMI Measurements, Modeling, and Reduction of 32-Bit High-Performance Microcontrollers[END_REF].

τ C avg T I I . 2 max ⋅ = (6)
By solving the equations ( 3), ( 5), and ( 6) the maximum amplitude of the IA is calculated using the formula [START_REF] Sicard | Using ICEM model Expert for TC1796 IC-EMC application note[END_REF]. Fig. 3 summarizes the steps to get the required data from an FPGA design tool to solve the equation [START_REF] Sicard | Using ICEM model Expert for TC1796 IC-EMC application note[END_REF], which leads to the IA construction. The proposed workflow of construction of the IA is based on the average dynamic power consumption, hence, the approach proposed is deterministic; the amplitude of the IA is constant at each clock cycle. In practical cases, the IA profile is random due to the variation of the toggle rate and the power consumption of the active logic blocks. Thus, the peak-to-peak amplitude of the IA in the time domain could be under estimated and the envelope of the emission spectrum inaccurate. In the next part, an alternative approach is proposed for the construction of a random IA.

C. Statistical method for construction of a random IA

The construction of a random IA block is possible with a vector based simulation [START_REF] Liehui Ren | Prediction of Power Supply Noise From Switching Activity in an FPGA[END_REF] but this may require long simulation time. However, the power analysis tool XPA gives the switching probability of each logic block of the circuit. The variability of the switching activity around its mean value can be estimated in a short time simulation. For the construction of a random IA, a methodology based on the statistical distribution of the transferred charge per clock cycle is proposed.

XPA tool gives a detailed report about the dynamic power consumption and the toggle rate for the routed signals and the logic blocks. With these data, it is possible to calculate the probability density function (pdf) of switching of each logic block of the circuit and thus the pdf of the transferred charge to each logic block per clock cycle. Although the logic states of each logic block are partially dependent, we neglect these interdependencies since they cannot be determined through the vectorless estimation of XPA. Considering that each logic block of the circuit switches independently from the others, the pdf of the transferred charge to the circuit is the convolution product of the pdf of the transferred charge of each logic block and the toggle rate. Statistical parameters such as the average and standard deviation can be calculated, and the most representative statistical law can be determined. In practice, due to the large number of switching logic nodes, the distribution tends to be Gaussian. Under this assumption, the probability that a logic block i transfers a charge quantity q(i) per clock cycle is given by the toggle rate α(i). Hence, the average transferred charge quantity ) (i q for each logic block is calculated using the equation [START_REF] Liehui Ren | Prediction of Power Supply Noise From Switching Activity in an FPGA[END_REF]. Considering the charge quantity q(i) as a random variable, the expected variance 2 i σ is given by the formula (9), allowing the calculation of the pdf of each logic block i, where M is the number of logic states, for a binary case M=2.

( ) ( )

2 . 1 = ⋅ = = ∑ = M if q j p j q q i i M j i i i α (8) ( ) ( ) ( ) ∑ = = ⋅ - = - ⋅ = M j i i i i i i M if q q j q j p 1 2 2 2 2 1 ) ( α α σ ( 9
)
The pdf of the transferred charge to the circuit follows the normal distribution N(Q L ,σ²), where Q L is the average transferred charge quantity (10), σ² its variance (11) and N the number of logic blocks of the circuit.

∑ = = N i i q Q 1 (10) ∑ = = N i i 1 2 2 σ σ (11)
Finally, in order to generate a SPICE compatible IA source, a series of random charge quantity Q(k) is generated at each clock cycle according to the previous pdf. Considering a symmetric triangular pulse, the amplitude I max (k) of the IA is given by the expression (12), where k is the number of generated points. The IA is then modelled using a current source based on PWL file. Fig. 4 summarizes the approach of construction of the IA using the statistical distribution.

( ) τ ) ( 2 max k Q k I ⋅ = ( 12
)
Power Analyzer Equations ( 7), ( 8), ( 9)

: Mean quantity of charge of logic block i α(i) : Toggle rate of logic block i (i) : Dynamic power of logic block i

Pdyn

) (i q σ 2 , Q L N random value of from Q L       σ 2 , Q N L

N random value of Imax

III. PRESENTATION OF THE CASE STUDIES

A. The circuit under test

The circuit under test is a XC6SLX16-FT256 Spartan 6 Xilinx FPGA, manufactured with a CMOS 45-nm process. The circuit includes 9152 configurable logic blocks (CLB) and up to 186 user IOs, it is mounted in a Fine pitch Thin Ball Grid Array 256 balls (FTBGA256). The internal structure of this component is complex; it comprises six different power supply planes: VCCINT dedicated to the CLB (1.2 V), VCCOx (x = 1 to 4) dedicated to the I/O organized in four banks (3.3 V), and VCCAUX dedicated to the JTAG configuration (3.3 V), and a ground plane for the VSS pins.

B. Description of the test board

A specific test board has been designed for the external voltage measurements and the validation of the constructed Internal Activity. It consists of a six layer board with complete power and ground planes in a 10x10 cm format for radiated emission measurements in TEM cell. Several test points have been placed for CE measurements and characterization of the board impedance. A 1 Ω probe, as defined by the IEC 61967-4 [11], has been placed between the ground pins of the FPGA and the ground reference of the test board in order to measure the return current flowing outside the circuit. Models of the PDN of the FPGA and the test board have been constructed from S parameter measurements and electromagnetic simulations and validated previously [START_REF] Ghfiri | Construction of an Integrated Circuit Emission Model of a FPGA[END_REF].

C. Configurations

The construction of a predictive internal activity of the FPGA is performed for a given activity. The workflows presented in part II are verified for different case studies of basic synchronous digital designs. . The clock frequency is set to 16 MHz for the different case studies. Fig. 5 shows the structure of the first case study: a delay line. For optimum dynamic power consumption, 90 delay lines have been cascaded in series, each delay line has N = 100 inverters. Due to its regular structure, its dynamic power consumption is deterministic; i.e. the number of switching gates remains identical at each clock transition. As another validation case, the 7-bit counter is a good example of synchronous circuit for which the internal signals present different toggle rates. In this case, 800 7-bits counters have been cascaded in parallel. For the different cases, no output buffer is switching, hence, the 1 Ω measurements and simulations shows only the contribution of the internal logic blocks and the clock tree activity of the FPGA. Table 1 summarizes the dynamic power consumption, data path delay and the average toggle rate collected from the estimation tools of Xilinx.

IV. CONSTRUCTION OF THE INTERNAL ACTIVITY BLOCK

A. Case study 1: delay line

The first case study (delay line) has a toggle rate of 100 % because all the signals are switching at the rising edge of the clock. The clock tree presents non-negligible dynamic power consumption, its toggle rate is about 200 %, and hence, the signals representing the clock tree consumes energy at both rising and falling transitions of clock signal. Therefore, the internal activity of the clock tree is separated from the internal activity of the signals and the logic blocks. As shown in Fig. 6, Fig. 8 shows a comparison between the measured and simulated external voltages with a 1 Ω probe in the frequency domain. In general, the comparison presents a good correlation between the measured and the simulated spectrum with a RMS error of 5,15 dB. Simulation predicts that the frequency content of CE falls down above 1.2 GHz. This is confirmed by measurements.

B. Case study 2: 7-bit counter 1) Modelling with the deterministic approach

The case of the 7-bit counter is particularly different from the previous case. The signals representing every bit of the counter have different toggle rates. Firstly, the IA is constructed following the deterministic approach. Fig. 9 presents a comparison between the measured and simulated external voltage using a 1 Ω probe in the time domain. The simulated peak-to-peak amplitude of the voltage fluctuation reaches 13 mV and is underestimated compared to the maximum measured peak-to-peak amplitude of 21 mV because we consider that the instantaneous transferred charge at each clock cycle is constant and equal to the average transferred charge per clock cycle. However, if the activity is random, the instantaneous transferred charge varies around the average value. 2) Modelling with the random approach In this part, a random IA is constructed from the approach based on the statistical distribution for the case of the 7-bit counter. Fig. 11 shows the simulated and the measured external voltages using a 1 Ω probe. The simulated maximum peak-to-peak amplitude of the voltage fluctuation of 22 mV allows the prediction of the maximum peak-to-peak amplitude of 21 mV with a good precision. In general, the simulated time domain profile reproduces the random character of the CE with a good approximation when the statistical method is used to construct the IA. V. CONCLUSION In this paper, a new methodology of construction of an ICEM-CE model of FPGA circuits has been presented. Whatever the internal dynamic current waveform is, the simulated conducted noise will not vary much if the transferred charge and the pulse duration are determined correctly. Also, the dynamic current of an integrated circuit depends mainly on three parameters: the dynamic power consumption, the toggle rate and the clock and data path delay. Using the estimation tools given by the manufacturers of FPGA, these parameters can be extracted rapidly and without extra steps in the FPGA design flow, and allow the modeling of the internal activity following two different approaches: the deterministic approach and the random approach. Different validation cases have been discussed and generally the comparisons between the measured and simulated emission spectrum present a RMS error calculated for the clock harmonics less than 6 dB, and globally the envelope of the spectrum is predicted with a good approximation. This error can be related to the extraction of the passive distribution network block of ICEM-CE model, the assumptions of our proposed methodology and to measurement accuracy. Future works will be carried out to better evaluate the intrinsic error of the proposed method.

The methodology presented in this paper is adapted to FPGA and other integrated circuits when the manufacturer provides power and delay estimation tools to IC end-users. On the other hand, the principle is still adapted to ICs when the manufacturers do not provide estimation tools but the needed parameter should be extracted from the measurement. This methodology can be proposed within the ICEM standard as user guidelines.

Fig. 1 .Fig. 2 .

 12 Fig. 1. Three basic pulsed waveforms to model the dynamic current consumption of a circuit

Fig. 3 .

 3 Fig. 3. Construction workflow of the Internal Activity block for FPGA circuit

Fig. 4 .

 4 Fig. 4. Workflow of the IA construction using the statistical approach

Fig. 5 .

 5 Fig. 5. Structure of a delay line

Fig. 6 .

 6 Fig. 6. Internal Activity of the core blockThe constructed internal activity block is combined to the PDN and the board equivalent model. A transient simulation using ADS is performed to simulate the CE produced by the FPGA and measured through a 1 Ω probe. The time-domain simulation and measurement results are compared in Fig.7. The simulated peak-to-peak amplitude of the voltage fluctuation is equal to 9 mV and presents a good correlation with the measured amplitude equal to 11 mV.

Fig. 7 .

 7 Fig. 7. Comparison between the measured and simulated external voltage using a 1 Ω probe in the time domain generated by the delay line configuration

Fig. 8 .Fig. 9 .

 89 Fig. 8. Comparison between the measured and simulated external voltage using a 1 Ω probe in the frequency domain of the delay line configuration

Fig. 10 .

 10 Fig. 10. Comparison between the measured and simulated external voltage using a 1 Ω probe in the frequency domain of the 7-bit counter using the deterministic approach

Fig. 11 .Fig. 12 .

 1112 Fig. 11. Comparison between the measured and simulated external voltage using a 1 Ω probe in the time domain of the 7-bit counter using the random approach Fig. 12 presents the comparison of the measured and simulated external voltages using a 1 Ω probe in the frequency domain. Overall, the envelope of the simulated CE shows a good correlation with the measured envelop. The RMS error in this case is about 4,86 dB.

TABLE I .

 I ESTIMATED PARAMETERS FOR THE DIFFERENT

CONFIGURATIONS Configuration Dynamic Power (mW) Data path delay (ns) Average toggle rate (%) Logic Clock Logic Clock

	Delay Line	24	1,5	40	0,75	100
	7-bit counter	9,60	1,5	3,8	0,75	28,35

This work is supported by the IRT Saint-Exupéry project Robustesse Electronique sponsored by Airbus Operations, Airbus Group Innovations, Continental Automotive France, Hirex Engineering, Nexio, Safran Electrical & Power, Thales Alenia Space France, Thalès Avionics and the French National Agency for Research (ANR)