Amélie Gruel
email: amelie.gruel@univ-cotedazur.fr

Jean Martinet
email: jean.martinet@univ-cotedazur.fr

Teresa Serrano-Gotarredona

Bernabé Linares-Barranco

Event data downscaling for embedded computer vision

Keywords: Event cameras, Computer vision, Data reduction, Preprocessing, Visualisation

come

INTRODUCTION

Event-based camera are slowly but surely integrating many embedded systems, such as autonomous vehicles and other robotic applications. However, this novel type of asynchronous data comes with a heavy flow of information which can be too much to handle and not always relevant for the task at hand. In order to optimize the information carried by events, we believe that it is necessary to formalize the different methods for event data spatial downscaling, as defined in Fig 1.

Event cameras

The idea of a novel bio-inspired event-based sensor, akin to a "silicon retina" [START_REF] Mahowald | The Silicon Retina[END_REF], has been developed since the 1990s. In bio-inspired retinas, the transmitted information is coded as spikes. Pixels emit spikes whenever relevant information is captured in the visual field. Retinas converting luminance to spike frequency, spatial contrast retinas as well as retinas implementing some spatio-temporal filtering have been proposed. In 2008, Lichtsteiner, Posch and Delbruck presented the first complete design of a Dynamic Vision Sensor (DVS) [START_REF] Lichtsteiner | A 128x128 120 db 15 us latency asynchronous temporal contrast vision sensor[END_REF] responding only to temporal brightness change in a scene with no consideration for colours, similar to the organic retina.

Figure 1: A set of fullscale events captured by a sensor of size width × height is downscaled according to a downscaling factor f actor, set to 2 in this example. Here, all the events of the original sample occurring in the blue area's pixels will correspond to the events occurring in the red pixel after reduction.

The main advantages of such an artificial retina are [START_REF] Gallego | Event-based vision: A survey[END_REF]:

• the high temporal resolution, thanks to which an event can be emitted on the timescale of microseconds and avoiding motion blur,

• the high dynamic range, which makes it possible to use them for extremely dim as well as under bright sun illumination,

• the high contrast range which allows for highly contrasted images avoiding dazzling effect caused by sudden illumination changes, thus scenes with illumination expanding over 120dB can be sensed without suffering saturation,

• the low latency and asynchronicity enabled by the independence between each pixel,

• the absence of redundancy in the information transmitted, as compared to frame-based sensors,

• the low power consumption, following the model of biological retinas and substituting the biological photoreceptors by photodiodes in the electrical circuits. Fig. 2 shows the conceptual schematic of an event camera pixel. A photosensor produces a current I ph proportional to the illumination. A transimpedance logarithmic circuitry converts the detected current into a voltage V log = A T log(I ph ) which depends logarithmically on the photocurrent. This voltage is the input to a differencing capacitive amplifier, so that the voltage difference

∆V di f f = -C 1 /C 2 ∆V log = -C 1 /C 2 A T ∆I ph /I ph (1)
That way voltage V di f f is proportional to the relative temporal variation of the photocurrent. Voltage V di f f is compared with an upper and a lower voltage thresholds. Each time it goes over the upper (or below the lower) threshold, the pixel will generate an output ON (or OFF) event and voltage V di f f is reset to a resting value V R . Fig. 3 illustrates the resulting behavior of the event camera pixel. The upper subfigure plots the variation of a pixel photocurrent along time, while the lower subfigure illustrates the generated ON and OFF output events. As can be observed, each time the photocurrent increases (or decreases) by a certain relative variation given by,

∆I ph /I ph = C 2 (V t -V R )/(C 1 A T ) (2)
the pixel generates an ON (OFF) output event.

Figure 3: Illustration of event-based encoding of visual signals with respect to the predefined threshold ∆I, from [START_REF] Lagorce | Spatiotemporal features for asynchronous event-based data[END_REF]. The luminance captured by the sensor varies over time, and an event is only produced when its log intersects with lines of equation y = ∆I. The corresponding event is positive when the log-luminance's slop is positive at the intersection, and negative otherwise. The interval between each t i and t i+1 is not monotonous, which is consistent with the event data asynchronicity.

The event cameras' particularities described above grant them advantages in many application cases compared to more traditional visual sensors [START_REF] Gallego | Event-based vision: A survey[END_REF]. A significant field of applications is robotics: autonomous vehicles, drones and other embedded systems requiring low latency and low power data handling. Furthermore, they evolve in uncontrolled lightning conditions thus require a high dynamic range and high contrast range.

Because of their operating mode, event based cameras are useful in tasks featuring movements, such as object tracking [START_REF] Glover | Event-driven ball detection and gaze fixation in clutter[END_REF], gesture recognition [START_REF] Amir | A Low Power, Fully Event-Based Gesture Recognition System[END_REF], optical flow estimation [START_REF] Orchard | A spiking neural network architecture for visual motion estimation[END_REF][START_REF] Paredes-Valles | Unsupervised learning of a hierarchical spiking neural network for optical flow estimation: From events to global motion perception[END_REF], Simultaneous Localization and Mapping (SLAM) [START_REF] Kim | Realtime 3d reconstruction and 6-dof tracking with an event camera[END_REF][START_REF] Vidal | Ultimate slam? combining events, images, and imu for robust visual slam in hdr and high-speed scenarios[END_REF], etc. Their high resolution and contrast range also valuable for astronomical studies [START_REF] Cohen | Event-based sensing for space situational awareness[END_REF][START_REF] Chin | Star tracking using an event camera[END_REF]. Research on their combined use with traditional RGB camera becomes more common, with applications on high-resolution image reconstruction [START_REF] Zhang | Image reconstruction from events. why learn it?[END_REF][START_REF] Rebecq | High speed and high dynamic range video with an event camera[END_REF] and video deblurring (Zhang et al., 2021a).

Motivations

While desktop event vision solutions can afford large amount of computing resources, embedded application typically face limitations in CPU, memory, and energy usage. Event data is sparse, and yet in some highly dynamic and textured scenes, the event flow can be very high, up to 25M event per second, which requires significant processing resources to handle.

Most existing event vision approaches preprocess event data by simply downscaling the pixel coordinates, which is actually a max-pooling operation in the event domain. One obvious issue of this simple event funnelling is that the resulting event density drastically increases in the reduced spatial dimension, causing an important information loss.

In embedded systems, only relevant information should be stored and processed under the form of downscaled event data. For example, event-based odometry applied to robotics requires only movement information in order to process the information in real time: this need is typically answered by pooling the events in frames [START_REF] Rebecq | Real-time visual-inertial odometry for event cameras using keyframe-based nonlinear optimization[END_REF], resulting in the loss of the data's asynchronicity.

In neuromorphic machine learning, spiking neural networks usually can't handle the flow of information the event data represents and downscale this input by a first preprocessing layer, before actually learning anything.

We can also mention the residual graph convolutionnal techniques which handles each input event as a node in the graph [START_REF] Bi | Graph-based object classification for neuromorphic vision sensing[END_REF]: this leads to a low computational efficiency and a substantial simulation run, which can be improved solely by spatially reducing the input neuromorphic data.

The aforementioned examples are just a few reasons why event downscaling is a real challenge in neuromorphic computer vision and needs to be formalized.

The contributions of this paper are twofold: we design and implement a number of methods to reduce the size of event data and we assess the proposed reduction methods by analysing reduced data and also based on the classification results.

The remainder of the paper is organized as follows: firstly an introduction to event-based cameras and the motivations behind this work, then a review of previous use of event data, followed by a complete description of the 6 spatial downscaling methods implemented in this work, and finally a a comparison summarising the advantages and disadvantages of each method.

RELATED WORK

Reducing the size of visual data is straightforward for grayscale and RGB images with interpolation. When it comes to event data, size reduction is much less trivial. And yet, many papers discussing neuromorphic computer vision are faced with a substantial event flow, sometimes too great to perform the task at hand. In those case, the authors address this issue with various techniques presented bellow.

A first solution simply consists in creating event frames, then processing this data as standard visual images, as seen in [START_REF] Fang | Incorporating learnable membrane time constant to enhance learning of spiking neural networks[END_REF] and [START_REF] Huang | Event-based timestamp image encoding network for human action recognition and anticipation[END_REF]. This enables the same model to be applied to any king of visual data, neuromorphic or not. However its main drawback is that all the data's asynchronicity, which is a real advantage of event data, is disregarded; all events are gathered and their timestamps discretised.

Spiking neural networks handling event data can consider each pixel as an input neuron, and each event produced as an input spike to the neural model. However this first layer is often too important for efficient computation, thus most authors applying this type of neuromorphic computing on event data resolve to convolve the input. For example, [START_REF] Cordone | Learning from Event Cameras with Sparse Spiking Convolutional Neural Networks[END_REF] apply multiple convolution layers to the dataset DVS128 Gesture before processing it.

The non uniform sampling employed by [START_REF] Bi | Graph-based object classification for neuromorphic vision sensing[END_REF] can also be mentioned, although it only produces satisfying results on a sparse dataset with well-defined empty areas like the one they use. This simply amounts to implement a non-uniform grid sampling inspired from traditional data processing to event data.

To the best of our knowledge, the only existing tool for event data downscaling is Tonic's "Downsample" method [START_REF] Lenz | Tonic: event-based datasets and transformations[END_REF]. Tonic is a Python library for neuromorphic data handling, which has been created on an original idea dating back from the 2019 Telluride neuromorphic workshop. It facilitates dataset downloads and its conversion to different event representations (frames, time surfaces, voxel grids, etc). It also proposes event transformation, such as denoising, event dropping, polarities merging, spatial and temporal jittering and downsampling.

The "Downsample" method simply multiplies the event coordinates by a spatial factor without dropping any event. It can also be applied to temporal downsampling following the same mechanism. Although it is extremely fast, it has the major flaw of keeping all the original events thus leading to an increasingly high density in the produced visual data.

The following subsections present the 7 methods considered in this work to downscale spatially the data produced by an event camera. Each method takes as input a sample of events at maximum resolution as well as the factor by which to reduce this sample, and returns the downscaled events as illustrated in Fig. 1. Depending on the downscaling method used, the number and frequency of the downscaled events will not necessarily equal those of the events occurring in the corresponding fullscale regions.

Simple Event Funnelling

A first solution to reduce event data simply is to divide each event's coordinates in x and y by the downscaling factor, then remove all the reduced events duplicates (i.e., keep only one occurrence of each event for a specific set of coordinates (x, y,t, p) -see Fig. 4). This method follows the same logic as Tonic's downscaling approach and can also be extended to temporal reduction.

Thanks to its quickness and simplicity, this solution could answer the need for downscaling events in real time on embedded systems. However, the main drawback is that this process does not drop any event.

The reduced area undergoes a high density of events, yielding an information loss due to the accumulation of pixel activation (see Fig. 4). Event funnelling tends to drastically increase pixel activity (see Fig. 4), especially when the downscale factor is large. In the extreme yet unlikely case where the data is downscaled to a 1 × 1 size, the resulting pixel is likely to be constantly active.

Finally, the behavior does not correlate with the physical mechanism of an actual event camera: the optical consistency is very low for this method.

Log-luminance Reconstruction

From an optical point of view, downscaling the spatial size of an event recording boils down to averaging the luminance captured by a subset of pixels in the original sensor. In a physical implementation of an event camera formed by a 2D array of pixels as the one shown in Fig. 2, this method could be implemented by averaging the photocurrents I ph generated by neighbor pixels and performing the transimpedance logarithmic conversion and posterior voltage amplification and differentiations as explained in Section. Introduction. In order to implement this behavior, the log-luminance is reconstructed for each pixel in the fullscale dataset: each event occurring on the same pixel increases (if positive) or decreases (if negative) the log-luminance level by one unit. The mean logluminance is then computed for each region of size f actor 2 in the sensor. Lastly, the downscaled events are generated: an event occurs at every timestep where the average log-luminance crosses a threshold line; the event is positive if the gradient is positive, and vice-versa.

To achieve a pixel's log-luminance reconstruction, all events from this pixel is translated into points of coordinates (x show the timestamps, y is the level reached) where the initial level is arbitrarily set to 0 and contrast threshold set to 1 (see Fig. 3).

Event Count

A first approach to log-luminance reconstruction consists in estimating the mean log-luminance reached by a downscaled pixel every time an event occurs in the corresponding fullscale region -in other words, realise a normalised event count in the fullscale region as presented in Fig. 4. If its value crosses a contrast threshold, an event is produced; an additional value keeps track of the slope, in order to set this event's polarity depending on the slope. It is also necessary to handle the constraint posed by the fact that an event is only produced if the entire threshold has been covered since the last event. The slope value allows to answer this by identifying the extrema, thus preventing events to occur when the log-luminance crosses a threshold directly following an extremum. This approach's main disadvantage is the loss of neuromorphic asynchronicity. A reduced event can only occur at the timesteps where a fullscale event took place, which wouldn't be the case with actual log-luminance reconstruction, as depicted in the following paragraphs. This drawback is also an asset: this reduction method can easily downscale events as they are being recorded, and its algorithm simplicity is a plus for an embedded system.

Linear Estimation

The second approach is to actually reconstruct the log-luminance curve as a polyline, where each event in the log-luminance graph is linked to its neighbors in time by a line as presented in Fig. 5a. This allows for a quick computation and interpolation of the inter-sections between the thresholds and the curve. However, it is not as faithful to the physical reality as we would like it to be.

Cubic Estimation

In this final approach, the log-luminance of each neuron is interpolated as a cubic curve strictly running through the events on the log-luminance graph as seen in Fig. 5b. The cubic interpolation was implemented using Scipy's method PChipInterpolator, an implementation of the Piecewise Cubic Hermite Interpolating Polynomial. This algorithm is a third-degree piecewise polynomial function which seeks to match the derivatives of each point with its neighbors. Thus its extrema coincide with the data's extrema, which is not the case of all cubic interpolation algorithm who tend to greatly overshoot [START_REF] Rabbath | A comparison of piecewise cubic hermite interpolating polynomials, cubic splines and piecewise linear functions for the approximation of projectile aerodynamics[END_REF]. As similar tools, this cubic interpolator I takes as input a set of values (x, y, x ′ ) and returns the y ′ data interpolated for all x ′ values with respect to the relation between y and x:

I(x,y,x') = y' with x' = ∑ x max //threshold x=x min //threshold+threshold (x) (3)
In our case, the coordinates are those of the events in the corresponding log-luminance graph: the x-axis correspond to the events' log-luminance and the ordinates to the timestamps. From a set of known log-luminance levels x ′ , the cubic interpolation outputs the timestamps y ′ where the events' logluminance crosses those levels. This approach reconstructs a curve close to an actual log-luminance curve recorded by an event camera, granting a high score of optical consistency to this approach. However, the existing tools allowing the cubic interpolation of y values according to a set of x parameters requires the x set to be strictly monotonous, without any duplicates. The first hinder can be removed by splitting the (x, y) dataset presented in Eq. 3 at the different log-luminance's extrema:

S i (x, y) = ∑ n ext i=0 ∑ t y ext i+1 t=t y ext i (x t , y t ) (4)
with S i the ith subset of the events comprised between two extrema, n ext the total number of extrema in the dataset i.e. the number of segments S i , and t y ext i and t y ext i+1 respectively the timestamps of the i and (i + th extrema.

The second hinder mentioned above is more complex to avoid: although it is a rare occurrence, multiple events still happen at the same coordinates and in the same timestamps with a frequency of X. This requires us to only keep one specimen of duplicates, thus lightly skewing the data processing.

Furthermore, it shares one disadvantage with the linear estimation approach: both those approach cannot be computed strictly in real time since they need to know the future behavior of the curve, that is to say the next events that will occur on the monitored pixel. We can work around this problem by inserting a slight delay between the data recording and processing.

Events to video to events

Since it is possible to reconstruct a grayscale video from an event stream, a last simple technique would be to use existing downscaling methods on a grayscale reconstruction of the event data, then translate this reduced image back into events. This is equivalent to reconstructing a downscaled luminance and computing downscaled events from it, but it has some important drawbacks. The main one lies in the total loss of event data's asynchronicity since the grayscale video produced from the neuromorphic data is built from the pooling of information into frames.

Moreover, its implementation in practice is computationally heavy, which is not suited for embedded applications. Lastly, this method needs two sets of grayscale data to be created during the process (fullscale from events, then downscale) leading to high memory consumption, while embedded systems are limited in data storage.

A pipeline implementing such a downscaling process is the following: 1. translate the event data into grayscale frames thanks to Events-to-Video library introduced in [START_REF] Rebecq | Events-to-video: Bringing modern computer vision to event cameras[END_REF]. This module actually consists in a machine learning model trained on a set of datasets comprising neuromorphic and grayscale recordings of the same scenes. 2. rescale the grayscale frames thus obtained with one of the many existing image processing tools. 3. finally translate the downscaled grayscale frames into events by using the library Video-to-Events from [START_REF] Gehrig | Video to events: Recycling video datasets for event cameras[END_REF]. This library reconstructs the log-luminance variations between each grayscale frame, then outputs the corresponding events according to user-defined negative and positive contrast thresholds. This pipeline requires the event dataset to be downscaled either to be one of the datasets the Eventsto-Video model has been trained on, or to contain a set of grayscale frames recorded simultaneously as the neuromorphic data. Since this method is very arguable considering the motivations for event downscaling, it was not implemented to be compared in this paper. Note that the dataset used in our experiments (see Section Datasets) does not meet either of these two criteria.

EXPERIMENTS

Datasets

The different methods of reduction where applied to the neuromorphic dataset DVS128 Gesture widely used as a benchmark in event data processing, and we assessed the results according to the criteria described in the following sections.

Hand gesture recognition is a skill used daily in the human society and is tightly integrated with verbal communication, hence the need of computational learning of such data. Since it relies heavily low latency, this task is well suited for event-based computation. Building on this notion, Amir et al. presented at CVPR 2017 a complete hand gesture neuromorphic dataset called DVS128 Gesture [START_REF] Amir | A Low Power, Fully Event-Based Gesture Recognition System[END_REF]. To this end, 29 subjects were recorded performing 11 different hand gestures under 3 kinds of illumination conditions, by a DVS128 camera (see Fig 6). A total of about 133 samples are available for each gesture, each composed roughly of 400,000 events and of dimension 128 × 128 pixels, for a duration of 6 seconds approximately. The dataset is split in two sub-datasets to facilitate machine learning training: the train subcategory received 80% of the recorded samples and the test 20%, with an even distribution of the 11 gestures between them.

This dataset was chosen according to its relevance to our motivations: DVS128 Gesture represents dynamic information to be processed temporally, which would be the case in most embedded systems.

Comparison

Table 1 compares between the 6 methods presented above and with the Tonic's "Downsample" tool according to 5 criteria described in the following section.

Number of events

The mean number of events per sample was calculated over the whole DVS28 Gesture dataset at fullscale and downscaled by the 7 methods mentioned in Table 1.

Temporal density Events temporal density D corresponds to the activation probability of pixels averaged over the whole sensor:

D = ∑ w x=0 ∑ h y=0 P x,y w • h (5)
with w and h respectively the width and height of the sensor. The activation probability P x,y is calculated as the number of events (positive or negative) occurring at a given pixel divided by the time length of the sample:

P x,y = ∑ t max t=t min δ(x t , x) • δ(y t , y) t max -t min (6)
with P x,y the activation probability of one pixel of coordinates (x, y), t min and t max respectively the minimum and maximum timestamp of the sample, and δ the Kronecker delta function, which returns 1 if the variables are equal, and 0 otherwise.

Information entropy If we consider the information brought by positive (or negative) events according to Shannon's information theory, we can estimate the entropy H of the dataset as the average over all pixels and polarities p of the entropy H x,y,p :

H = ∑ p∈{-,+} ∑ w x=0 ∑ h y=0 H x,y,p 2 • w • h with H x,y,p =
∑ P e ∈{P x,y,p ,P x,y,p } -P e •log 2 (P e )

(7) where P x,y,p is the probability of event (probability for the pixel of being active), and P x,y,p = 1 -P x,y,p is the dual probability of no event, for the polarity p at location (x, y). H x,y,p becomes lower when the pixel is either never active or always active. Entropy values in Table 4.2 are averaged from both polarities.

Computation time

The computation time of each method presented in this paper was calculated on a panel of over 500,000,000 events. The values presented in the Table 1 correspond to the average calculation period for the downscaling of 100,000 events, expressed in seconds.

It should be noted that the values produced for the SNN Pooling method are not obtained from the similar panel of event data as the other. As a matter of fact, since the computation time for one sample using either technique exceeds 10 min, we chose to assess our criteria on a small set of downsampled events using SNN Pooling.

The values of these criteria confirm the hypotheses we outlined above. Tonic's "Downscale" method has the highest number of events, thus the highest event density; this spatial reduction does not lead to a significant drop of event flow, thus does not answer our need for reduced data to handle. The same goes for the Simple Event Funneling, which does ignore all event duplicates. Even though the computation time of those two methods is low, the output data is too important to be processed more efficiently than the original one. We should also point out that their information entropy is quite low, due to the constant activation of the sensor's pixels.

The log-luminance techniques are the most consistent with the optical reality behind event data recording. It drops a large amount of data, thus significantly improving any embedded computation, while maintaining a relatively high level of information entropy. SNN Pooling is also promising by its behavior closely related to one of an event-camera, but it is extremely sensible to the weight of its connection: here the weight is probably too high hence its poor results. We intend to take advantage of the high computational efficiency offered by SpiNNaker [START_REF] Furber | Overview of the spinnaker system architecture[END_REF] or Loihi [START_REF] Davies | Loihi: A neuromorphic manycore processor with on-chip learning[END_REF] in order to assess the optimal hyperparameters.

Conclusion

Event cameras take an increasing part in current trend in computer vision, with a number of possible applications in robotics and autonomous driving. While standard frame-based data (grayscale and RGB) are easily downscaled -which is a very common preprocessing step, there currently exists no straightforward equivalent for event data.

In this paper, we introduce and compare 7 methods for event data downscaling for embedded computer vision applications. Our results show that the choice of the reduction methods greatly influences the event features of the reduced dataset, namely the number of events, the density, and information entropy. The proposed methods offer a panel of solutions to choose from depending on the target task (classification, detection, tracking, etc.) Our future work include the assessment of the proposed methods on a classical computer vision tasks such as classification, in order to evaluate the accuracy loss brought by the proposed methods. Along this line of research, we intend to develop event reduction using spiking neural networks, a relevant spatial downscaling method for the neuromorphic processing of event based data. This new type of asynchronous artificial neural network is closer to biology than traditional artificial networks, mainly because they seek to mimic the dynamics of neural membrane and action potentials over time. The Leaky Integrate-and-Fire neuron model more specifically simulates the increase of the membrane potential lead by the frequency of input spikes coupled with a constant slow decrease. We believe that this mechanism is well suited to implement an event downscaling method which temporally pools input events. Preliminary results lead us to understand that hyperparametrisation may have a significant influence on this preprocess, which we would like to discuss in the foreseeable future. Besides, one important aspect for real time applications is the ability of the proposed methods to operate in real time, so as to directly reduce the input event flow and save compute, memory and energy in embedded systems. Therefore we will evaluate the real time capability of the methods. A relevant application of the proposed methods is to implement a foveation mechanism, such as in [START_REF] Gruel | Stakes of foveation on event cameras[END_REF]. Finally, it will be useful to extend the spatial downscaling to temporal downscaling to further adjust the density of event flow.

Figure 2 :

 2 Figure 2: Block diagram of an event camera pixel.

Figure 4 :Figure 5 :

 45 Figure4: Schematic illustration for event funnelling and event counts methods. A sensor of size 2 × 2 outputs negative and positive at each pixel across time, which are downscaled using Simple Event Funnelling (red frame on top) and Log-Luminance Event count (green frame at the bottom). In this example, the downscaling factor is set to 2.

Figure 6 :

 6 Figure 6: Events grouped in a frame from a sample of the gesture "left hand clockwise" (class 7) of the DVS128 Gesture dataset. Positive events are depicted in green, negative in blue.

Table 1 :

 1 Features of the original DVS128 Gesture dataset, and of the 5 downscaling methods described in Section 3.

	Methods	Number of Temporal Information Computation	Optical
		events	density	entropy	time consistency
	Original fullscale data	194,398	1.95e -7	2.94e -6	-	-
	Simple Event Funnelling	186,414	2.95e -6	3.65e -5	103.20 ms	*
	Tonic Downscale	194,312	5.40e -5	4.31e -4	3.20 ms	*
	Log-luminance
	Event Count	7,778	1.25e -7	1.88e -6	0.83 s	**
	Linear Estimation	624	8.58e -9	1.56e -7	2.63 s	***
	Cubic Estimation	435	8.92e -9	1.66e -7	2.94 s	****

ACKNOWLEDGEMENTS

This work was supported by the European Union's ERA-NET CHIST-ERA 2018 research and innovation programme under grant agreement ANR-19-CHR3-0008.

The authors are grateful to the OPAL infrastructure from Université Côte d'Azur for providing resources and support.


G De Loubens

A Riegler

B Pigeau

F Lochner

F Boust

K Y Guslienko

H Hurdequint

L W Molenkamp

G Schmidt

† A N Slavin

V S Tiberkevich

N Vukadinovic

O Klein

Bistability of vortex core dynamics in a single perpendicularly magnetized nano-disk

Microwave spectroscopy of individual vortex-state magnetic nano-disks in a perpendicular bias magnetic field, H, is performed using a magnetic resonance force microscope (MRFM). It reveals the splitting induced by H on the gyrotropic frequency of the vortex core rotation related to the existence of the two stable polarities of the core. This splitting enables spectroscopic detection of the core polarity. The bistability extends up to a large negative (antiparallel to the core) value of the bias magnetic field Hr, at which the core polarity is reversed. The difference between the frequencies of the two stable rotational modes corresponding to each core polarity is proportional to H and to the ratio of the disk thickness to its radius. Simple analytic theory in combination with micromagnetic simulations give quantitative description of the observed bistable dynamics.

Magnetic vortices are singular topological states found in the equilibrium magnetic configuration of sub-micron size ferromagnetic dots [1,2]. In a certain range of dot aspect ratios (ratio β = t/R of the dot thickness t to its radius R) the equilibrium ground state of the static magnetization consists of the curling in-plane magnetization and a nanometer size core of the out-of-plane magnetization at the dot center. The magnetization of the vortex core can point either up or down, both polarities p = ±1 being degenerate at zero field. This bi-stable property of magnetic vortices, as well as the switching from one polarity to the other, have been intensively studied in the past few years because of their possible applications in magnetic storage devices [3,4,5,6]. It has already been established : (i) that the lowest excitation mode of the vortex state is the gyrotropic mode corresponding to a rotation of the vortex core about the dot center, (ii) that the frequency of this mode is linearly proportional the dot aspect ratio β [7], and (iii) that the sense of gyration of the vortex core is determined by a right-hand rule to the core polarity [4].

In this Letter, we report that by using the exquisitely sensitive method of magnetic resonance force microscopy (MRFM) [8], we were able to observe bistability of the vortex core dynamics in a single magnetic disk subjected to a perpendicular bias magnetic field, that was varied in a wide range from positive (parallel to the vortex core) to negative (antiparallel to the vortex core) values. We demonstrate that in a certain range of the bias field magnitudes there are two stable gyrotropic modes of the vortex core rotation having different frequencies and opposite circular polarizations, and corresponding to opposite orientations of the vortex core relative to the direction of the bias magnetic field. The difference in frequencies of these two stable gyrotropic modes is proportional to the magnitude of the applied bias field, H, and, also, to the dot aspect ratio β. We believe that this effect might be important for the development of novel magnetic memory elements. It allows one to determine the polarity of the vortex core by measuring the frequency of the resonance absorption in a nano-disk subjected to a perpendicular field, which is substantially easier to implement than the detection of the sense of signal circular polarization, as it was suggested in [4].

Our experiments were performed at room temperature on individual nano-disks of the thickness t = 43.8 nm and two different radii (R 1 = 130 and R 2 = 520 nm). The nano-disks were patterned from a film of NiMnSb, a soft conductive magnetic material having very low magnetic losses (typical Gilbert damping constant is α = 0.002 -0.003), epitaxially grown on an InP(001) substrate [9]. A reference sample of a continuous film was cut out of the same film for characterization purposes. A 50 nm thick Si 3 N 4 cap layer was deposited on top of the disks for protection, and a 300 nm thick Au broadband strip-line microwave antenna was evaporated on top of the disks. This antenna generates a linearly polarized microwave magnetic field oriented in-plane perpendicularly to the stripe direction. The disk samples were placed in the uniform external bias magnetic field oriented perpendicular to the disk plane and having magnitude that was continuously varied from -10 to +10 kOe.

The detection scheme of MRFM is inspired by magnetic force microscopy (MFM) [8]. It consists of an ultrasoft cantilever with a 800 nm diameter sphere of amorphous Fe (with 3% Si) glued to its apex. The magnetization curve of this probe is typical of a soft Fe sphere (coercitivity < 10 G). Its magnetic moment always follows the direction of the applied field H and senses a dipolar force proportional to the perpendicular component M z of the magnetization of the nano-disk. MRFM spectroscopy is achieved by placing the center of the sphere above the center of the nano-disk. A ferromagnetic resonance (FMR) spectrum is obtained by recording the vibration amplitude of the cantilever as a function of H at constant microwave excitation frequency, that is switched ON and OFF at the cantilever resonance frequency. The MRFM signal originates from the diminution of M z of the nano-disk produced by the absorbtion of the microwave field [8]. During the scan, the cantilever resonance frequency is also recorded (see Fig. 1a). This enables static magnetometry of the sample. The saturation field of the R 1 = 130 nm nano-disk is H s = ±6 kOe, which marks the region of the maximum radial susceptibility.

Fig. 1b shows the FMR spectrum of the smaller nanodisk at the frequency of 7.5 GHz. Above the saturation field H s , the excitation spectrum of the disk consists of a series of peaks corresponding to the confined dipoleexchange spin-wave modes of the disk. Their quantized resonance frequencies (resonance fields) are given by Eq.( 1) from [10]. These modes are labeled with a pair (l, m) of azimutal and radial mode indices [8]. The lowest (0, 0) (and the most spatially uniform) spin wave mode of the disk is situated at 8.1 kOe and is marked by a blue circle symbol. The frequency of this lowest quasi-uniform Kittel mode ω K can be approximately described as:

ω K (H) = γ {H -H K } , (1)
where

H K = 4πM s (N zz -N rr ) -H A .
Here M s is the saturation magnetization, N ii are the diagonal elements of the effective demagnetization tensor of the uniformly magnetized disk, H A is a uniaxial perpendicular anisotropy field (easy-plane for H A < 0), and γ is the gyromagnetic ratio. All the parameters used in Eq. ( 1) were measured independently using the reference film sample by means of standard magnetometry and cavity-FMR techniques (results are presented in table I). The calculated position of the Kittel mode (as well as the positions of the higher modes calculated using Eq. ( 1) from [10]) are shown near the corresponding index pairs in Fig. 1b. This calculation did not use any adjustable parameters and took into account the stray field of the cantilever probe (around 500 Oe) [8]. The analytically calculated values of H K for the smaller and larger disks are H K ≈ 5.7 kOe and H K ≈ 8.0 kOe, respectively. When the external magnetic field H is reduced below the saturation field H s , a single magnetic vortex is formed inside the nano-disk (which can be visualized by standard MFM). Fig. 1c shows the dynamical behavior of the vortex-state disk (unsaturated regime). The absorption peak observed in the spectra corresponds to the excitation of the gyrotropic mode in the disk (see inset of Fig. 1a) [11]. It produces a mechanical signal mainly because the contribution of the vortex core to M z diminishes when it is excited [6]. Such signal is sensitive to the polarity of the vortex core: it is positive when the sphere magnetization and the vortex core are parallel and negative in the opposite case. Moreover, due to the field dependence of the magnetic moment of the sphere, the amplitude of the MRFM signal decreases as |H| is reduced to zero.

The baseline of each spectrum in Fig. 1c is set at the excitation frequency. The spectrum for 1.15 GHz has an intense peak at around +3.5 kOe (blue circle) corresponding to the excitation of the gyrotropic mode of the vortex core, as well as its mirror image at -3.5 kOe. The gyrotropic frequency decreases with the bias field H, and in the zero field (H = 0) has the value of 0.7 GHz. In the region of negative bias fields H < 0 the mode continues to exist, but the corresponding resonance peak has a negative amplitude, which indicates that now the direction of the bias field and the amorphous sphere magnetization is opposite to that of the vortex core. The gyrotropic frequency continues to decrease with the same slope until the bias field value of H r = -2.8 kOe, where the resonance peak abruptly changes its sign, frequency, and slope, which indicates reversal of the vortex core polarity. Below the switching field H < H r the gyrotropic frequency of the inverted vortex core increases like it did in positive bias fields before the core reversal. It is important to note, that upon increasing the bias field, the gyrotropic mode persists at reduced frequency until the symmetric reversal field of H r = +2.8 kOe is reached, and the new reversal of the vortex core polarity takes place.

The complete phase diagrams demonstrating the dynamic hysteresis loops of the gyrotropic mode frequency in a perpendicular bias magnetic field for two disk sizes are presented in Fig. 2. The dark blue dots correspond to the case when the bias field is reduced starting from large positive values, while the light red dots correspond to the opposite case, when the field is varied starting from large negative values.

Several important features emerge in these phase diagrams: (i) For |H| > H s the excitation frequency of the lowest mode in the disk is well described by the Kittel's Eq. ( 1), and at |H| = H s the field slope of the mode frequency changes abruptly, while the expected discontinuity of the mode frequency is not seen in the experiment; (ii) In the interval |H| < |H r | each value of the bias field (except H = 0) corresponds to two different gyrotropic mode frequencies corresponding to two opposite orien-tations of the vortex core relative to the bias field, and the difference between these frequencies increases with the bias field H and with the disk aspect ratio β; (iii) The magnitude of the bias field H = H r , at which the reversal of the vortex core polarity occurs, seems to be approximately the same for both studied disks.

The frequency of the gyrotropic mode in the external perpendicular bias magnetic field H can be calculated as a ratio ω G (H) = κ(H)/G(H) of the field-dependent vortex stiffness κ(H) to the field-dependent magnitude G(H) of the z-component of the vortex gyrovector using the method of Thiele's equation that was used in [7] to calculate this frequency in the case when H = 0. According to the general definition given in [7] (see Eq. ( 2) in [7]) when the perpendicular bias field is introduced we can write the vortex gyrovector as G(H) = G(0)(1p cos θ), where p = ±1 is the vortex core polarity, G (0) is the gyrovector in zero bias field defined in [7], and θ is the polar angle of the static magnetization at the disk lateral boundary. For a sufficiently large disk radius (much larger than the radius of the vortex core) is is possible to estimate θ from the standard electrodynamic boundary conditions at the vertical boundaries of the disk as cos θ = H/H s .

The field-dependent vortex stiffness κ(H) is found from the assumption that the main contribution to the vortex energy comes from the dipolar interaction of the volume magnetostatic charges created by the in-plane magnetization component of the shifted vortex outside the core [7]. In this region the in-plane magnetization components depend of the direction of the external bias field as sin θ, and the stiffness κ(H), that is proportional to the square of the radial derivative of the in-plane magnetization, is expressed as κ (H) = κ (0) sin 2 θ, where κ(0) is the vortex stiffness in the zero bias field defined in [7]. Thus, we can write the explicit expression for the gyrotropic mode frequency in the perpendicular bias field as

ω G (H) = ω G (0) 1 + p H H s , (2)
where ω G (0) is the gyrotropic mode frequency in the zero bias field defined by Eq. ( 7) of [12]. The approximate expression for ω G (0) obtained in the limit of small aspect ratios β ≪ 1 has the form ω G (0) = 5/(9π)4πγM s β [7].

The value of the saturation field H s can also be determined as the crossing point of the dispersion curves ( 1) and (2). It occurs at H s ≃ H K + 2ω G (0)/γ found to be ≈ 6.1 kOe for the smaller disk, a result that is close to the experimentally measured value shown in Fig. 2a. The theoretical curves calculated using Eq. ( 2) are shown by solid lines in Fig. 2, and it is clear from this figure that Eq. ( 2) gives a good quantitative description of the experimental data for two different values of the disk aspect ratio. This simple analytic theory does not account for the reversal of the vortex core polarity observed experimentally at H = H r (see Fig. fore, we performed micromagnetic simulations to reproduce this experimentally observed effect. The results of these simulations are presented in Fig. 3.

Our micromagnetic numerical code [13] calculates the stable configuration of the magnetization vector in the nano-disk by solving the Landau-Lifshitz (LL) equation in the time domain. A 3D mesh with a cubic cell of size 3.125 nm gives a discrete representation of the nanodisk magnetization. The nano-disk dimensions were chosen as R = 100 nm, t = 43.75 nm, while the values of the magnetic parameters were taken from table I. The numerically calculated static hysteresis loop of M z (H) (where the vertical component of the static magnetization M z was averaged over the disk volume) is shown in Fig. 3a by a dashed line. Color images of M z distribution across the radial section of the disk taken at four points corresponding to progressively decreasing values of the perpendicular bias magnetic field H are shown as inserts in Fig. 3a. Although the vortex core switching is not seen in this averaged static hysteresis loop, it is clearly revealed when the averaging of the static M z component is done over the spatial region occupied by the vortex core (≃ 15 nm), as demonstrated by the solid lines in Fig. 3a, where the core polarity switching appears as a jump at |H r | = 3.25 kOe.

It should be noted, that the vortex core reversal in the negative perpendicular bias field was studied analyt-ically and numerically in [14] and was attributed to the formation of a Bloch point. Experimentally this effect was studied on an array of magnetic dots in [15]. The magnitudes of the core reversal fields obtained in [14,15] are consistent with corresponding magnitudes obtained in our experiment Fig. 2 and numerical modeling Fig. 3a.

The dynamic hysteresis loop (i.e. the dependence of the gyrotropic mode frequency on the perpendicular bias magnetic field H) shown in Fig. 3b was numerically calculated using a second code developed by the authors, which computes the full dynamic susceptibility tensor χ ′′ from linearization of the LL equation around the equilibrium configuration [13]. It clearly demonstrates the effects of the bias-field-induced dynamic bistability [16] in the field interval |H| < H r . Additional numerical calculations are in progress to study the details of the gyrotropic frequency behavior in the vicinity of the points H = ±H s , where the vortex is formed, and H = ±H r , where the reversal of the vortex core polarity takes place.

This research was partially supported by the ANR PNANO06-0235 from France, by the European Grants DynaMax (FP6-IST 033749) and Master (NMP-FP7 212257), by the MURI Grant No. W911NF-04-1-0247 from the U.S. Army Research Office, by the Contract W56HZV-08-P-L605 from the U.S. Army TARDEC, RDECOM, and by the Grant No. ECCS-0653901 from the U.S. National Science Foundation. K.G. acknowledges support by the Ikerbasque Science Foundation.

FIG. 1 :

 1 FIG. 1: (Color online) (a) The cantilever frequency, proportional to Mz, the vertical component of the static magnetization of the R1 = 130 nm NiMnSb nano-disk, as a function of the perpendicular bias magnetic field H; (inset) Detection scheme: the vortex gyrotropic mode is sensed through the dipolar force induced on the spherical Fe probe; (b) Nanodisk excitation spectrum in the saturated regime (|H| > Hs) recorded at 7.5 GHz; (c) Excitation spectra in the vortex (unsaturated) state (|H| < Hs).

FIG. 2 :

 2 FIG. 2: (Color online) Frequency of the lowest excitation mode in a magnetic disk as a function of the perpendicular bias magnetic field for two sizes: (a) R1 = 130 nm, (b) R2 = 520 nm. Dark blue (light red) dots are experimental points corresponding to the field variation starting from large positive (negative) values. Solid lines are theoretical predictions from Eq. (1) for |H| > Hs and from Eq. (2) for |H| < Hs.

  (2)). There-

FIG. 3 :

 3 FIG. 3: (Color online)(a) Numerically calculated field dependence of the static magnetization of the NiMnSb disk (thickness t = 43.75 nm, radius R = 100 nm): the hysteretic behavior of the static magnetization, Mz, averaged over the disk volume (dashed loop) and over the vortex core (solid loop). The inserts show the spatial distribution of Mz at four progressively decreasing values of H. (b) Resonance locus calculated by the 3D micromagnetic code (dots) along with the analytical model (solid lines).

TABLE I :

 I Physical parameters of the reference NiMnSb layer.

4πMs (G)

γ (rad.s -1 .G -1 ) HA (G) α 6.9 × 10 3

1.8 × 10 7 -1.85 × 10 3 2.3 × 10 -3