Emmanuel D Bello
email: emabello42@gmail.com

Pablo A Salvadeo
email: pablo.a.salvadeo@gmail.com

Mimicking the human retina in the extraction of image descriptors

In this paper, we consider two models of the human retina, that describe the distribution of midget retinal ganglion cells (mRGC), to make modifications to the original sampling pattern of FREAK, an actual bio-inspired binary descriptor extraction method. We conclude that the more accurate, Watson's model, presents in average a better performance than the simple model, Exponential. Besides, new retinas obtain competitive results respect the classical algorithms and better than FREAK. In addition, we found that the bio-inspired patterns present the same behavior than other studied algorithms. In all cases appears the same ranking, where the natural scenes are more difficult to process than the artificial ones and the textured scenes than the structured ones. Finally, it is clear that the more complex retinal patterns allow getting the best performance and the Watson's model is a good way to obtain these ones.

Introduction

A widely employed technique to effectively represent visual information is through the computation of descriptors. Hence, a function is applied to a region of interest in an image to represent it in a way that is invariant to all the image changes that could appear in a Computer Vision application, (e.g. noise, illumination changes, viewpoint changes, etc.). Such function returns a descriptor vector, so that two descriptors can be compared with a distance function to determine their similarity. In comparison to working directly with image pixels, descriptors represent an alternative of reduced dimensionality, which can represent regions of interest in images allowing to reduce the complexity of computer vision applications [START_REF] Szeliski | Computer vision algorithms and applications, chapter Feature detection and matching[END_REF].

Well known examples of methods to compute descriptors are SIFT [START_REF] Lowe | Object recognition from local scale-invariant features[END_REF] and SURF [START_REF] Bay | Speeded-up robust features (SURF)[END_REF]. These methods generate descriptors computing histograms of gradients, which implies the calculation of the gradient of each pixel in the regions of interests, resulting in a high computational cost. Although SURF uses integral images to overcome this lack of performance, it is still not enough for some applications. Other alternatives have been the methods based on binary descriptors, that compute binary strings just by comparing pixel intensities on the regions of interests, which results in a higher performance that is suitable for applications that run in autonomous and mobile devices with reduced hardware resources or that must meet real-time requirements. In this sense, the binary alternatives are 10 times faster [START_REF] Canclini | Evaluation of low-complexity visual feature detectors and descriptors[END_REF].

The methods based on binary descriptors generally consist in three steps:

• Pattern sampling: a small patch centered around the keypoint is sampled. Before this sampling, the patch can be smoothed with a Gaussian filter to reduce sensitivity to noise.

• Orientation compensation: the patch is rotated, making it rotation invariant.

• Pairs sampling: to build the binary descriptor a set of pixel pairs (a, b) is selected from the patch. The values of each pair are compared, obtaining a 0 or 1, thus generating a sequence of 1's and 0's that form the descriptor.

In BRIEF (Binary Robust Independent Elementary Features) method [START_REF] Calonder | BRIEF: Binary robust independent elementary features[END_REF], the binary descriptor is obtained by comparing the intensity of 512 pairs of pixels, that are randomly selected according to a Gaussian distribution around the patch center, without any specific sampling pattern or orientation compensation mechanism. The ORB (Oriented Fast and Rotated BRIEF) method [START_REF] Rublee | ORB: An efficient alternative to SIFT or SURF[END_REF], unlike BRIEF, uses an orientation compensation mechanism, making its descriptor rotation invariant, and it learns the optimal sampling pairs.

On the other hand, we have binary descriptors methods that have handcrafted sampling patterns, e.g. the sampling pattern of the BRISK (Binary Robust Invariant Scalable Keypoints) method [START_REF] Leutenegger | BRISK: Binary robust invariant scalable keypoints[END_REF] is composed of concentric set of circles without overlapping, as it is shown in Fig. 1a. During pairs selection, BRISK distinguishes between short pairs and long pairs, i.e. pairs whose distance between them is below a certain threshold and pairs whose distance is above a different threshold, respectively. Long pairs are used in BRISK to determine orientation and short pairs are used for the intensity comparisons that build the descriptor. Another example is the FREAK (Fast Retina Keypoint) method [START_REF] Alahi | FREAK: Fast Retina Keypoint[END_REF] that, inspired on the human retina, employs a sampling grid composed of concentric circles, rings, with a set of receptive fields whose resolution exponentially decrease from the center to the outer of the patch, as is shown in Fig. 1b.

In fact, FREAK employs a similar sampling pattern to DAISY [START_REF] Tola | A Fast Local Descriptor for Dense Matching[END_REF]. DAISY is not bio-inspired and their rings are equidistant (see Fig. 1c). In both patterns, the overlapping and the increasing radius of the circles in the outer rings achieve soft transitions, thus allowing to gain robustness to rotation. One extra advantage of the circular patterns and using symmetric kernels is that, the descriptor can be computed in any orientation simply by rotating the sampling grid without the need to recompute convolved orientation maps [START_REF] Tola | A Fast Local Descriptor for Dense Matching[END_REF]. Besides, using sampling patterns instead of histograms, unlike SIFT and SURF, is faster. DAISY is 50 times faster than SIFT in the generation of descriptors, while FREAK is 140 times faster.

Winder et al. in [START_REF] Winder | Picking the best daisy[END_REF] discover a set of rules to improve the performance of the sampling pattern of DAISY. They find that the error rate decreases when the number of rings increases, achieving an additional increase in performance when the number of receptive fields increases from 6 to 8 by ring. However, an even greater number of receptive fields by ring do not improve the performance results, since the overlapping increases, thus producing an unnecessary oversampling. Such rules may be employed to build the sampling pattern of FREAK, which present: 7 rings with 6 circles each one, plus the center (Fig. 1b), totalizing 43 receptive fields.

Although Alahi et al. in [8] present a sampling grid inspired on the human retina, i.e. it mimics the spatial distribution of retinal ganglion cells (RGC), they do not use a biological model for such purpose. On the other hand, Watson in [START_REF] Watson | A formula for human retinal ganglion cell receptive field density as a function of visual field location[END_REF] presents a formula that describes the midget retinal ganglion cell (mRGC) density in the human retina as a function of location in the visual field.

The purpose of our work is to analyse the impact on image descriptors performance when the Watson's model is employed to build the sampling grid. Particularly, we work on the FREAK algorithm, changing the original retinal sampling grid presented in [START_REF] Alahi | FREAK: Fast Retina Keypoint[END_REF] by another one obtained from the Watson's model. Moreover, we contrast such modifications with the obtained by a simpler model of mRGC distribution, an exponential function. Our goal is not only to evaluate the descriptor performance, but also to give an insight into the properties of the sampling grid that could increase the descriptor performance and to observe the behavior of new patterns. The rest of the paper is organized as follows: Section 2 presents an overview of the method for filtering of the sampling grid; Section 3 describes the Watson's and Exponential models, and how they create the sampling grid. Section 4 explains the methodology to evaluate descriptors, while Section 5 exposes the results of such evaluations for different types of retinal patterns. Finally, Section 6 shows conclusions and further work.

Filtering of sampling pattern

In order to reduce noise sensitivity and to give extra invariance to translation, a Gaussian filter is applied to the sampling pattern [START_REF] Jain | Machine Vision, chapter Image Filtering[END_REF]. As it is explained in [START_REF] Wandell | Foundations of Vision, chapter The Retinal Representation[END_REF], the Gaussian filter can model the impulse response of RGCs. If G(x, y, σ) is the 2D (two-dimensional) Gaussian function shown in equation [START_REF] Szeliski | Computer vision algorithms and applications, chapter Feature detection and matching[END_REF].

G(x, y, σ) = 1 2πσ 2 e -(x 2 +y 2 )/2σ 2 (1)
Where σ corresponds to the standard deviation of the Gaussian probability distribution and, I(x, y) is the image intensity, then the patch is filtered through the convolution [START_REF] Lowe | Object recognition from local scale-invariant features[END_REF].

L(x, y, σ) = G(x, y, σ) * I(x, y) (2)
The term 1 √ 2πσ in front of the equation ( 1) is the normalization constant and it comes due to the fact that the integral over the exponential function is not unity.

Because we are working with digital images, i.e. discrete information, a convolution mask G nxm (also called Gaussian kernel) is used, so that the convolution (2) becomes the weighted sums of the image pixels of the patch I nxm , as shown in [START_REF] Bay | Speeded-up robust features (SURF)[END_REF].

L[i, j, σ] = I[i, j] * G[i, j, σ] = n k=1 m l=1 I[k, l]G[i -k, j -k] (3)
Where G[i, j, σ] is obtained from samples of G(x, y, σ). The Gaussian kernel size is determined by σ, so that I[i ± 3σ, j ± 3σ] pixel values (i.e.: µ ± 3σ ≈ 99.7% of the data values) are convolved with a Gaussian kernel of the same size. Instead of a 2D convolution, we use a separable convolution strategy [START_REF] Bonato | A parallel hardware architecture for scale and rotation invariant feature detection[END_REF]. This technique consists of a vertical convolution followed by a horizontal one, with the same 1D Gaussian kernel, obtaining the same result as the convolution with a 2D Gaussian kernel, but with less computational effort.

As it is shown in Fig. 1b, the FREAK sampling pattern consists of 7 concentric circles from inner to outer C 1 ...C 7 , plus the patch's center C 0 . In turn each C i =0 contains 6 receptive fields R i0 ...R i5 , and C 0 contains just one receptive field R 00 . Mimicking the RGC distribution we use a different Gaussian filter for each C i , with an increasing σ i from inner to outer of the patch.

Retinal sampling grid

Midget retinal ganglion cells

In the human eye, the initial sampling is performed by the cone and rod photoreceptors, and after that a resampling of their signals is performed by the retinal ganglion cells (RGC) [START_REF] Wang | Physiological and morphological characterization of ganglion cells in the salamander retina[END_REF]. RGCs are the output cells of the human eyes and consequently their properties limit the visual information that travels to the rest of the brain, then these are very important in the human vision system. Visual information includes light intensity and color information. The mechanism which is used to process this information is called early vision system. This one represents the first stage of visual information processing mechanisms. One type of RGCs are the midget retinal ganglion cells (mRGC), that are the most numerous (about 80% of RGCs are midget cells). Near the fovea (retina center) they appear to sample a single cone while in the peripheral retina they obtain signals from multiple cones [START_REF] Schaeffel | Handbook of Machine Vision, chapter Processing of Information in the Human Visual System[END_REF]. Then, two mRGCs distribution models are presented, the complex (Watson's model) and a simpler (Exponential), that are employed to obtain the values of σ to build the retinal patterns following the procedure presented in Section 2.

Watson's model

Due to the important role in modeling the human visual spatial processing, in [START_REF] Watson | A formula for human retinal ganglion cell receptive field density as a function of visual field location[END_REF] Watson develops an analytic formula that describes the distribution of midget retinal ganglion cells receptive fields (mRGCf) as a function of location in the visual field. Such formula tries to satisfy anatomical constraints and it is based on empirical data [START_REF] Curcio | Topography of ganglion cells in human retina[END_REF] [START_REF] Curcio | Human photoreceptor topography[END_REF].

The reason to describe the distribution of receptive fields instead of talking about cell bodies is due to the displacement zone near the fovea of RGCs. Although RGC density declined rapidly with eccentricity, unlike photoreceptors they do not peak at the fovea. This is because, in a central retinal zone, the ganglion cell bodies are displaced centrifugally some distance from the inner segments of the cones to which they are connected through the bipolar cells, and thus from their receptive fields. Thus the local density of the cell bodies does not reflect the local density of the RGC receptive fields (RGCf).

The equation ( 4) shows the Watson's model that describes the mRGCf density.

d mf (r, k) = 2 d c (0)(1 + r r m ) -1 × a k (1 + r r 2,k ) -2 + (1 -a k ) exp(- r r e,k ) (4)
Where d mf (r, k) is the density in function of eccentricity r and the meridian k (Temporal, Nasal, Superior, Inferior), d c (0) is the density of cone photoreceptors in the fovea, r m = 41.03 • , a k is the weighting of the first term, r 2,k is the eccentricity at which density is reduced by a factor of four, and r e,k is the scale factor of the exponential.

Exponential model

A simple model of mRGC distribution, to calculate each σ i , is an exponential function [START_REF] Calonder | BRIEF: Binary robust independent elementary features[END_REF].

σ i = σ i-1 β (5)
Where β is a value in the interval (1.2, 3.2), because values bigger than 3.2 result in many σ i equal or close to zero and values lesser than 1.2 produce linear or inverted curves; and σ corresponds to the standard deviation of the Gaussian probability distribution (see Section 2). In some cases it can be beneficial to set σ 0 = σ 1 . For this, in Table 2, the half has such property. The values of Table 2 are obtained choosing a random β value and using this in (5) from outer to inner σ i , because the outer σ is fixed to get equal size patterns (93px) with comparative purposes. In the next point we explain how obtain σ values from Watson's model.

Retinal sampling grid

As a supplement of [START_REF] Watson | A formula for human retinal ganglion cell receptive field density as a function of visual field location[END_REF], a Mathematica Notebook is provided as a file, which contains the equation ( 4). This file can be used with the Wolfram CDF Player [START_REF] Research | Inc. "Cdf player 10[END_REF] and then we use it to extract samples of (4) in order to build our retinal patterns. We take samples of the equation ( 4) from the center of the retina to the periphery, so that the retinal sampling grid employed can be modeled using a known biological model. These samples are taken following only one horizontal direction. With them each σ is modified, thus changing the receptive fields of the retinal pattern.

As it was explained in Section 2, we use a retinal pattern that consists of 7 concentric circles C 1 ...C 7 plus the patch center C 0 , where each one uses a Gaussian filter with a different σ i to smooth each of their receptive fields R ij . Hence, we decide to adjust each σ i accordingly to the samples of the equation ( 4), and thus the size of each receptive field.

Each σ should be inversely proportional to the mRGCf density in each point of the retinal sampling patch. We propose the equation ( 6) to model such proportion.

σ i = α d n (6)
Where d n is the density, i.e. a sample of (4), and α is a constant that allows to get a retinal sampling patch with a determined size. Thus, it must be decided which samples will be taken into account to calculate each of the eight new σ i , corresponding to the eight concentric circles of the retinal pattern. Hence, we propose different possible selections over 100 equally spaced samples d 0 , ..., d 99 , see Fig. 2: • Homogeneous (WH): eight equally spaced samples are taken.

• Heterogeneous: we propose nineteen different random selections named: W n (n = 1, 2...19).

Fig. 2 shows the plot of the curve obtained from the samples that we taken from the Watson's model, and the eight taken samples for each retinal pattern.

Table 1 shows the samples (mRGCf) taken from the Watson's model ( 4), the values of σ for each one and the corresponding α. The values of α are selected in order to get the same pattern size (93px), thus doing the different retinal sampling patterns fair comparable.

Fig. 3 shows four E s retinal patterns: E, E8, E9, E15, while Fig. 4 shows four W s retinal patterns: WH, W2, W3 and W12. All of them are shown at the same scale.

Descriptor evaluation

To compare the impact on the descriptors performance of the different retinal pattern presented in Table 1 and Table 2, we employ the framework proposed by Mikolajczyk and Schmid in [START_REF] Mikolajczyk | A performance evaluation of local descriptors[END_REF] for evaluation of local descriptors. Such a framework consists of a method for descriptors evaluation and a dataset with real images, and it is widely used in the literature.

The dataset depicts different types of geometric and photometric transformations, in which the descriptors must demonstrate invariance and robustness. The images in the dataset are classified in structured and textured scenes. Structured scenes contain homogeneous regions with distinctive edge boundaries, and the textured ones contain repeated textures of different forms. Fig. 5 and Fig. 6 show example images of the dataset and the different transformation represented on it: rotation and scale change (Fig. 5a and5b); viewpoint change (Fig. 6a and6b); image blur (Fig. 6c and5c); JPEG compression (Fig. 5d); and illumination (Fig. 6d). Regarding the types of scenes, Fig. 5a, 6a, 6c, 5d and 6d contain structured scenes, while Fig. 5b, 6b and 5c contain repeated textures of different forms.

In addition, we propose a second classification of scene types: natural (Fig. 5) and artificial (Fig. 6) scenes, i.e. natural scenes correspond to the nature while artificial ones contain human constructions. In this sense, Fig. 6a, 6b, 6c and 6d represent artificial scenes; Fig. 5b and5c contain natural scenes; while Fig. 5a and 5d contain a mixture of both scene types. The criterion to evaluate the descriptors is recall vs. 1-precision [START_REF] Fawcett | An introduction to {ROC} analysis[END_REF]. Recall is the ratio between the number of correctly matched regions and the total number of corresponding regions between two images. Given a descriptor D A for region A in a reference image, a descriptor D B for a region B in the compared image, and the homography between the images H, the two regions are corresponding if the overlap error S (7) is less than 0.5. The two regions are correctly matched if they are corresponding and the distance between their descriptors is below a threshold t. Varying t is obtained the recall vs. 1-precision curve, where 1-precision is the ratio between the number of incorrectly matched regions and the number of matched regions. Therefore, a good descriptor should have a high recall rate for low 1-precision. An implementation of such descriptors evaluation method is presented in the OpenCV library 2.4 [START_REF] Bradski | The OpenCV Library[END_REF], which we employed to run our experiments.

It is important to consider the keypoint detection and matching methods that are employed in the evaluation of descriptors. For our experiments we use the SURF keypoint detector and the Hamming distance to match descriptors, because with such configuration we obtained the best general results, as it is shown in [START_REF] Canclini | Evaluation of low-complexity visual feature detectors and descriptors[END_REF]. Other important parameters for our experiments are the number of scales and octaves considered in the computation of descriptors, i.e. the octaves and scales of the multi-scale representation where the keypoints could have been detected [START_REF] Szeliski | Computer vision algorithms and applications[END_REF]. The octave o 0 contains the original image and its representation at differents scales, and every octave o i is obtained down-sampling the original image of the octave o i-1 .

As is proposed in [START_REF] Levi | LATCH: learned arrangements of three patch codes[END_REF], to compare the performance of the descriptors we calculate the area under the recall vs. 1 -precision curve, averaged over all five image pairs in each set of images, so that an ideal descriptor performance should be when such area is equal or close to one. In this way we can also compare our results with the evaluated performance of descriptors presented in [START_REF] Levi | LATCH: learned arrangements of three patch codes[END_REF], as it is shown in Section 5.

Results Analysis

Performance of the retinal patterns

Table 3 shows the area under the recall vs. 1-precision curve for each retinal pattern and averaged over the five pairs of images presented in each subset, and also an average over all the dataset is shown. For all our experiments the number of scales and octaves were settled to two and one, respectively, unless otherwise stated.

At the beginning, we can see that all sampling points are not equally important to model the retinal pattern (see Fig. 2). For example, albeit the samples taken for WH are equally distributed, too many samples are taken from the distant points from the fovea, and thus the most of the samples are irrelevant because they do not differ significantly. Unlike WH, the heterogeneous sampling W2 and W3 that take more sampling points near the inflection point of the curve that describes the Watson's model, thus presenting better performance. Between them, W3 shows the best performance, since it takes samples from the inflection point of the curve. W2 follows it, showing that samples of the flat part of the curve do not contribute to an increasing in descriptor performance.

However, W12 presents the best performance and all its samples are in the flat part. Then, it is not clear which is the more beneficial segment of the curve to take samples, near to inflection point or flat part. Indeed, W15 has a performance similar to W3, but its samples are in the flat portion of the curve. In this sense, we note that the better performance results are obtained when the resulting retinal pattern is more complex independently of its origin. If we see Fig. 2 and compare the retinal patterns W12 and E8 with the rest, it is clear that these two patterns are more complex. Even more, the Watson's model has an advantage over the simple Exponential model because we can take samples from two different segments and obtain good results. Then, we can say the former is more robust.

Moreover, independently of the model used, the best and worst performance results are obtained with similar distributions, where the best performance is obtained from retinal patterns whose σ distribution slightly grows, following a concave or convex curve, from the inner to the outer the pattern (Fig. 7a). On the other hand, the worst performance is obtained from σ distributions that grow sharply in the outer the pattern, leaving σ values close to zero in the rest of the concentric circles (Fig. 7b).

Finally, if we compare the average performance obtained with the Watson's model and the Exponential model, in Fig. 9 it can be noted that the most precise model gets better results. In such figure, the half with lower performance (the twenty worst) contains the 70% of the patterns derived from the Exponential model, while the half with higher performance (the top twenty), conversely, contains the 70% of the retinal patterns derived from the Watson's model. Thus, the simpler model gives some times also good performance results, but less frequently.

For comparison purpose, Fig. 10 shows the descriptor performance of twelve different descriptor extraction methods presented in [START_REF] Levi | LATCH: learned arrangements of three patch codes[END_REF], namely SIFT, SURF, LDA-HASH [START_REF] Strecha | Ldahash: Improved matching with smaller descriptors[END_REF], LDA-DIF [START_REF] Strecha | Ldahash: Improved matching with smaller descriptors[END_REF], DBRIEF [START_REF] Trzcinski | Efficient Discriminative Projections for Compact Binary Descriptors[END_REF], BinBoost [START_REF] Trzcinski | Boosting Binary Keypoint Descriptors[END_REF] [28], BRIEF, ORB, BRISK, FREAK, A-KAZE [START_REF] Alcantarilla | Fast explicit diffusion for accelerated features in nonlinear scale spaces[END_REF] and LATCH [START_REF] Levi | LATCH: learned arrangements of three patch codes[END_REF], with their original parameter values left unchanged. Additionally, the best performance results obtained with our models, W12 and E8 (both with the parameters scales = 2 and octaves = 3), are presented in Fig. 10. On the other hand, Table 4 shows a ranking of the best descriptor performance of the algorithms presented in Fig. 10. In this case, we note that our results are competitive with the other algorithms evaluated with the same framework.

Parameters variation of the retinal patterns

Now we present the performance results obtained for different numbers of scales and octaves. The results presented in Fig. 8a are obtained changing the number of scales used in the descriptors generation, setting the pattern size equal to 59px and the octaves to one. Fig. 8a depicts that an increasing number of scales do not necessarily mean a better performance. On the other hand, the results presented in Fig. 8b are obtained changing the number of octaves, setting the pattern size equal to 93px and the number of scales to two. This figure shows that with more octaves the performance improves, but for more than 3 octaves the performance tends to decrease. Experiments with more than 6 octaves produce incomplete results, independently of the number of scales considered.

Furthermore, it is important to consider the size of the retinal pattern, because we have observed that with larger retinal patterns, performance tends to improve independently of the other parameters, like it is shown in Fig. 8c for patterns obtained using the exponential model. However such improvement in performance is misleading, because with larger retinal patterns the effective size of the images becomes smaller. For this reason, we decided to fix the retinal pattern size for each experiment.

Behavior of the retinal patterns

In general, the performance exhibits intrinsic difficulties related to some images and transformations. If the subsets of the image dataset (Fig. 5, Fig. 6) are ordered according to the best performance results (see Fig. 10), we get the next ranking:

1. Bikes

Leuven and UBC

Graffiti and Wall

Boat, Bark and Trees

We find a similar ranking to the mentioned above when we analyse the subsets for each retinal pattern. Such orders may respond to the difficulties of the scene types presented in the images. Then, we analyse the performance averaging the results by scene types: structured vs. textured and artificial vs. natural.

In Fig. 11a it is shown the case for structured and textured scenes, from which is clear that the performance is weaker for textured scenes. Considering the classification between natural (Bark, Trees), artificial (Graffiti, Wall, Bikes, Leuven) and mixed (Boat, UBC) scenes that we proposed earlier (Fig. 5, Fig. 6) and, as it is shown in Fig. 11b for our performance results, we can conclude that natural scenes are more difficult to identify than artificial ones. Although some transformations do more difficult than others the correct matching of descriptors, note the difference between Trees and Bikes subsets, that represent the same transformation (blurring), but with Bikes better performance results are obtained.

Note that, contrary to what was expected, the bio-inspired retinal patterns do not present better performance results over natural scenes. Indeed, the behavior does not change respect of structured and textured images like any other cited algorithm.

Conclusion

We found that the more accurate mRGC model, Watson's model, presents in average a better performance than the simpler model, Exponential. Besides, new bio-inspired retinas obtain competitive results respect the classical algorithms and better than FREAK. Finally, it is clear that the more complex retinal patterns allow getting the best performance and the Watson's model is a good way to obtain these ones.

In addition, we conclude that the bio-inspired patterns present the same behavior than other studied algorithms. In all the cases appears the same ranking, where the natural scenes are more difficult to process than the artificial ones, and the textured scenes than the structured ones.

Acknowledgment

This work was partially supported by the Universidad Tecnológica Nacional.

Fig. 1 .

 1 Fig. 1. Sampling patterns. Each circle represents a receptive field. a BRISK b FREAK c DAISY

Fig. 2 .Fig. 3 .Fig. 4 .

 234 Fig. 2. Samples of the Watson's model. a WH b W2 c W3 d W12

Fig. 5 .

 5 Fig. 5. Natural scenes of the image dataset used for evaluation of descriptors. a Boat (rotation and scale change): structured, mixture of artificial and natural b Bark (rotation and scale change): textured, natural c Trees (image blur): textured, natural d UBC (JPEG compression): structured, mixture of artificial and natural

Fig. 6 .

 6 Fig. 6. Artificial scenes of the image dataset used for evaluation of descriptors. a Graffiti (viewpoint change): structured b Wall (viewpoint change): textured c Bikes (image blur): structured d Leuven (illumination change): structured

Fig. 7 .

 7 Fig. 7. σ distribution of each retinal pattern along its concentric circles. a Retinal patterns with the best descriptor performance results. b Retinal patterns with the worst descriptor performance results.

Fig. 8 .

 8 Fig. 8. Impact of parameters variation in descriptor performance a Descriptor performance vs. number of scales. b Descriptor performance vs. number of octaves. c Descriptor performance vs. retinal pattern size.

Fig. 9 .

 9 Fig. 9. Average performance for all the retinal patterns.

Fig. 10 .

 10 Fig. 10. Area under recall vs. 1-precision curve averaged by scene types. a Averaged by structured and textured scenes. b Averaged by natural and artificial scenes.

Table 1 .

 1 Parameters for retinal paterns (W s ). Model mRGCf density [deg -2 ] from inner to outer the retina α σ from inner to outer the pattern (C 0 , C 1 , C 2 , C 3 , C 4 , C 5 , C 6 , C 7 ) WH 28879.49, 220.73, 60.52, 30.57, 10.29, 4.38, 2.12, 0.95

Table 2 .

 2 Parameters for retinal paterns (E s ). Model β σ from inner to outer the pattern (C 0 , C 1 , C 2 , C 3 , C 4 , C 5 , C 6 , C 7 )

	E	1.4142 1.61, 1.61, 2.28, 3.22, 4.55, 6.44, 9.11, 12.88
	E1	2.9768 0.0062, 0.02, 0.06, 0.16, 0.49, 1.45, 4.33, 12.88
	E2	2.7695 0.0103, 0.03, 0.08, 0.22, 0.61, 1.68, 4.65, 12.88
	E3	2.9531 0.0066, 0.02, 0.06, 0.17, 0.5, 1.48, 4.36, 12.88
	E4	2.5991 0.02, 0.04, 0.11, 0.28, 0.73, 1.91, 4.96, 12.88
	E5	2.9271 0.007, 0.02, 0.06, 0.18, 0.51, 1.5, 4.4, 12.88
	E6	2.6998 0.0123, 0.03, 0.09, 0.24, 0.65, 1.77, 4.77, 12.88
	E7	2.4206 0.03, 0.06, 0.15, 0.38, 0.91, 2.2, 5.32, 12.88
	E8	1.2792 2.3, 2.94, 3.76, 4.81, 6.15, 7.87, 10.07, 12.88
	E9	2.0756 0.08, 0.16, 0.33, 0.69, 1.44, 2.99, 6.21, 12.88
	E10	1.3768 1.37, 1.89, 2.6, 3.59, 4.94, 6.8, 9.36, 12.88
	E11	3.1915 0.01, 0.01, 0.04, 0.12, 0.4, 1.26, 4.04, 12.88
	E12	2.6025 0.04, 0.04, 0.11, 0.28, 0.73, 1.9, 4.95, 12.88
	E13	2.4514 0.06, 0.06, 0.15, 0.36, 0.87, 2.14, 5.25, 12.88
	E14	1.4769 1.24, 1.24, 1.83, 2.71, 4.0, 5.9, 8.72, 12.88
	E15	1.6347 0.67, 0.67, 1.1, 1.8, 2.95, 4.82, 7.88, 12.88
	E16	2.8509 0.02, 0.02, 0.07, 0.19, 0.56, 1.58, 4.52, 12.88
	E17	2.4765 0.06, 0.06, 0.14, 0.34, 0.85, 2.1, 5.2, 12.88
	E18	3.0713 0.02, 0.02, 0.05, 0.14, 0.44, 1.37, 4.19, 12.88
	E19	1.9129 0.26, 0.26, 0.5, 0.96, 1.84, 3.52, 6.73, 12.88

Table 3 .

 3 Area under recall vs. 1-precision curve. .02044 0.00005 0.00357 0.01020 0.01900 0.01793 0.00871 0.0100 W10 0.01259 0.14146 0.00005 0.00518 0.09557 0.04852 0.23409 0.06137 0.0749 W11 0.01838 0.19713 0.00005 0.00005 0.19179 0.09475 0.39056 0.06340 0.1195 W12 0.00004 0.35035 0.00488 0.02227 0.24975 0.09848 0.39999 0.11795 0.1555 W13 0.00927 0.02206 0.00385 0.01901 0.16405 0.02398 0.14991 0.06514 0.0572 W14 0.01096 0.06476 0.00453 0.01757 0.15882 0.04348 0.22034 0.07887 0.0749 W15 0.00004 0.09177 0.01584 0.00005 0.29718 0.06168 0.29857 0.08818 0.1067 W16 0.00004 0.02166 0.00343 0.00598 0.04254 0.01351 0.06819 0.01740 0.0216 W17 0.00533 0.04068 0.01139 0.01701 0.19678 0.00885 0.15951 0.03675 0.0595 W18 0.00004 0.00896 0.00342 0.00459 0.02137 0.00776 0.00958 0.00859 0.

	Model	Bark	Bikes	Boat	Graffiti Leuven	Trees	UBC	Wall	Average
	WH	0.00408 0.01933 0.00344 0.00827 0.13597 0.02475 0.08488 0.03397 0.0393
	W1	0.00477 0.04755 0.00611 0.01432 0.17558 0.01408 0.12201 0.04839 0.0541
	W2	0.01211 0.02148 0.02077 0.02445 0.29138 0.04395 0.25980 0.09397 0.0960
	W3	0.01405 0.14681 0.02350 0.00005 0.31006 0.05209 0.17872 0.11198 0.1047
	W4	0.00859 0.14109 0.00005 0.00005 0.23527 0.02958 0.25433 0.08836 0.0947
	W5	0.00004 0.00997 0.00005 0.00561 0.10293 0.01379 0.06623 0.03014 0.0286
	W6	0.00004 0.05722 0.00005 0.00005 0.13339 0.02561 0.12855 0.04505 0.0487
	W7	0.01497 0.19520 0.00542 0.00005 0.18844 0.06361 0.25046 0.06916 0.0984
	W8	0.00924 0.03003 0.00005 0.02104 0.13450 0.03284 0.19205 0.06862 0.0610
	W9	0.00004 00080
	W19 0.00004 0.01991 0.00005 0.00478 0.08551 0.00802 0.05158 0.02322 0.0241
	E	0.00004 0.32054 0.02549 0.02376 0.21854 0.09552 0.37463 0.10264 0.1451
	E1	0.00004 0.01812 0.00005 0.00005 0.05395 0.00441 0.03352 0.00648 0.0146
	E2	0.00004 0.02218 0.00005 0.00005 0.04647 0.00355 0.03365 0.01900 0.0156
	E3	0.00004 0.01670 0.00005 0.00005 0.05680 0.01199 0.03315 0.01694 0.0170
	E4	0.00004 0.01143 0.00343 0.00005 0.05809 0.01407 0.03708 0.02039 0.0181
	E5	0.00004 0.01879 0.00005 0.00391 0.06128 0.00807 0.03713 0.01316 0.0178
	E6	0.00004 0.01320 0.00005 0.00381 0.03793 0.01102 0.05659 0.01126 0.0167
	E7	0.00004 0.01329 0.00343 0.00462 0.02204 0.01251 0.05721 0.02123 0.0168
	E8	0.00004 0.35153 0.02012 0.00005 0.19696 0.09821 0.40430 0.12620 0.1497
	E9	0.00004 0.01042 0.00347 0.00005 0.10390 0.00989 0.09974 0.03900 0.0333
	E10	0.00004 0.23645 0.00005 0.02124 0.19366 0.04297 0.37940 0.12649 0.1250
	E11	0.00004 0.00889 0.00343 0.00005 0.02452 0.00774 0.02888 0.01060 0.0105
	E12	0.00004 0.01063 0.00343 0.00005 0.05888 0.01491 0.03590 0.01959 0.0179
	E13	0.00004 0.01108 0.00437 0.00005 0.07268 0.01276 0.05519 0.02286 0.0224
	E14	0.00004 0.13149 0.00005 0.02368 0.27634 0.08950 0.36844 0.11443 0.1255
	E15	0.00004 0.13834 0.01386 0.01798 0.23656 0.03882 0.20258 0.10373 0.0940
	E16	0.00353 0.00521 0.00005 0.00005 0.04015 0.00890 0.05117 0.01883 0.0160
	E17	0.00004 0.01417 0.00343 0.00522 0.06110 0.00471 0.05685 0.02025 0.0207
	E18	0.00004 0.00011 0.00005 0.00005 0.03777 0.01142 0.03178 0.01063 0.0115
	E19	0.00004 0.01917 0.00436 0.00368 0.11296 0.00928 0.12081 0.05027 0.0401

Table 4

 4 Ranking of descriptor performances

	#	Algorithm Avg. performance
	1	LDA-DIF	0.2886
	2 LDA-HASH	0.2704
	3	W12	0.2086
	4	E8	0.2010
	5	LATCH	0.1878
	1	SURF	0.1800
	6	BinBoost	0.1656
	7	BRIEF	0.1458
	8	SIFT	0.1314
	9	A-KAZE	0.0948
	10	FREAK	0.0884
	11	ORB	0.0833
	12	BRISK	0.0748
	13	DBRIEF	0.0098

S = 1 -(A ∩ H T BH)/(A ∪ H T BH)(7)


Fabrice Casset
email: fabrice.casset@cea.fr

Jean-Sébastien Danel

Philippe Renaux

Cedrick Chappaz

Gwenaël Le Rhun

Christel Dieppedale

Marie Gorisse

Skandar Basrour

Stéphane Fanget

Pascal Ancey

F Cassee

J S Danell

P Renauxl

G Le Rhunl

C Dieppedalel

M Gorisse3

S Basrour3

S Fangetl

P Ancel

A Devos4

E Defa

Characterization and post simulation of thin-film PZT actuated plates for haptic applications

niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

The tremendous development of tactile interface in many customers' applications such as Smartphone, tablet PC or touch pad leads industrials to study "haptic interfaces" or "touch screen" solutions. This technology is already used but with limitations such as high power consumption and limited feedback effect (simple vibration). PZT is a good candidate for many actuator applications due to its high piezoelectric coefficient. In particular, it can be used for haptic interfaces to create squeeze-film effect. It consists in changing the friction between the finger and a plate resonator. It provides high granularity level of haptic sensation (texture rendering), using low power consumption compared to existing solutions. We manufactured demonstrators using a generic technology. We proved the concept through electro mechanical characterizations and haptic feedback effect was noticeable with one's fmger on our thin-film PZT demonstrators. In this paper, we presented the characterization and post-simulation of PZT-actuated plates with vanous actuator configurations. Measurement results, in good agreement with simulation, indicate that 2 actuator columns separated by a wavelength allow obtaining the highest substrate displacement amplitudes.

Introduction

Many applications can be concerned by high performances, low voltage haptic interfaces which allow the user to interact with its environment by the sense of touch. We developed sol-gel thin-film Pb(Zros 2 ' Tio4s)0 3 (PZT) actuated plates allowing the feeling of textured surfaces. It consists in changing the friction between the finger and a plate resonator due to the squeeze-film effect, which occurs when the plate displacement amplitude reaches about l/lm in a flexural anti symmetric Lamb mode [I, 2]. [t provides high granularity level of haptic sensation (texture rendering), using low power consumption compared to existing solutions.

We manufactured demonstrators using a generic technology compatible with RF MEMS [START_REF] Cueff | A fully package piezoelectric switch with low voltage actuation and electrostatic hold[END_REF], loudspeakers [4] or micro mirror [5]. We proved the concept through electro-mechanical characterizations. Laser vibrometer measurements indicate that our PZT actuator design promotes the desired mode with the targeted amplitude. Haptic feedback effect was noticeable with one's fmger on our thin-film PZT demonstrators [6].

This paper reports on the design, the characterization and the post-simulation of thin-film PZT actuated haptic plates. We developed predictive model, and in particular we studied the best actuator configuration in order to promote the highest substrate displacement amplitude.

Design

We designed high performances thin-film PZT actuated haptic plates using Finite Element Method (FEM) approach (CoventorWare ™ ). All simulations are performed using an undamped 725 flm thick Silicon substrate as resonant plate. Our model neglects top and bottom electrodes which sandwiched the 2flm thick PZT layer due to their weak impact on simulation results. We used Manhattan bricks to mesh the haptic plate. As input data, we used the thin-film PZT d We already proved that the maximum substrate displacement amplitude of our haptic plate can be obtain by taking the deformed shape of the selected mode into account, and matching the actuators position with the substrate maximum displacement amplitude as shown in Figure 1. PZT actuators promote the desired mode by bimorph effect [6]. As shown in Figure 1, various PZT actuator configurations are possible. In particular, we studied:

1 actuator column, 2 actuator columns separated by a wavelength: in this case the actuator columns are located on successive in-phase maximal substrate displacement amplitude areas, 2 actuator columns located at the extremities of the plate: the actuator columns are located on in-phase maximal substrate displacement areas near the ends of the plate. For each studied configuration, we first performed a modal simulation in order to defme the frequency of the desired mode. Then relative harmonic simulations were performed using arbitrary damping parameters and a given actuation voltage. As shown in Figure 2, two actuators columns separated by a wavelength induce the highest substrate displacement amplitude for both 60x40 and 40x30 mm2 plates.

Technology

Devices were manufactured out of 200 mm standard silicon wafers. First step is a 500 nm thick thermal oxidation (Si0 2

). Then, the piezoelectric stack, which consists of 2 flm thick sol-gel PZT in between 100 nm thick Pt bottom electrode and 100 nm thick Ru top electrode, was deposited and etched. We deposited and patterned a passivation silicon oxide layer followed by gold lines and pads. Figure 3 gives a cross section view of the technological stack as well as a SEM cross section of the PZT actuator. Only 5 photolithography masks were necessary to build the demonstrators.

Finally, we sawed the substrate to transfer the individual plate onto a mechanical support allowing to hold the vibrating plate and to connect it electrically. The plate was fixed by an adhesive tape and the actuators were wire bonded to electrical connectors. Figure 4 gives an example of a 60x40 mm2 plate ready for electrical characterization. Figure 4: 60x40 mm2 plate fIXed on a carrier using adhesive tape and with PZT actuators bonded to electrical connectors.

Electromechanical characterization and post simulation

Optical measurements were performed using a laser vibrometer (POLYTEC) by applying a dc-ac mixed voltage to the top electrode of plates (60x40 or 40x30 mm2). Laser vibrometer measurements give the substrate displacement amplitude for a given actuation voltage. Several measurements have been necessary to build the plate's whole picture given in Figure 5. This picture allows us to validate that the studied mode is the flexural anti symmetric Lamb mode required for haptic applications. We measured the desired resonant mode at 32.2 kHz for the 60x40 mm2 plate. The agreement between measurements and simulations is satisfactory. Indeed a discrepancy lower than 1% is noticed. Moreover, we note that the substrate displacement amplitude reached 500 nm under 4V ac/peak to peale This value is very satisfactory for such a low actuation voltage.

Using FEM approach, we adj usted a damping parameter to fit the substrate displacement amplitude previously measured. Figure 6 gives the comparison between the simulated substrate displacement amplitude and the measured one under 4V. It allows us to determine that a damping parameter of 2.5.10-8 is necessary to fit measurement for 60x40 mm2 plates.

Only one damping parameter per plate sizes was used as reported in Table 1. For example, with a damping parameter of 2.5.1O-8 , we can post-simulate a 60x40 mm2 plate as shown in Figure 7 (discrepancy < 5%). One can note the micrometric displacement amplitude, namely I.I/lm, obtained using only 8V ac/peak to peak. A haptic feedback effect was felt with the finger on this plate by using an actuation signal modulated at 10 Hz in amplitude [6]. Both on 60x40 and 40x30 mm2 plates, the best actuator columns configuration were studied. As shown in Figure 8, measurements results obtained using 12V de + 4Vac/pp, in good agreement with simulation, indicates that 2 actuator columns separated from each other by a wavelength allow obtaining the highest substrate displacement amplitude. Unfortunately, most of the targeted applications require transparency, at least of the center of the plate, and this actuator configuration induces a large opaque zone (dead area). Consequently it will only be retained if transparency is not required by the focused application.

We note on Figure 8 that the measured substrate displacement amplitude with 2 actuator columns located at the plate ends is only 5% smaller for 40x30 mm2 plates and 11% smaller for 60x40 mm2 plates, compared to the best configuration discussed previously. This configuration presents a larger area dedicated to touch experience. It will be favored, for example for Smartphone applications, for which we have to preserve the largest transparent surface as possible, dedicated for touch experience.

Finally the last configuration with only one actuator column presents the smaller dead area. [t induces measured substrate displacement amplitude respectively 46 and 88% smaller for 40x30 and 60x40 mm2 plates, compared to the best configuration discussed previously. In the case of the characterized 40x30 mm2 plate, the measured substrate displacement amplitude is all the same satisfactory for such a low actuation voltage. FEM simulation indicates that we can reach the micrometric substrate displacement amplitude required for haptic application under 14 Vac.

[n the case of the characterized 60x40 mm2 plate, the low measured substrate displacement amplitude can result from a malfunction of the device. Further measurements must be performed to confirm this experimental result. This configuration can be retained for applications that need the smallest non-transparent dead area, to the detriment of low actuation voltage.

Conclusion

We have developed a unique solution based on PZT thin-film actuated plates allowing haptic feedback effect using low bias voltage due to squeeze-film effect. In this paper, we presented the characterization and post-simulation of PZT-actuated plates with various actuator configurations. Measurement results, in good agreement with simulations, indicate that 2 actuator columns separated by a wavelength allow the highest substrate displacement amplitudes. But this configuration, inducing a large non-transparent zone, occupied by actuators, will be retained if transparency in not required by the focused application. The second configuration, with 2 actuator columns located at the plate ends presents substrate displacement amplitude slightly smaller compared to the best configuration, but exhibits a larger area dedicated to touch experience. This configuration will be favored for example for Smartphone applications for which we have to preserve the largest transparent surface as possible. Finally, the configuration with only one actuator column presents the smaller dead area. It induces smaller substrate displacement amplitude compared to the best configuration. This configuration will be favored for the applications requiring the smallest non-transparent dead area, to the detriment of low actuation voltage.

  namely 165 pm/V [7], the PZT Young's modulus EpZT = 82 OPa and Poisson's ratio v = 0.39 [8].

Figure 1 :

 1 Figure 1: Modal simulation (51.89 kHz) on a 40x30 mm2 plate and PZT actuator columns possible localization.

Figure 2 :

 2 Figure 2: Harmonic simulations with arbitrary damping parameter study -Actuator columns configuration study for both, 60x40 and 40x30 mm2 plate.

Figure 3 :

 3 Figure 3: Schematic cross section of the technological stack and SEM cross section of the PZT actuator.

Figure 5 :Figure 6 :

 56 Figure 5: Laser vibrometer measurement on the 60x40 mm2 plate (12Vdc +4Vaclpp). Resonant frequency measured at 32.2 kHz.

  ac actuation voltage peak to peak (V)

Figure 7 :

 7 Figure 7: Laser vibrometer measurements and simulated substrate vertical displacement (pictures) amplitude comparison on a 60x40 mm2 plate (12 Vdc +variable Va clpp ).

Figure 8 :

 8 Figure 8: Laser vibrometer measurements -Substrate vertical displacement amplitudes on 60x40 and 40x30 mm2 plate presenting various actuator configurations (12Vdc +4Vaclpp)'

ACKNOWLEDGEMENTS

The authors wish to acknowledge all participants at the MINALOGIC project Touch It. Moreover we wish to acknowledge the OSEO funding to give us the opportunity to work on this subject.