Dounia Awad

Vincent Courboulay

Arnaud Revel

A new hybrid texture-perceptual descriptor: application CBIR

Content based image retrieval (CBIR) has been the center of interest for a long time. A lot of research have been done to enhance the performance of these systems. Most of the proposed works focused on improving the image representation(bag-of-features) and classification methods. In this paper, we focus on enhancing the second component of CBIR system: region appearance description method. In this context, we propose a new descriptor describing the spatial frequency property of some perceptual features in the image. This descriptor has the advantage of being lower dimension vs. traditional descriptors as SIFT (60 vs 128), thus computationally more efficient, with only 5% loss in performance using a typical CBIR algorithm on VOC 2007 dataset.

). According to [6], local approaches consists in 5 steps: region selection, region appearance description, region appearance encoding, derivation of image features from the set of region appearance codes by spatial pooling, classification . A baseline of CBIR method [6] is represented in figure 1. In the following, we present the different methods used in each step.

, a typical CBIR algorithm first scans the image to select the regions of interest. In the literature, there are two concepts to accomplish such task:

-The first concept considers the entire image. Most researchers in this category use dense grid in their algorithms [15] [38]. It consists in dividing images into small regions and extracting information within each region. The main strength of this technique is that is there no loss in data. However, processing all this amount of information is computationally intensive in terms of time and memory: most of the processing time is spent on analyzing useless data.

[25].

-The second concept, in opposition to the first, considers that not the whole image is necessary for an image retrieval task. The researchers in this category use techniques as:

-Interest point detector [START_REF] Tuytelaars | Local invariant feature detectors: a survey[END_REF]: they extract the points of interest basing on simple geometric forms as blobs, corners. They have the advantage of being invariant to scale changes. However, the main limitation of using interest point detector is that they consider that the interest in the image is directly correlated with the presence of geometric structures. This constraint is well known as semantic gap [START_REF] Santini | Emergent semantics through interaction in image databases[END_REF]. -Saliency based-region selection: other researchers used more perceptual techniques for selecting regions based on human attention. These systems, known as bottom-up visual attention systems [START_REF] Borji | State-of-the-art in Visual Attention Modeling[END_REF], have an objective to select the regions that attract our interest according to some discriminate features [START_REF] Frintrop | Computational Visual Attention[END_REF]: color, orientation, intensity. These features were chosen basing on some psychological and physiological theories [START_REF] Treisman | A Feature-Integration Theory of Attention[END_REF] [34] [START_REF] Wolfe | Guided Search 2.0: A revised model of visual search[END_REF].

An evaluation of these two concepts based on comparison between interest point detectors and dense grid [START_REF] Winn | Object categorization by learned universal visual dictionary[END_REF] demonstrated that for some image classification tasks, selecting regions from a regular grid outperforms the use of interest point detectors.

region appearance description The second step of CBIR algorithm consists in describing each selected region by a set of regions appearance attributes, called local descriptors or local features. As a matter of fact, deciding which feature is more fitted to describe a region, is delicate and depends on several factors as the task of the proposed system, the class of the considered object. Generally, this kind of decision is based on compromise between the accuracy of the system and the generality of the chosen feature. Therefore, a lot of descriptors have been proposed. According to their feature basis for describing regions of interest, we can categorize them in 4 families [START_REF] Mikolajczyk | A performance evaluation of local descriptors[END_REF]:

1. Descriptor based distribution: they are the most popular descriptors in CBIR domain, as they are usually handcrafted to have some geometric and photometric invariance. The region appearance features are represented by an histogram of gradient locations and orientation or of Haar wavelet responses, e.g: SIFT [START_REF] Lowe | Distinctive image features from scale-invariant keypoints[END_REF], [START_REF] Bosch | Image classification using random forests and ferns[END_REF]. This descriptor known as "Scale Invariant Feature Trasnform" was proposed by Lowe in 2004. Since then, this descriptor is widely used by the researchers and it demonstrated an excellent performance in many fields, especially in object recognition domain. Nevertheless, the high dimensionality of SIFT is its main limitation [START_REF] Ledwich | Reduced sift features for image retrieval and indoor localisation[END_REF]: it prevents researchers from using it in retrieving images in large databases as ImageNet. 2. Differential descriptor: they are one of the first proposed methods to describe the regions of an image. The features in this family are represented by a set of derivatives computed up to given order, e.g: steerable filters [START_REF] Freeman | The design and use of steerable filters[END_REF]. The descriptors generated are vectors of very low dimension, e.g: 9 for steerable filters. According to Schmid [START_REF] Mikolajczyk | A performance evaluation of local descriptors[END_REF], although steerable filters was the best efficient descriptor among other low dimensional descriptors, its performance was very low against high dimensional descriptors as SIFT. Actually, they are still used to describe some biomedical images. 3. Textural descriptor: they describe the texture properties of the selected region, the descriptor is computed according to the type of texture of these regions: statistical, structural or random [START_REF] Jain | Fundamentals of Digital Image Processing[END_REF]. Thus, the type of the region texture must be known in advance: if it is statistical we can compute the descriptor using cooccurrence Matrix, Markov model, spatial frequency descriptors [START_REF] Zhang | Brief review of invariant texture analysis methods[END_REF]. In general, they are used in biomedical domain and document processing. 4. Others: other descriptors was proposed to describe features as: color, spectral nature of the image region. Van Gool [START_REF] Gool | Affine/ photometric invariants for planar intensity patterns[END_REF] proposed "Generalized moment invariant" which represents the multi-spectral nature of image region. It consists in computing central moment of high order and degree. However, these moments are sensitive to small geometric and photometric distortions.

Image representation Once these descriptors computed, they can serve as basis to construct the image representation. There are different proposals of image representation in the literature. In this paper, we focus on the most popular image representation in CBIR domain: bag-of-features representation [START_REF] Sivic | Video Google: A text retrieval approach to object matching in videos[END_REF].

It consists in encoding an image as a spatial histogram of visual words derived from a given vocabulary. The construction of this histogram starts by learning a k-means [START_REF] Lloyd | Least squares quantization in pcm[END_REF] visual vocabulary: it consists in partitioning the local descriptors into informative clusters µ 1 , .., µ k (visual words). Given a set of descriptors x 1 , ..., x k sampled from an image, each local descriptor is assigned to the corresponding visual word as given by equation 1 [START_REF] Zhou | Image classification using super-vector coding of local image descriptors[END_REF], resulting a non negative vector

f hist ∈ R k such that [f hist ] k = |{i : q i = k}|.
This vector represents the histogram encoding the local descriptor for each spatial region. Harris [START_REF] Harris | A combined corner and edge detector[END_REF] and its variants [START_REF] Mikolajczyk | Scale & affine invariant interest point detectors[END_REF] Blob detector It detects the regions which differ from their neighborhood in some properties, e.g: color, intensity

q ki = argmin k ||x i -µ k || 2 (1)
Hessian [START_REF] Beaudet | Rotationally invariant image operators[END_REF] and its variants

Region detector Based on segmentation methods MSER [START_REF] Matas | Robust wide baseline stereo from maximally stable extremal regions[END_REF] These spatial regions were obtained using spatial pyramid method [START_REF] Lazebnik | Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories[END_REF]: it consists in dividing the image into 1 × 1, 3 × 1 (three horizontal stripes), and 2 × 2 (four quadrants) grids, for a total of 8 regions. Once the encoding is computed for each region, L1 normalization is employed. Thus, the image representation is an additive combination of the region encoding [START_REF] Chatfield | The devil is in the details: an evaluation of recent feature encoding methods[END_REF]. After pooling, a chi-square feature map is applied to the entire image representation. To make this representation suitable for use within Chatfield linear SVM framework [START_REF] Chatfield | The devil is in the details: an evaluation of recent feature encoding methods[END_REF], L2 normalization is used and a linear SVM classifier is applied.

A lot of enhancements have been proposed to improve the performance of CBIR systems. Most propositions focused only on improving the image representation step [START_REF] Chatfield | The devil is in the details: an evaluation of recent feature encoding methods[END_REF]. We are interested in enhancing the first two components of a CBIR system: many experiment [START_REF] Schmid | Evaluation of interest point detectors[END_REF] [23] demonstrated that the region selection and appearance description methods can increase or decrease the capacity of a CBIR system on precising which category of object is in the image. In a previous work, we demonstrated that not all the points of interest extracted by interest point detector are pertinent for retrieving task: filtering 60% of points of interest using visual attention system didn't affect the performance of a CBIR system [START_REF] Awad | Saliency filtering of sift detectors: Application to cbir[END_REF].

Inspiring from this work, and according to psychophysical research which has given the evidence that the human brain does a frequency analysis of the image [START_REF] Campbell | Application of fourier analysis to the visibility of gratings[END_REF] [10], we propose a new hybrid descriptor which describes the spatial frequency of the perceptual features considered in a visual attention system: color, intensity, orientation. This idea is motivated by the fact that a lot of researchers used a combination of different types of descriptors to describe at the same time the color and the orientation features of a region of interest, and thus more computational complexity for their algorithm [START_REF] Everingham | The pascal visual object classes (voc) challenge[END_REF]. In addition, this descriptor can be considered as a first step to solve the limitations of using SIFT in large image databases as it has the advantage of being of lower dimension than SIFT: 60 vs 128, and thus computationally more efficient. In this context, we compare the impact of using these two descriptors on the performance of the CBIR system shown in figure 1 on VOC 2007 Challenge [START_REF] Everingham | The pascal visual object classes (voc) challenge[END_REF].

This paper is organized as follows. Section 1 presents in details the different proposed techniques for region selection and region appearance description. In section 2, we will present new hybrid descriptor, and its evaluation vs. SIFT for each region selection method presented in section 1.1.

1 Related works

Region Selection

In the literature, there are two selection strategies: the first one considers the entire image and use dense grid for this. Conversely, the second one selects image regions that are considered as pertinent to predict which object is in the image, and uses techniques as interest point detectors and visual attention systems. In the following, we will present these techniques:

Dense grid as mentioned before, there is no guarantee that the selected regions of interest will yield image representation that is optimal for solving a given image classification [START_REF] Nowak | Sampling strategies for bag-of-features image classification[END_REF]. Thus, researchers used dense grid: if the image is completely covered by regions, the whole image can be reconstructed from the set of selected regions, and therefore no information is lost [START_REF] Nowak | Sampling strategies for bag-of-features image classification[END_REF]. In our paper, we follow the approach of Bosch et al. [START_REF] Bosch | Image classification using random forests and ferns[END_REF]: our dense grid consists of dividing the image into small uniform regions with spacing M pixels, here M = 10 and the descriptors are computed at each grid points.

Interest point detectors Interest point detectors select the image patterns that differ from its immediate neighborhood and at the same time that are stable and robust to different image variations, e.g: contour, blobs. During the last decades, different detectors have been proposed. According to Schmid [START_REF] Tuytelaars | Local invariant feature detectors: a survey[END_REF], we can categorize them into three families (c.f. Table 1). An evaluation of these different detectors [START_REF] Schmid | Evaluation of interest point detectors[END_REF] demonstrated that Harris-Laplace is the detector the most efficient and informative. Thus, we decided using Harris-Laplace [START_REF] Mikolajczyk | Scale & affine invariant interest point detectors[END_REF] as the reference detector in our evaluation. As Harris-Laplace is dedicated only to corner-like region, we decided to use another detector Laplacian [START_REF] Lindeberg | Feature detection with automatic scale selection[END_REF], dedicated to blob-like region and considered as a complementary detector to Harris-Laplace.

Saliency-based detector-visual attention system some researchers used the bottom-up visual attention system as a new saliency-based detector. These systems scan the image for regions that attract our interest. In addition, they are validated against eye movements of human observers. Fig. 2. Perreira Da Silva visual attention system system as a filtering approach for points of interest extracted by Harris-Laplace Laplacian and they demonstrated that filtering 60% of these points didn't affect the performance of the reference CBIR system. Inspiring from this work, we decided using Perreira Da Silva visual attention system [START_REF] Da Silva | Objective validation of a dynamical and plausible computational model of visual attention[END_REF] as reference for saliency based detector. Perreira Da Silva system Perreira proposed a new competitive and hierarchical system that allows modeling the temporal evolution of the visual focus of the attention and its validation. As shown in figure 2, it is based on the classical algorithm proposed by Itti in [START_REF] Itti | Others: A model of saliency-based visual attention for rapid scene analysis[END_REF]: an input image I of width I w and height I h is subsampled into multi-resolution pyramids, and each pyramid level P t,σ is decomposed into low level features t for Red (R), Green (G), Blue (B), Yellow (Y ), Intensity (I on , I of f ) and local orientations (O 0 , O 45 , O 90 , O 135 ). From these channels, center surround feature maps f l for the different features are constructed and normalized. In each channel, maps are summed across scale and normalized:

f l = σ P t,σ (2)
with σ ∈ {1, .., N -1}, N = 1 + log 2 (min (I w , I h )). These maps are summed and normalized once to yield three conspicuity maps. These three conspicuity maps are representative of the three main human perceptual channels: color, intensity and orientation. Perreira Da Silva et al. propose to substitute the second part of Itti's model by an optimal competitive approach: a preys / predators system. [START_REF] Da Silva | Evaluation of preys / predators systems for visual attention simulation[END_REF] shows that despite the non deterministic behavior of preys / predators equations, the system exhibits interesting properties of stability, reproducibility and reactiveness while allowing a fast and efficient exploration of the scene. We applied the same optimal parameters used by Perreira Da Silva in [START_REF] Da Silva | Evaluation of preys / predators systems for visual attention simulation[END_REF] to evaluate our approach. The output of this algorithm is a saliency map, computed by a temporal average of the focalization computed through a certain period of time t.

In the following, we tackle the descriptor used as reference in our evaluation.

Region appearance description

After extracting regions of interest, each region is described by a set of scalar values representing one of the image region features. In this paper, we used SIFT [START_REF] Lowe | Distinctive image features from scale-invariant keypoints[END_REF] as it is one of the most popular and used descriptor in CBIR domain. This can be explained by the fact that it provides descriptors robust to scale changes and small geometric distortions. Therefore, we used original SIFT to describe the points of interest computed by interest points detectors. For dense grid, at each grid point, SIFT descriptor are computed over circular support patches with radii=4, 8, 12 and 16 pixels [START_REF] Bosch | Image classification using random forests and ferns[END_REF]. After computing the descriptor, the rest of typical CBIR remains the same. In the next section, we present our approach for region appearance description. As mentioned in the introduction, we propose a new hybrid texture-perceptual descriptor that describes the spatial frequency of the three perceptual features: color, orientation, intensity.

Our proposition: textural-perceptual descriptor

In this section, we present a new texture-perceptual descriptor that analyze the frequency content of the perceptual features in a multi-resolution pyramids computed in the first phase of Perreira's system (c.f 1.1). Usually, spatial frequency is analyzed using spatial frequency descriptors. These descriptors are based on image transform matrix as Fourier [START_REF] Campbell | Application of fourier analysis to the visibility of gratings[END_REF]. Fourier transform is one of the most powerful descriptor in texture analyzing domain. For our descriptor, a transform closely related to Fourier called Hartley transform H k,l is used: it contains the same information that Fourier does. In addition, contrary to Fourier, it has the advantage of being a real function and this offer computational advantages in signal processing application [START_REF] Millane | Analytic properties of the hartley transform and their implications[END_REF]. As shown in figure 3, Hartley transformation matrix of size M × M is computed as given:

H k,l = M -1 l,k=0 cos 2πlk M + sin 2πlk M (3)
where k, l ∈ {0, ..., M -1}. In general, Fourier descriptor is blamed for not efficient in capturing a local features [START_REF] Zhang | A comparison of shape retrieval using fourier descriptors and short-time fourier descriptors[END_REF]. Several researchers have proposed methods attempting to overcome this drawback. According to Unser, the local texture property of an image region can be characterized by a set of energy measures computed at the output of a filter bank [START_REF] Unser | Local linear transforms for texture measurements[END_REF]. In this context, Unser proposed an interesting way to exploit the Fig. 3. Example of computing our descriptor features using one of the multi-resolution pyramids spatial dependencies that characterize the texture of a region, more computationnally efficient called local linear transform [START_REF] Unser | Local linear transforms for texture measurements[END_REF]: it consists in computing for each point of interest x k,l , a local linear property y k,l as given in equation:

y k,l = l,k=0,..M -1 T M .x k,l (4)
where T M represents an image transform matrix of size M × M . In our case, T M represents Hartley transform H k,l . Extending this method, each point of interest x k,l is tracked back in the Perreira's system to multi-resolution feature pyramids computing phase. For each pyramid level P t,σ , a neighborhood window w x k,l of the same size M × M as Hartley matrix, is centered around x k,l . In our method, y k,l represents texture energy measures and it is defined by:

y c k,l = y k,l = l,k=0,..M -1 w x k,l .T c M 2 (5)
where T c M as shown in Fig. 4 is the convolution of each column with each row in the matrix H k,l and c ∈ 1, ..., M 2 . By combining the M 2 channels as given in the equation 6, we obtained M × M +1 2 channels(noted TR) invariant to some rotation transformations [START_REF] Rachidi | Application of laws masks to bone texture analysis: An innovative image analysis tool in osteoporosis[END_REF]:

T R k,l = y k,l + y l,k 2 (6)
To provide better visual perception, a contrast enhancement is applied using equation 7 [START_REF] Ritika | Article: Contrast enhancement techniques for images -a visual analysis[END_REF], resulting an histogram f t i of dimension M × M +1 2 computed for each multi-resolution feature pyramid level P t,σ : and > 0 is a suitably small value(we use = 0.05). Concatenating these histograms for each point of interest x k,l the results in a descriptor of dimension P × M × M +1 2 with P =number of multi-resolution pyramids. In our evaluation, M = 3 and P = 10, resulting a descriptor of dimension 60. In the next section, we present the evaluation of this hybrid descriptor vs. SIFT.

f i = log ( ) -log (T R k,l + ) log ( ) -log ( + 1) (7

Evaluation of our descriptor vs. SIFT

This section describes the experimental evaluation of the descriptor mentioned above vs. SIFT on VOC 2007 dataset. This dataset contains about 10,000 images split into train, validation, and test sets, and labelled with twenty classes. Observing these results shows that for traditional region selection methods, althougth that the dimension of our descriptor is reduced to the half regardless SIFT, thus gain in run-query time and in memory (c.f table 3), only an average of 5% loss of performance is obtained. Furthermore, For perceptual region selection methods, about 9% gain of performance is obtained using our descriptor regardless SIFT. This results validate our hypothesis concerning the fact our descriptor can be considered as a first step to solve the memory and time consuming by the use of high dimensional SIFT descriptors in object recognition systems. Indeed, this consumption of memory constitute a serious problem in object recognition, especially with large image bases. Furthermore, we suggest that by using our texture-perceptual descriptor, we describe not only the features of region of interest that are are relevant for human perception, but also the features which are relevant for CBIR systems (color, orientation).

Conclusion

ln this paper, we proposed a new perceptual-texture descriptor, more computationally efficient which describes different perceptual features for regions of interest and at the same time lower dimension than SIFT descriptor, approximately the half. Furthermore, it performs approximately the same as SIFT (with 5% loss in performance) on different region selection methods. In the future, it would be interesting to build an interactive perceptual CBIR system. We think it can be done by using a suitable image representation as [START_REF] Sanchez | Image Classification with the Fisher Vector: Theory and Practice[END_REF] and incorporating relevance feedback step by taking advantage of top-down techniques already implemented in the visual attention system.

Fig. 1 .

 1 Fig. 1. pipeline of content based image retrieval algorithm

)Fig. 4 .

 4 Fig. 4. Computation of a running 3x3 Hartley by successive row and column filtering

Table 1 .

 1 Interest detector category

	Interest point de-	Description	The detector the
	tector family		most used in this
			family
	Corner detector It detects the points
		which corresponds to
		rough variation in image
		intensity

Table 2 .

 2 Impact descriptors for each region selection method on image classification results using Pascal VOC 2007 dataset (%) -vs-rest classifier is learned for each class and the performance is evaluated by Average Precision. As mentioned before, in this experiment we study the impact of each descriptor for each region selection method mentioned in section 1.1: dense grid, Harris Laplace + Laplacian, Perreira's system. The rest of the CBIR system remains the same. The results are shown in table 2.

	Region	descriptoraero-	bike	bird	boat bottle bus	car	cat	chair cow	dining	dog	horse motor-	person potted	sheep sofa	train tv
	selection		plane									table		bike	plant	moni-
	methods														tor
	Dense	SIFT	69,58 56,28 40,63 64,17 24,56 60,63 75,06 56,17 49,19 38,90 50,54 36,79 75,69 63,59 81,51 26,55 45,21 46,78 74,36 50,10
	grid
	Dense	our de-	68,02 44,96 36,90 58,99 21,98 46,57 63,84 43,67 41,53 27,07 51,37 34,26 70,13 54,41 81,29 35,08 46,44 47,93 68,23 45,91
	grid	scriptor
	Harris-	SIFT	61,45 48,12 37,76 51,27 21,34 47,30 65,60 46,16 41,45 31,62 35,85 28,37 67,62 50,95 73,06 15,21 29,74 22,51 62,95 37,12
	Laplace,
	Lapla-
	cian
	Harris-	our de-	59,90 36,80 34,03 46,09 18,76 33,24 54,38 33,66 33,79 19,79 36,68 25,84 62,07 41,77 72,83 23,74 30,97 23,66 56,81 32,93
	Laplace,	scriptor
	Lapla-
	cian
	Perreira's	SIFT	19,17 10,51 17,34 7,51	6,15	9,35	26,55 21,14 13,12 13,33 5,46	21,65 21,64 18,90 50,78 6,22	3,72	7,41	10,28 10,28
	system
	Perreira's	our de-	44,72 22,28 21,68 14,23 10,99 18,66 44,41 25,62 19,59 8,37	12,96 24,96 36,38 23,72 63,32 20,03 11,03 12,94 32,72 21,36
	system	scriptor
	A 1

Table 3 .

 3 computational complexity of descriptors with N is the number of points of interest

	Region appearance description methods SIFT our descriptor
	Computational complexity	O(N 2 )	O(N )