Zenonas Theodosiou
email: zenonas.theodosiou@cut.ac.cy

Nicolas Tsapatsoulis
email: nicolas.tsapatsoulis@cut.ac.cy

Modelling Crowdsourcing Originated Keywords within the Athletics Domain

Keywords: Crowdsourcing Annotation, Keyword Modelling, Image Classification

Image classification arises as an important phase in the overall process of automatic image annotation and image retrieval. Usually, a set of manually annotated images is used to train supervised systems and classify images into classes. The act of crowdsourcing has largely focused on investigating strategies for reducing the time, cost and effort required for the creation of the annotated data. In this paper we experiment with the efficiency of various classifiers in building visual models for keywords through crowdsourcing with the aid of Weka tool and a variety of lowlevel features. A total number of 500 manually annotated images related to athletics domain are used to build and test 8 visual models. The experimental results have been examined using the classification accuracy and are very promising showing the ability of the visual models to classify the images into the corresponding classes with the highest average classification accuracy of 74.38% in the purpose of SMO data classifier.

Introduction

Image tagging helps search engines to better retrieve desired images in response to text queries. Automatic image annotation has concentrated on the difficulty of relating high-level human interpretations with low-level visual features. The interpretation inconsistency between image descriptors and high-level semantics is known as the semantic gap [START_REF] Smeulders | Content-based image retrieval at the end of the early years[END_REF] or the perceptual gap [START_REF] Jaimes | Multimedia information retrieval: what is it, and why isn't anyone using it?[END_REF]. This is due to the fact that the visual image features extracted from an image cannot be automatically translated reliably into high-level semantics [START_REF] Datta | Image retrieval: Ideas, influences, and trends of the new age[END_REF]. A manually annotated set of multimedia data is used to train a system for the identification of joint or conditional probability of an annotation occurring together with a certain distribution of multimedia content feature vectors [START_REF] Athanasakos | A Framework for Evaluating Automatic Image Annotation Algorithms[END_REF]. Different models and machine learning techniques are developed to learn the correlation between image features and textual words from the examples of annotated images and then apply the learned correlation to predict words for unseen images [START_REF] Zhang | A probabilistic semantic model for image annotation and multi-modal image retrieval[END_REF].

On the other hand, manual image tagging annotation is an extremely difficult and elaborate task and cannot always be considered as correct due to visual information that always lets the possibility for more individual interpretation and ambiguity [START_REF] Volker | Modelling human judgment of digital imagery for multimedia retrieval[END_REF]. Crowdsourcing [START_REF] Howe | Crowdsourcing: Why the Power of the Crowd Is Driving the Future of Business[END_REF] has magnetized the interest of several researchers, since it is a very attractive solution to the problem of cheaply and quickly acquiring annotations and has a potential to improve evaluation of information retrieval systems by scaling up relevance assessments and creating test collections with more complete judgments [START_REF] Kazai | Crowdsourcing for book search evaluation: Impact of quality on comparative system ranking[END_REF]. Amazon Mechanical Turk [START_REF]Amazon Mechanical Turk -Artificial Artificial Intelligence[END_REF] opened a new way of satisfying the need for large collections of human-annotated data as presented in the recent past [START_REF] Eickhoff | How Crowdsourcable is Your Task?[END_REF] by extending the interactivity of crowdsourcing tasks using more comprehensive user interfaces and micro-payment mechanisms.

In this paper we get the advantage of the availability of a large dataset related to the athletics domain created during the FP6 Boemie project [START_REF]BOEMIE -Bootstrapping Ontology Evolution with Multimedia Information Extraction[END_REF] and deal with the experimental evaluation of the efficiency of various low-level features and data classifiers in modelling crowdsourcing originated keywords. A set of images was annotated by 15 users using a predefined set of keywords. Images sharing a common keyword are grouped together and used to create the visual model which corresponds to this keyword. Eight different keyword models are created using low-level features and tested with the aid of well known data classifiers. We have used publicly available tools for the computation of the low level features [START_REF]MPEG-7 Visual Experimentation Model (XM)[END_REF], [START_REF] Ludwig | Trainable Classifier-Fusion Schemes: An Application To Pedestrian Detection[END_REF] and the model creation (the Weka tool [START_REF] Witten | Data Mining: Practical machine learning tools and techniques. 2nd Edition[END_REF]) and classified the images into 8 keyword classes.

This paper is organized as follows: Section 2 gives a detailed description of the method we have followed to create the dataset and build the visual models while Section 3 presents and discusses the experimental results. Finally, conclusions are drawn and further work hints are given in Section 4.

Method Overview

This section presents the method we have followed to model the crowdsourced keywords. It consists of 3 main steps: the dataset creation, the feature extraction, and the keyword modelling. The overall procedure is illustrated in Fig. 1.

Dataset Creation

The crowdsourcing annotation was based on a randomly selected set of 500 images taken from a large dataset created during the BOEMIE project [START_REF]BOEMIE -Bootstrapping Ontology Evolution with Multimedia Information Extraction[END_REF]. The dataset was manually annotated by 15 users using the MuLVAT annotation tool [START_REF] Theodosiou | MuLVAT: A Video Annotation Tool Based on XML-Dictionaries and Shot Clustering[END_REF] with the aid of a structured xml dictionary consists of 33 different keywords. For our experiments we have selected 8 representative keywords and for each keyword, 50 images that were annotated from more than 5 annotators with this keyword were grouped together to create a set of 8 different groups of images (Fig. 2).

Feature Extraction

Among the possible low-level features that can be extracted from the image groups, we have chosen the following popular and widely used features:

Histogram of Gradients Features. The HOG features exploit the idea that local object appearance can be described by the distribution of intensity gradients or edge directions. The image is divided into small connected regions, called cells. For each cell, a histogram of gradient directions or edge orientations within this cell is compiled. For the implementation of HOG, each pixel within the cell casts a weighted vote for an orientation-based histogram channel. For the current study we have used the implementation proposed in [START_REF] Ludwig | Trainable Classifier-Fusion Schemes: An Application To Pedestrian Detection[END_REF] with the aid of 25 rectangular cells and 9 bins histogram per cell. The 16 histograms with 9 bins were then concatenated to make a 225-dimensional feature vector.

Scale-Invariant Feature Transform features. SIFT transforms image data into scale-invariant coordinates relative to local features and performs a set of features that are not affected by object scaling and rotation. Key points are detected as the maxima of an image pyramid built using difference-of-Gaussians.

The multi-scale approach results in features that are detected across different scales of images. For each detected key-point, a 128 dimensional feature vector is computed describing the gradient orientations around the key-point. The strongest gradient orientation is selected as reference, thus giving rotation invariance to SIFT features. For our experiments each SIFT vector is quantized into a 100-dimensional feature vector using k-means clustering. MPEG-7 features. MPEG-7 visual descriptors include the color, texture and shape descriptor. A total of 22 different features are included, nine for color, eight for texture and five for shape. The dominant color features include color value, percentage and variance and require especially designed metrics for similarity matching. Furthermore, their length is not known a priori since they are image dependent (for example an image may be composed from a single color whereas others vary in color distribution). The previously mentioned difficulties cannot be easily handled in machine learning schemes, therefore we decided to exclude these features for the current experimentation. The texture browsing features (regularity, direction, scale) have not been included in the description vectors since in the current implementation of the MPEG-7 experimentation model [START_REF]MPEG-7 Visual Experimentation Model (XM)[END_REF] the corresponding descriptor cannot be reliably computed (it is a known bug of the implementation software). The scalable color and shape descriptor features have been also excluded because vary depending on the form of an input object and can not be used for the holistic image description. Among all MPEG-7 descriptors only the Color Layout (CL), Color Structure (CS), Edge Histogram (EH) and Homogenous Texture (HT) descriptors are used in our experiments. The combination of the selected descriptors creates a 186-dimensional feature vector.

Keyword Modelling

To overcome the multiclass classification problem and facilitate effective and efficient learning, each keyword is treated as a separate binary classification problem. We have followed the one-against-rest approach [16] and we have built a total number of 8 models, one for each keyword. The feature vectors of each keyword class were split into two groups, called the training (80%) and testing (20%) set. Each model is trained and tested between one class and the 7 other classes. The training and testing set for each model contain the feature vectors of the corresponding keyword class and the same number of randomly selected feature vectors of the the rest 7 classes. Keywords models were created using Weka tool [START_REF] Witten | Data Mining: Practical machine learning tools and techniques. 2nd Edition[END_REF]. Since statistical methods have their limitations, particularly in relation to distributional assumptions and to the restrictions on data input, we have decided to use artificial intelligence classifiers such as Support Vector Machines (SVM) and Decision Trees (DT).

Support Vector Machines.

SVM separates the classes with a decision surface that maximizes the margin between the classes. The surface is often called the optimal hyperlane and the data points closer to the hyperplane are called support vectors. For our experiments we decided to use two of the state of the art implementations of the Support Vector Machines (SVMs), the SMO [START_REF] Platt | Fast Training of Support Vector Machines using Sequential Minimal Optimization[END_REF] and the LibSVM [START_REF] Fan | Working set selection using the second order information for training SVM[END_REF]. They have been reported in several publications as the best performing machine learning algorithms for a variety of classification tasks. The performance of SVM classifiers can vary significantly with variation in parameters of the models. During training we experimented with different parameters and kernels and for each kernel we built models for several combinations of the parameters, with the Pearson VII universal and polynomial kernel performing better than the others for the SMO and LibSVM classifier respectively.

Decision Trees.

Unlike other classification approaches that use a set of features jointly to perform classification in a single decision step, the decision tree is based on a multistage or hierarchical decision scheme or a tree like structure. The tree is composed of a root node (containing all data), a set of internal nodes (splits) and a set of terminal nodes (leaves). Each node of the decision tree structure makes a binary decision that separates either one class or some of the classes from the remaining classes. The processing is generally carried out by moving down the tree until the leaf node is reached. Turning to the classifiers, Random Forest [START_REF] Breiman | Random Forests[END_REF] and Logistics Model Tree (LMT) [START_REF] Landwehr | Logistic Model Trees[END_REF] have been employed to model the keywords.

Experimental Results

We used the dataset and keyword modelling process described in the previous Section to examine the performance and effectiveness of the created models: "Discus", "Hammer", "High Jump", "Hurdles", "Javelin", "Long Jump", "Running", and "Triple Jump". Fig. 3, 4, 5, 6 show the accuracy of correctly classified instances for all classes using the various data classifiers. The results shown in these figures were examined under three perspectives: First, in terms of the efficiency of the various classifiers in modelling crowdsourced keywords, second in terms of the efficiency of the low-level features to create accurate visual models and third, in terms of the ability of the created models to classify the images into the corresponding classes.

The efficiency of the training algorithms is examined through the effectiveness of the created models, the time required to train the models and the robustness to the variation of learning parameters. The SMO and Random Forest classifiers require by far the lower time and effort to create an effective model, while LMT is the slowest classifier among all. In the case of SMO and Random Forest the learning takes no more than a few seconds for the majority of the keyword models. Furthermore, the fluctuation in classification performance during parameters tuning is significantly lower than that of the LibSVM and LMT. There is a significant difference on the performance of the models created using the individual classifiers. It is evident from Table 1 that SMO is the most reliable classifier with a total average classification accuracy equal to 74.38%. The Lib-SVM and Random Forest give the same average classification accuracy with the difference that LibSVM performs better for the HOG while the Random Forest performs better for the MPEG-7 features. The most disappointing classifier is the LMT which has the worst average classification accuracy values. The best classification accuracy was occurred in the case of SMO classifier using MPEG-7 features and the worst in the case of LMT classifier using SIFT features.

Concerning the efficiency of the various low-level features, the experimental results indicate that the MPEG-7 features perform better than HOG and SIFT. The classification accuracy obtained using these features is quite good in comparison with the other two and has average classification accuracy values in the range of 71.25%-81.25%, with the lowest value given by LMT and the highest by SMO classifier respectively. The second more reliable low-level features for modeling keywords are the HOG that can obtain the maximum average classification accuracy value of 72.5% in the purpose of the SMO classifier. The most disappointing classification performance is achieved by the SIFT which can obtain the maximum value of 75% only for "Discus", "Hammer" and "Hurdles" classes. Nearly all models are able to classify the images into the corresponding classes with classification accuracy in the range of 55%-95%. The best efficiency is per- ceived when testing models created by images having objects with a well defined shape such as "Discus" and "Hurdles". As a consequence, the worst results are occurred when testing the "Running" and "Triple Jump" because the content of images belong to these keywords has many similarities with the content of images belong to other keywords. We present an experimental evaluation of modelling crowdsourcing originated keywords within the athletics domain. Specifically, 8 different keywords were modeled using various low-level features and data classifiers. According to our experimental results, nearly all created models can classify the images into the 8 classes with medium to high classification accuracy. Although there is a significant variation on the efficiency of the various classifiers with the SMO having the highest performance, a great improvement can achieved when the MPEG-7 features are used. Our future perspectives involve the evaluation of the proposed method on larger and different datasets as well as the experimentation of additional training algorithms and other classifications schemes. In addition, the efficiency of more low-level features in creation of visual models will be investigated.

Fig. 1 .

 1 Fig. 1. The flowchart of overall method.

Fig. 2 .

 2 Fig. 2. The set of image groups.

Fig. 3 .

 3 Fig. 3. Accuracy of the correctly classified instances using the SMO data classifier.

Fig. 4 .

 4 Fig. 4. Accuracy of the correctly classified instances using the LibSVM data classifier.

Fig. 5 .

 5 Fig. 5. Accuracy of the correctly classified instances using the Random Forest data classifier.

Fig. 6 .

 6 Fig. 6. Accuracy of the correctly classified instances using the LMT data classifier.

Table 1 .

 1 Average classification accuracy values (%) for the different classifiers.

	Classifier HOG SIFT MPEG-7 Overall
	SMO	72.5 69.38	81.25	74.38
	LibSVM	73.13 68.13	71.25	70.83
	Random Forest 70.0 69.38	73.13	70.83
	LMT	71.88 61.25	72.5	68.54
	4 Conclusions and Future Work

Acknowledgements: This work falls under the Cyprus Research Promotion Foundation's Framework Programme for Research, Technological Development and Innovation 2009-2010 (DESMI 2009-2010), co-funded by the Republic of Cyprus and the European Regional Development Fund, and specifically under Grant ΠEN EK/0609/95.


Eleni Vrochidou

Petros Alvanitopoulos

Ioannis Andreadis

Anaxagoras Elenas

Correlation between Seismic Intensity Parameters of HHT-Based Synthetic Seismic Accelerograms and Damage Indices of Buildings

Keywords: Artificial seismic signals, Hilbert-Huang transform, correlation, seismic parameters, damage indices

In this work, a correlation study between well-known seismic parameters and damage indices is presented. The correlation analysis is first performed on a set of natural seismic signals and afterwards on a set of artificial accelerograms generated from natural records. The artificial seismic signals are generated by a combination of techniques; the Hilbert-Huang transform and an optimization algorithm. In addition, they are compatible with the design spectra of a chosen seismic area. Results reveal that the seismic parameters of the synthetic earthquake accelerograms provide the same degree of correlation with the used damage indices as the natural earthquakes. Thus, the proposed synthetic accelerograms technique appropriately represents a seismic event and, therefore, it is a useful tool in earthquake engineering.

Introduction

In order to examine the resistance of a building, a dynamic analysis is required. Disaster scenarios need seismic accelerograms that could lead the building to the limits of its strength. Accelerograms are signal records of the acceleration versus time measured during an earthquake ground motion. Unfortunately, there is lack of natural strong motion earthquake records for some areas. Thus, there is need of artificial seismic records compatible with the design spectra of the area under observation. Methods for generating artificial earthquakes have been reported in the literature [START_REF] Suarez | Generation of artificial earthquakes via the wavelet transform[END_REF][START_REF] Mavroeidis | A Mathematical Representation of Near-Fault Ground Motions[END_REF][START_REF] Das | Wavelet-based simulation of spectrum-compatible aftershock accelerograms[END_REF][START_REF] Spanos | Synthesis of accelerograms compatible with the Chinese GB 50011-2001 design spectrum via harmonic wavelets: artificial and historic records[END_REF]. The presented methodology uses the Hilbert-Huang transform (HHT) which appears to be an effective technique for analyzing non-stationary and non-linear signals such as seismic signals [START_REF] Huang | The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis[END_REF].

The HHT decomposes the initial signal into a number of intrinsic mode functions (IMFs) and presents the results to the energy-frequency-time field, designing the socalled Hilbert spectrum. The IMFs extraction is based on the local characteristics of the signals and not on some predetermined functions. This is the key feature characterizing the HHT providing a more physical meaning to the analysis of the examined signals [START_REF] Huang | The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis[END_REF]. Moreover, the finite number of the extracted IMFs, decreases the computational complexity of the algorithm. An additional advantage of the method is that the initial seismic signal used for the synthesis of the artificial signal, could be of any intensity and could belong to any region. This provides flexibility, since there can be selected signals with finite number of points and decrease even more the complexity of the algorithm.

The main challenge is to generate realistic simulated earthquake records with the characteristics of natural signals. The proposed methodology is applied to an eightstorey reinforced concrete frame structure. Both seismic parameters and damage indices have been calculated from natural and artificial signals. A comparison between them is made and correlation analysis demonstrates that the generated earthquake signals have the same behavior as the natural ones, and have approximately the same distribution of values among the seismic parameters and the damage indices. For example, the spectrum displacement and spectrum velocity are correlated with the global damage index of Park/Ang by 0.905 and 0.929, respectively for the natural accelerograms, while for the artificial accelerograms the values are 0.905 and 0.905, respectively. Additionally, the correlation study reveals that the seismic parameters that are known to be most correlated with the damage indices [START_REF] Elenas | Correlation study between seismic acceleration parameters and damage indices of structures[END_REF] have the same degree of correlation for both natural and artificial signals.

Earthquake Engineering

The intensity of a seismic accelerogram is expressed with the help of several seismic parameters that are connected to the damages caused to a specific structure [START_REF] Elenas | Correlation study between seismic acceleration parameters and damage indices of structures[END_REF]. Damages are described by the damage indices (DIs). The maximum inter-storey drift ratio (MISDR) and the global damage index of Park/Ang (DI P/A ) are used to characterize the structural damage caused to buildings.

Damage Indices

The level of the post-seismic corruption of a structure can be evaluated by the MISDR [START_REF] Elenas | Correlation between seismic acceleration parameters and overall structural damage indices of buildings[END_REF] and is given by the following equation:

[ ] % 100 max h u MISDR = (1)
where max u is the maximum absolute inter-storey drift and the inter-storey height. In addition, the global DI after Park/Ang is also used as a damage index [START_REF] Elenas | Correlation between seismic acceleration parameters and overall structural damage indices of buildings[END_REF]:

h ∑ ∑ = = = n i i n i i L A P E E DI DI 0 0 / (2)
where is the energy at the location i , is the number of location where the energy is estimated and is the local DI given by the equation:

Concrete Frame Structure

The examined reinforced concrete structure is depicted in Figure 1. The eigenfrequency of the 8-storey building is 0.85 Hz and its design is based on the recent Eurocode rules EC2 and EC8. The cross-sections of the beams are T-beams with 40 cm width, 20 cm slab thickness, 60 cm total beam height and 1.45 m effective slab width. The distance between the frames of the structure is 6 m. The structure has been characterized as an "importance class II and ductility class medium"-structure according to the EC8 Eurocode. The subsoil is of type C and the region seismicity of category 2. External loads are taken under consideration and are incorporated into load combinations due to the rules of EC2 and EC8.

Synthesis of Artificial Accelerograms

The proposed synthesis of artificial seismic signals is based on the HHT. According to HHT, each natural earthquake record can be decomposed into finite frequency and amplitude components. These components are appropriately modified to compose the artificial signal. This is achieved through an optimization algorithm which is used to minimize the mean square error between the response spectrum of the synthetic earthquake and the design spectrum of the selected seismic area.

The Hilbert-Huang Transform

The HHT [START_REF] Huang | The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis[END_REF] includes the Empirical Mode Decomposition (EMD) and the Hilbert Spectral Analysis (HSA).

The EMD decomposed the signal into a finite number of IMFs. The decomposition is based upon the local characteristics of the signal, thus it is an appropriate method for non-linear and non-stationary data analysis. Two conditions must be fulfilled for the generated IMFs:

1. the number of extrema and zero-crossings must be equal or differ the most by one and 2. at any point, the mean value of the envelop that is defined by the local maxima and minima must be equal to zero. For a seismic signal , all the local extrema must be connected by a cubic spline so as to form the upper envelope of the signal and the same process is

) (t X
) ( max t y followed for the lower envelope . The signal must be enclosed between these two envelopes. The mean value of the envelopes is:

) ( min t y 2 )) ( ) ( ( max t y = ) ( ) ( t X t ) ( min 1 t y t m - (4)
and the difference between the signal and the is the first component :

) ( 1 t m 1 h (5) ) ( 1 1 t m h - =
The procedure continues and now is considered to be the signal, so:

) ( 1 t h ) ( 1 ) ( t h t (6) ) ( 11 11 t m h - =
where is the new mean of the two envelopes of . The same process is repeated for times and finally the is:

) ( 11 t m k ) ( 1 t h ) ( 1 t h k ) 1 ( 1 k h ) ( 1 ) ( ) ( 1 t k m t t k h - - = (7)
where the is the first IMF. The process finishes when the standard deviation

) ( ) ( 1 1 t c t h k = 2 . 0 ≤ SD : ∑ = - - - h k h 0 2 ( 1 ) 1 ( 1
[ = T t t k t k h t SD ) ( ) 1 2 )] ( 1 ) ( (8)
where T is the duration of the seismic signal. The residue is derived by subtracting the first IMF from the initial signal:

) ( 1 t r (9) ) ( 1 1 t c r ) ( ) ( t X t = -
and it is considered as the new data. The new data is subjected to the same process until all functions are obtained:

) (t r j (10) . ,..., 3 , 2 ), ) ( n j r t r j j = = ( ) ( 1 t c t j - - ( ) 1 c n j j = ∑ =
The iteration process stops when one of the two criteria is fulfilled:

1. either the residue

) (t r n or the component ) (t c n
is less than a predetermined value or 2. the residue

) (t r n
is a monotonic function and no other IMF can be derived. Finally, the initial signal is the sum of all IMFs plus the residue:

) ( ) ( t r t t X n + (11)
The Hilbert Spectral Analysis derives the instantaneous frequency data and forms the energy-frequency-time distribution by applying the HHT to each IMF. For all components , the Hilbert transform is:

) (t c j τ τ τ π d t c t j y j ∫ +∞ ∞ - - Ρ = ) ( 1 ) ( (12)
where is the Cauchy Principal Value. The combination of the IMF and its

Hilbert transform form an analytic signal :

Ρ ) (t c j ) (t y j ) (t z j ) ( ) ( ) ( ) ( ) ( t i j j j j j e t a t iy t c t z θ = + = ( 13)
where the amplitude and the phase

) (t a j ) (t θ are: ) ( ) ( ) ( 2 2 t y t c t a j j j + = (14) ) ( ) ( 1 tan ) ( t j c t j y t j - = θ (15)
The instantaneous frequency is given by the equation:

dt t d t f t j j ) ( ) ( 2 ) ( θ π ω = = (16)
The IMF components are defined as follows:

) ( cos ) ( ) ) ( ) ( Re( ) ( t j t j a t j i e t j a t j c θ θ = = (17)
where is the real part. The initial signal can be written:

Re()

∑ ∫ = = n j j j dt t f t a t X 1 )) ) ( 2 cos( ) ( Re( ) ( π (18)
In the previous equation, the residue is not invoked as it is a constant or a monotonic function. According to this equation both amplitude and frequency can be expressed as functions of time. The time-frequency distribution of the amplitude is known as the Hilbert spectrum ). , ( t H ω

Method for Generating Artificial Seismic Accelerograms

The objective of the proposed method is to supply earthquake engineering analysts with artificial seismic accelerograms compatible with the design spectrum of the current Greek antiseismic code [START_REF]EAK: National Greek Antiseismic Code. Earthquake Planning and Protection Organization (OASP) Publication[END_REF] and it is a two step procedure.

The first step applies the HHT to the initial seismic signal which is decomposed into IMFs. The HHT amplitude and frequency components of the initial signal are then properly modified to form an artificial seismic signal which is given by the equation:

n ) , ( t x AE ∑ = ∫ + = n j dt t ix j j j j n e t a x t x AE 1 ) ( )) ( ( Re ) , ( ω (19)
where are the amplitude scaling parameters and are the frequency modifying parameters. The following equations describe the amplitude scaling and the frequency modification:

n x x x ,..., , 2 1 n n n x x x 2 2 1 ,..., , + + ) ( ) ( t a x t a i j i = , n j ,..., 2 , 1 = (20) k i i x t t + = ) ( ) ( ω ω , n n k 2 ,..., 1 + = (21)
The second step solves the optimization problem so as the response spectrum of the synthetic signal to be compatible with the design spectrum of the area understudy. The optimization problem is actually finding the appropriate parameters x={x 1 ,x 2 ,...,x 2n } of the equations ( 20) and ( 21) to match the aforementioned spectrum. In particular, the genetic algorithm used, tries to minimize a fitness function which in our case is the Mean Square Error (MSE) between the two spectra:

∑ = - = n j et t generated SA SA n MSE 1 2 arg 1 (22)
where is the number of sample cases.

Correlation Coefficients

Correlation analysis proves the statistical relationship between two sets of data. There are many correlation coefficients; among them the most widely used are the Pearson correlation coefficient and the Spearman rank correlation coefficient.

The Pearson correlation shows how well the data fit a linear relationship, while the Spearman correlation shows how close the examined data are to monotone ranking. The latter coefficient is more important in the present study. For a set of measurements of X and Y, where

n n i ,..., 2 , 1 =
, the sample correlation coefficient is given by the equation: where is the difference between the ranking degree of X and Y, respectively.

D

In case the correlation coefficient has a value greater than 0.8 means that there is a strong connection between the parameters. On the other hand, values less than 0.45 demonstrate a weak connection between the parameters. All other cases between 0.45 and 0.8 reveal medium connection [START_REF] Elenas | Correlation study between seismic acceleration parameters and damage indices of structures[END_REF].

Numerical Results

In this study, twenty natural earthquake records were used as initial signals in order to generate synthetic accelerograms. Information about these seismic events is included in Table 2. Twenty artificial seismic accelerograms have been generated from these seismic signals. The objective was that the response acceleration spectrum of every generated signal to be compatible with the Greek design spectrum. Figure 2 displays the results obtained for the Friuli (Feltre station) earthquake record. shows the original signal used to generate the synthetic earthquake signal that is shown in Figure 2(b). Figure 2(c) presents the response spectrum of the original signal, the response spectrum of the synthetic accelerogram and the Greek design spectrum of the considered seismic area. It is obvious from Figure 2(c) that the response spectrum of the generated signal is compatible with the desired design spectrum. This is also justified numerically, as the MSE for this experiment was 1.73x10 -3 which is significantly small. For this event, the response spectrum of the original accelerogram was lower than the design spectrum considered.

Results are also promising for original signals that their response spectra are higher than the design spectrum. In this category belongs the seismic event of Dinar. The seismic parameters were calculated for all 20 original and artificial seismic accelerograms. Afterwards, the considered global damage indices were evaluated for each accelerogram, considered as seismic load on the examined frame structure. Finally, the Spearman rank correlation coefficient between the seismic parameters and the damage indices is calculated and presented in Table 3. As it can be seen from the numerical results of Table 3, the values of correlation are approximately the same for the natural acceleration records and artificial generated accelerograms. For example, the parameter SA for the natural signal is correlated with the MISDR and the DI P/A by 0.833 and 0.881, respectively. The same parameter for the artificial signals is correlated with the same damage indices by 0.762 and 0.833 respectively, which is almost the same degree as the natural records.

The degrees of interdependences for all 8 parameters are in the same range of values for natural and artificial signals. This indicates that the proposed method produces realistic simulations of earthquake signals. The need of large sets of accelerograms for the dynamic analysis of structures and the shortage of natural records on many regions globally, justifies the search for generation of artificial spectrum compatible seismic signals. The proposed method generates spectrum compatible seismic signals based on the HHT and on a genetic algorithm that helps solve an optimization problem. One random seismic signal is used every time. This makes the method flexible as there can be chosen signals with a finite number of points and thus reduce the computational complexity of the method.

All generated signals were analyzed to prove that there are reliable simulations of natural signals. For this purpose, the values of the 8 well-known seismic parameters were calculated along with the damage indices for a certain frame structure, for both natural and artificial signals. The values extracted were then compared with the respective values of the initial seismic signals, and a correlation study was carried out. The numerical results indicate that the synthetic signals behave like real earthquake signals. The degree of interdependence was very similar for all damage indices, for natural and artificial earthquake signals, respectively. For example, the parameter SA is correlated with the DI P/A for the natural and artificial earthquake signals by 0.881 and 0.833, respectively.

Fig. 1 .

 1 Fig. 1. Reinforced concrete frame structure.

n

  In this study, twenty natural seismic signals where used to generate synthetic spectrum-compatible seismic signals. The generated signals were afterwards submitted into a correlation study, to examine the relation between the seismic parameters and the damage indices, compared with the same correlation percentages of the initial seismic signals.

Fig. 2 .

 2 Fig. 2. (a) Friuli (Feltre station) seismic signal (b) synthetic generated signal and (c) response spectra and design spectrum.

Table 1 .

 1 Seismic intensity parameters strong correlated with the structural damage indices.

	No. Seismic Intensity Parameters	Ref.
	1	Spectrum Intensity after Housner (SI H )	[8, 9]
	2	Spectrum Intensity after Kappos (SI K )	[8, 9]
	3	Spectral Displacement (SD)	[8, 9]
	4	Spectral Velocity (SV)	[8, 9]
	5	Spectral Acceleration (SA)	[8, 9]
	6	Seismic Intensity after Fajfar/Vidic/Fischinger (I FVF )	[8, 9]
	7	Peak Ground Velocity (PGV)	[8, 9]
	8	Seismic Energy Input (E inp )	[8]

Table 2 .

 2 Data for the 20 natural seismic signals.

	Seismic Event	Station	Date
	Round Valley	Mcgee Creek Surface (USGS 1661)	11/23/84
	Coyote Lake	Coyote Lake Dam-San Martin, (CDMG 57217)	08/06/79
	Friuli	Conegliano	05/06/76
	Loma Prieta	Coyote Lake Dam Downst, (CDMG 57504)	10/18/89
	Coalinga	Chp, (CDMG 46T04)	07/22/83
	Alaska	Sitka Observatory	07/30/72
	Mt Lewis	Halls Valley, (CDMG 57191)	03/31/86
	San Fernando	Fort Tejon, (USGS 998)	02/09/71
	West Moreland	Brawley Airport, (USGS 5060)	04/26/81
	Oroville	Medical Center, (CIT 1544)	08/02/75
	Morgan Hill	Apeel 1E Hayward, (USGS/CDMG 1180)	04/24/84
	Dinar	Dinar	01/10/95
	Lytle Creek	Devils Canyon, (CDWR 620)	09/12/70
	Friuli	Barcis	05/06/76
	Borah Peak	Cem	10/29/83
	Morgan Hill	Capitola, (CDMG 47125)	04/24/84
	Mammoth Lakes	Fis	05/27/80
	Friuli	Feltre	05/06/76
	Kocaelli	Eregli, (ERD)	08/17/99
	Imperial Valley	Ec Co Center FF, (CDMG 5154)	10/15/79
	Figure 2(a)

Table 3 .

 3 Spearman correlation coefficients between the seismic parameters and the damage indices.

	Seismic Intensity	Original Signal		Artificial Signal
	Parameters	MISDR	DI P/A	MISDR	DI P/A
	SI H	0.810	0.786	0.929	0.857
	SI K	0.786	0.762	0.810	0.881
	SD	0.857	0.905	0.690	0.905
	SV	0.857	0.929	0.690	0.905
	SA	0.833	0.881	0.762	0.833
	I FVF	0.857	0.929	0.690	0.905
	PGV	0.905	0.952	0.690	0.905
	E inp	0.714	0.738	0.929	0.857


Arman Mehrbakhsh

Alireza Khalilian Sama

Experiments with Face Recognition using A Novel Approach based on CVQ Technique

Keywords: Face Recognition, Classified Vector Quantization (CVQ), Vector Quantization (VQ)

Face recognition techniques attempt to identify faces according to the patterns of mouth, lip, eyes and so on. However, the effectiveness of existing approaches degrades in presence of uncontrolled conditions such as variations of background light and image sizes. To deal with this problem, we propose a novel approach based on Classified Vector Quantization (CVQ) technique. The new approach divides images into some blocks and each block is classified into several patterns. Then, the Vector Quantization (VQ) technique is applied on the vectors of each pattern. In order to evaluate our approach, we have conducted a family of experiments on some standard image databases, MIT, YALE, and AR. The results demonstrate that the new approach is steadily capable of identifying faces in different situations.

Introduction

Currently, due to the increment of violence and crime rate, the usage of systems to establish security and safety is a critical requirement to the current human's life [START_REF] Zhao | Face recognition: A literature survey[END_REF]. To deal with this requirement, face recognition of convicts is considered as the major purpose in development of such systems [START_REF] Butavicius | An Experiment on Human Face Recognition Performance for Access Control[END_REF]. In addition, since the events of September 11, the process of development and implementation of automated biometric systems have been noticeably increased [START_REF] Butavicius | An Experiment on Human Face Recognition Performance for Access Control[END_REF][START_REF] Blackburn | [END_REF]. Moreover, there are many practical situations in which we need to the techniques of pattern recognition, and in particular face recognition. The wide range usage of these techniques in commercial and law enforcement applications, such as biometric authentication [START_REF] Blackburn | [END_REF], video surveillance [START_REF] Zhong | Crowd surveillance using Markov Random Fields[END_REF] and information security [START_REF] Zhao | Face recognition: A literature survey[END_REF] has made it a popular and significant area of research. Nowadays, we are capable to identify people through automatic recognition techniques based on physiological and characteristic behaviours, such as finger print [START_REF] Hong; Jian | Fingerprint enhancement[END_REF][START_REF] Ito | A fingerprint recognition algorithm using phase-based image matching for low-quality fingerprints[END_REF], iris [START_REF] Estudillo-Romero | The Hermite Transform: An Alternative Image Representation Model for Iris Recognition[END_REF][START_REF] Bevilacqua | A face recognition system based on Pseudo 2D HMM applied to neural network coefficients[END_REF][START_REF] Roy | Optimal Features Subset Selection Using Genetic Algorithms for Iris Recognition[END_REF], ear [START_REF] Attarchi | A New Ear Recognition Approach for Personal Identification[END_REF], vein face [START_REF] Khan | Representation of Hand Dorsal Vein Features Using a Low Dimensional Representation Integrating Cholesky Decomposition[END_REF], and so on. Although many techniques have been proposed for pattern recognition [START_REF] Anil | Statistical Pattern Recognition: A Review[END_REF], especially face recognition [START_REF] Guo | Boosting for fast face recognition[END_REF][START_REF] Watanabe | Ultra-fast facial recognition system based on VanderLugt Correlator[END_REF][START_REF] Momeni | Fast face recognition using a combination of image pyramid and hierarchical clustering algorithms[END_REF], and a number of studies [START_REF] Sani | Evaluation of face recognition system using Support Vector Machine[END_REF][START_REF]The FERET Evaluation Methodology for Face-Recognition Algorithms[END_REF][START_REF] Utsumi | Performance Evaluation of Face Recognition in the Wavelet Domain[END_REF] have been performed in the literature to evaluate and compare the existing techniques, there is still a long road ahead to achieve optimal approaches.

Arman Mehrbakhsh, Alireza Khalilian

There are various natural characteristics in human's body such as finger print, ear, iris, and similar other parts whose structure are unique for each human [START_REF] Blackburn | [END_REF]. The uniqueness of the structure for the mentioned body features can be used to recognize and to distinguish people from each other [START_REF] Agushinta | Face Component Extraction Using Segmentation Method on Face Recognition System[END_REF]. Ear recognition [START_REF] Attarchi | A New Ear Recognition Approach for Personal Identification[END_REF][START_REF] Ali | Ear Recognition Using Wavelets[END_REF] is a new class of biometrics that has certain advantages over most of the established biometrics; it has a rich and stable structure that is preserved from birth into the old ages [START_REF] Attarchi | A New Ear Recognition Approach for Personal Identification[END_REF]. Iris recognition [START_REF] Estudillo-Romero | The Hermite Transform: An Alternative Image Representation Model for Iris Recognition[END_REF][START_REF] Roy | Optimal Features Subset Selection Using Genetic Algorithms for Iris Recognition[END_REF][START_REF] Ma | Efficient iris recognition by characterizing key local variations[END_REF][START_REF] Lim | Efficient Iris Recognition through Improvement of Feature Vector and Classifier[END_REF] is one of the most accurate biometric systems when a high level of security is required. Therefore, designing such systems has attracted the attention of a large number of researches [START_REF] Estudillo-Romero | The Hermite Transform: An Alternative Image Representation Model for Iris Recognition[END_REF][START_REF] Roy | Optimal Features Subset Selection Using Genetic Algorithms for Iris Recognition[END_REF][START_REF] Lim | Efficient Iris Recognition through Improvement of Feature Vector and Classifier[END_REF][START_REF] Ma | Efficient iris recognition by characterizing key local variations[END_REF]. Unlike of other biometric methods, recognition based on human's face does not have any reliable and dependable mechanism that controls any undesirable conditions in the image of the face. These conditions include (1) changing the position of the face, (2) changing the size of the image, (3) the lack of sufficient light, etc. In ear recognition method, we need a half face image of each person. Hence, it is not an appropriate technique for facial expression recognition [START_REF] Attarchi | A New Ear Recognition Approach for Personal Identification[END_REF][START_REF] Ali | Ear Recognition Using Wavelets[END_REF]. In iris recognition method, by closing the eyes, the recognition process fails and we still need a reliable control mechanism. In addition, if the recognition relies only on the geometric structure of face [START_REF] Kanade | Picture processing by computer complex and recognition of human faces[END_REF][START_REF] Brunelli | Face recognition through geometrical features[END_REF], it will also need a reliable control mechanism because face in various angles loses its potential to be recognized properly [START_REF] Lu | A Novel Face Recognition Algorithm for Distinguishing Faces with Various Angles International Journal of Automation and Computing[END_REF].

In this paper, we propose a novel approach for face recognition based on Classified Vector Quantization (CVQ) [START_REF] Quweither | Classified Vector Quantization using Principal components[END_REF][START_REF] Kim | A transform domain Classified vector Quantizer for image coding[END_REF][START_REF] Tseng | A Very Low Bit Rate Image Compressor Using Transformed Classified Vector Quantization[END_REF][START_REF] Kim | A transform domain classified vector quantizer for image coding[END_REF] technique. At first step, this approach divides the face image into some blocks, converts each block to a vector, and classifies the vectors into some predefined patterns. These patterns are the major and common curves on the faces of different people. At second step, it uses Vector Quantization (VQ) [START_REF] Gersho | Vector Quantization and Signal Compression[END_REF][START_REF] Hsieh | A two-codebook combination and three-phase block matching based image-hiding scheme with high embedding capacity[END_REF][START_REF] Kekre | Fast codevector search algorithm for 3-D Vector Quantized codebook[END_REF][START_REF] Chang | Fast plannar-oriented ripple search algorithm for Hyperspace VQ codebook[END_REF][START_REF] Ramamurthi | Classified Vector Quantization of images[END_REF][START_REF] Kim | Image Vector Quantizer based on a classification in the DCT domain[END_REF] on the classified vectors of each pattern. The selection of the predefined patters significantly affects the performance of this approach. To evaluate the functionality of the proposed approach, we have conducted experiments on some standard image databases used in the experimental studies in the literature. The overall results showed that the proposed approach is capable to recognize images with an appropriate recognition rate.

The remainder of this paper is organized as follows. In Section 2, we introduce the face recognition problem. Vector quantization techniques and classified vector quantization technique are described in Sections 3 and 4 respectively. In Section 5 the proposed approach is presented. The experimental setup and the obtained results are discussed in Sections 6. Finally, some concluding remarks are outlined in Section 7.

Face Recognition

Face recognition has been used in different fields of security and protection, namely automated crowd surveillance [START_REF] Zhong | Crowd surveillance using Markov Random Fields[END_REF], access control [START_REF] Butavicius | An Experiment on Human Face Recognition Performance for Access Control[END_REF], identification of convicts [START_REF] Daygman | High Confidence Visual Recognition of persons by a test of Statistical Independence[END_REF], face reconstruction [START_REF] Park | 3D Face Reconstruction from Stereo Video[END_REF][START_REF] Zhao | Morphable Face Reconstruction with Multiple Images[END_REF][START_REF] Wang | Reconstructing 3D Face Model with Associated Expression Deformation from a Single Face Image via Constructing a Low-Dimensional Expression Deformation Manifold[END_REF] and so on. Thus, it has been one of the challenging issues in the last decade and it is a necessity in our current life.

In computer vision, there are two important methods whose purpose is face recognition. The first group is holistic appearance base methods, including PCA [START_REF] Weingessel | Local PCA algorithms[END_REF][START_REF] Diamantaras | Principal Component Neural Networks: Theory and Applications[END_REF], and LDA [START_REF] Lu | Face Recognition Using LDA Based Algorithms[END_REF][START_REF] Martinez | PCA versus LDA[END_REF]. In these methods, a facial image is considered as an instance

Experiments with Face Recognition using A Novel Approach based on CVQ

Technique 3

in N dimensional feature space, where N is the number of pixels in the image. The second group is local facial based methods such as elastic bunch graph matching. In these methods, a set of orthogonal basis vectors, that maximize the variance of facial image data, are obtained by Eigen decomposition of the scatter matrix of facial images. Combination of the two above methods, that are named hybrid methods, can be used for face recognition purpose.

Face recognition algorithms have focused on some unique properties of face [START_REF] Blackburn | [END_REF] such as iris, ear, skin, etc. Among the different components of face, brow is the most important for face recognition purpose. According to the existing studies [START_REF] Ramamurthi | Classified Vector Quantization of images[END_REF][START_REF] Quweither | Classified Vector Quantization using Principal components[END_REF][START_REF] Kim | A transform domain Classified vector Quantizer for image coding[END_REF][START_REF] Kim | Image Vector Quantizer based on a classification in the DCT domain[END_REF], all of the face features or components do not have equal effect in the effectiveness of face recognition process. Most of the experimental results and researches [START_REF] Agushinta | Face Component Extraction Using Segmentation Method on Face Recognition System[END_REF] showed that in the field of face recognition, nose and mouth are more important than eyes. Although brows have equal effect in face recognition as compared with other face components, less attention has been considered toward the effects of brows against the other face features [START_REF] Gersho | Vector Quantization and Signal Compression[END_REF]. Thus, an algorithm that utilizes all of these features together would perform better than existing ones. The proposed method in this paper takes different features of the face into account to improve the efficacy of face recognition.

Vector Quantization

The Vector Quantization (VQ) technique [START_REF] Nasrabadi | Image coding using vector quantization: A review[END_REF] was first developed for image processing to compress the images [START_REF] Hsieh | A two-codebook combination and three-phase block matching based image-hiding scheme with high embedding capacity[END_REF][START_REF] Kekre | Fast codevector search algorithm for 3-D Vector Quantized codebook[END_REF]. Codebook is a 2D array which is initialized randomly by the vectors of some of the training images. The number of columns in the codebook depends upon the size of the blocks of image and is usually considered as 9 or 16. In the decompressing phase, each code word by index i is replaced by the code word with index i in the codebook. One of the benefits of this technique is its low computational complexity. However, it is considered as a lossy technique. The CVQ [START_REF] Quweither | Classified Vector Quantization using Principal components[END_REF][START_REF] Kim | A transform domain Classified vector Quantizer for image coding[END_REF][START_REF] Tseng | A Very Low Bit Rate Image Compressor Using Transformed Classified Vector Quantization[END_REF][START_REF] Kim | A transform domain classified vector quantizer for image coding[END_REF] and TSVQ [START_REF] Makwana | Image Compression Using Tree Structured Vector Quantization with Compact Codebook[END_REF][START_REF] Chen; Bouman | Fast image database search using treestructured VQ[END_REF] are two major extensions of this technique.

When applying the VQ technique in face recognition, each block of the original image is identified with an index (number of the nearest code word in codebook). The response time is an important factor in real time applications [START_REF] Chang | Fast plannar-oriented ripple search algorithm for Hyperspace VQ codebook[END_REF] and VQ-based recognition provides reasonable response time in these situations.

In Vector Quantization (VQ) technique, the original image is partitioned into several blocks with the size of n × m. Then, then each block is arranged in the form of a vector using the row major or the column major method. In the next step, the codebook is updated using the obtained vectors of the image. Each row of this matrix is called a code word. Next, the difference between each vector of the original image and each code word in the codebook is computed by using of Euclidean distance in Equation (1):

D (B i ,C j ) = 2 , 1 , ) ( k j n k k i C B - ∑ = (1)

Arman Mehrbakhsh, Alireza Khalilian

In this equation, B i is a vector from the original image, C j is a code word from the codebook, B i,k is the kth element of the ith vector, and C j,k is the kth element of the jth code word. By this computation, the nearest code word to each vector with respect to the Euclidean distance is found. Finally, code words are updated by the centroid of all training vectors, which were mapped during coding [START_REF] Gersho | Vector Quantization and Signal Compression[END_REF]. Finding the optimized codebook is the major goal in VQ technique. It has been shown [START_REF] Jushi | Digital image processing An algorithmic approach[END_REF] that the design of VQ is optimal. This design is illustrated in Figure 1.

Fig. 1. Optimality of VQ design 4 Classified Vector Quantization

Edge is a very significant feature perceptually in an image. In our proposed method, we have used edge information for face recognition. Each coding technique that preserves the edge information is preferable because it has been proved [START_REF] Lindeberg | Edge Detection and Ridge Detection with Automatic Scale Selection[END_REF] that the human's eye can recognize objects through their edges and this is adapted in image processing methods. The Classified Vector Quantization (CVQ) technique has been proved [START_REF] Quweither | Classified Vector Quantization using Principal components[END_REF] to be an efficient technique for lossy image compression at low bit rates. CVQ technique can be used to reduce the computational complexity of VQ technique [START_REF] Gersho | Vector Quantization and Signal Compression[END_REF]. In this technique, each input vector is classified into a class. Then VQ is applied on vectors of each class [START_REF] Kim | A transform domain classified vector quantizer for image coding[END_REF]. Using this technique, there would be two indexes for each input vector: One for specifying the number of class and another for specifying the index of the nearest code word in codebook. This idea is shown in Figure 2. With CVQ technique, when applied to the face recognition, input vectors can be partitioned into some predefined patterns and then quantization is used for all vectors of each pattern. Since CVQ technique classifies the vectors in several classes, it is more precise than VQ.

The Proposed Approach

VQ technique can be very reliable in face recognition based on human's skin

Experiments with Face Recognition using A Novel Approach based on CVQ

Technique 5 because of the correlation of human's skin. But the main problem is when the number of training vectors of images is increased, VQ cannot distinguish among vectors because many of the vectors happen to be similar and the differences can be hardly observed. As a result, the efficiency of this technique degrades and it is the only limitation in the usage of this technique. Now, each block of the test image is indexed with the number of a pattern. When all of the image blocks were indexed, VQ technique is applied over all vectors that were classified for each pattern.

Fig. 2. A schematic of CVQ design

In order to deal with the issues of the VQ technique, CVQ can be applied. In our proposed approach, each block of the original image, the human's face, is compared with several predefined patterns. These patterns are defined according to the curves that are most seen in faces of each human. They can be extracted from a random number of faces using a pattern recognition approach like neural network. Figure 3 depicts some samples of patterns.

Fig. 3. Some samples of patterns

Several studies [START_REF] Gersho | Vector Quantization and Signal Compression[END_REF][START_REF] Young | Matching familiar and unfamiliar faces on internal and external features[END_REF][START_REF] Davies | Cue saliency in faces as assessed by the "Photofit" technique[END_REF] have shown that different features and components of the face can have unequal and various effects to the face recognition. Hence, taking all features of the face together into account has much potential for a face recognition technique to lead to better results. As a result, edge detection in face and also skin correlation can be considered together to enhance the face recognition process and improve the results. The

Arman Mehrbakhsh, Alireza Khalilian

Classified VQ technique enables us to consider the mentioned two features of the face simultaneously. This capability exposes the importance of CVQ technique in face recognition.

According to the above descriptions, we can present our method in the following steps: 1. The image is cropped to fit in the desired size. 2. The image is converted to the grayscale. 3. We need to detect the edges of the faces related to each original image. For example, Sobel filter can be used for this purpose. 4. According to the previous step, we can classify each block of the original image to one of the predefined patterns. Suppose that all blocks have been classified into classes C 1 to C n . Each class corresponds to a certain pattern. So, there would be a number for each block between 1 to n, so that it can describe the class number of the block. 5. After that, we can do vector quantization (VQ) technique on all original blocks which have the same class number. Note that there is a code book for each of the n classes of patterns.

6 Experimental Studies

VQ Results

In order to implement the VQ technique to be applied in face recognition, we have used a low pass filter (2D-Moving average with mask 9×9) on each of face image at the first step. The images were selected from the standard image databases, AR [START_REF] Martinez | The AR Face Database[END_REF], MIT [START_REF] Dimitri | Face databases[END_REF], and ORL [START_REF]ORL face database[END_REF]. It can remove the noises of image and is able to detect the components of the face. Then, we applied a code book with 128 code vectors. Next, we tested the code books of different sizes and we found that code book with size 128 is the optimal. Finally, for each training image, we created a histogram based on the number of similar vectors of each code word. Figure 4 shows a sample histogram for a face image. As a result, we will have a histogram for each of the training images. In order to find the output image, the histogram of input image and existing images in the database have been compared and the closest histogram was found.

We implemented this method using Delphi language, on standard image databases such as ORL, AR, Yale using a PC system with CPU 2.19 GHz and 1.87 GB of Ram. We observed the recognition rate between 95% and 97% (depending on the size of used mask). In this experiment, Error Recognition Rate (ERR) has been measured to 2.6%.

Experiments with Face Recognition using A Novel Approach based on CVQ Technique 7

Fig. 4. A sample of face histogram obtained with the proposed approach

CVQ Results

Observing the results obtained from the implementation of CVQ technique demonstrates that it has much potential for face recognition at high accuracy. As mentioned before, VQ is a robust technique for face recognition, especially in real time systems. However, when the number of input vectors of images is increased, it is no longer beneficial. Nevertheless, CVQ classifies the vectors into some patterns and then VQ can be used for vectors of each class. Moreover, by using this technique, we can utilize the two unique characteristics in human face, geometric structure of the face, and skin correlations. Hence, recognition rate is improved in comparison with other existing face recognition methods. The proposed method was implemented and tested using MIT-CBCL, AR, Yale and ORL image databases. At the first step, the faces in images are cropped. Then a filter such as Sobel filter is applied on each cropped face image in order to detect edges in faces. This enables us to classify each block of the size n×n in each of predefined patterns. Finally, VQ technique is used for all the blocks of the same pattern. We have implemented this method using the Java language, on a system with CPU 2.4 GHz and 512 MB of Ram. The recognition rate has been observed to be between 93% and 100% (depending on the type of database).

Table 1 shows the recognition rate in some major face recognition methods and that of our proposed approach with respect to MIT image database.

Table 1. Recognition rate of some face recognition method along with our proposed approach The required time for pre-processing has been measured as 12 msec and for recognition it has been measured at most 25 msec. Therefore, the total time would be 37 msec. The reason why we could achieve small processing time is that the proposed algorithm avoids from complex calculations. This will consequently improve the Arman Mehrbakhsh, Alireza Khalilian overall performance of the recognition with higher accuracy. We achieved the recognition rate of 45.83 and 37 percent for YALE and AR database images. For YALE database, there are 24 image groups, each group for an individual person. In each group, we have selected 5 images for training. In AR image database, there are images of different people in the same position and state. We have selected one image for each of the 65 persons. Then, we selected another image of each person to test such that it would be the same for all persons.

Conclusions

In this paper, a new face recognition approach based on Classified Vector Quantization (CVQ) has been proposed. We extended the concept of this technique in order to classify components of face images into some patterns and then we used Vector Quantization in order to create a code book for each pattern. In Comparison with other methods, our algorithm can be implemented without using hard mathematical computations.


Georgina Tryfou

Nicolas Tsapatsoulis

Extraction of Web Image Information: Semantic or Visual Cues?

Text based approaches for web image information retrieval have been exploited for many years, however the noisy textual content of the web pages makes their task challenging. Moreover, text based systems that retrieve information from textual sources such as image file names, anchor texts, existing keywords and, of course, surrounding text often share the inability to correctly assign all relevant text to an image and discard the irrelevant. A novel method for indexing web images is discussed in the present paper. The main concern of the proposed system is to overcome the obstacle of correctly assigning textual information to web images, while disregarding text that is unrelated to them. The proposed system uses visual cues in order to cluster a web page into several regions and compares this method to the use of semantic information and the realization of a k-means clustering. The evaluation reveals the advantages and disadvantages of the different clustering techniques and confirms the validity of the proposed method for web image indexing.

Introduction and Related Work

Numerous web image search engines have been developed as the amount of digital image collections on the web constantly increases [START_REF] Sclaroff | Unifying textual and visual cues for content-based image retrieval on the world wide web[END_REF]. These systems share the objective to minimize the necessary human interaction while offering an intuitive image search. The two main approaches that exist in the literature for content extraction and representation of web images are: (i) the text-based and (ii) the visual feature-based methods. The text-based approaches use associate text (i.e image file names, anchor texts, surrounding paragraphs) to derive the content of images. The text blocks that are used as concept sources for images may be extracted with several methods, with the following four being the most popular: (i) fixed-size sequence of terms [START_REF] Feng | A bootstrapping framework for annotating and retrieving www images[END_REF], (ii) DOM tree structure [START_REF] Hua | Semantic knowledge extraction and annotation for web images[END_REF][START_REF] Fauzi | Webpage segmentation for extracting images and their surrounding contextual information[END_REF], (iii) Web page segmentation [START_REF] He | Clustering and searching www images using link and page layout analysis[END_REF] and (iv) hybrid versions of the above [START_REF] Alcic | A clustering-based approach to web image context extraction[END_REF]. The first approach is time-efficient but yields poor results since the extracted text may be irrelevant to the image, or on the other hand, important parts of the relevant text may be discarded. Approaches that use the DOM tree structure of the web page are in general not adaptive and they are designed for specific design patterns. Web page segmentation is a more adequate solution to the problem since it is adaptable to different web page styles. Most of the proposed algorithms in this field though, are not designed specifically for the problem of image indexing and therefore often deliver poor results. The proposed system uses information obtained following the web page segmentation approach.

The paper is organized as follows: Section 2 presents the architecture of the proposed system. Section 3 presents the evaluation of the proposed system. Finally some conclusions and future perspectives are given in Sec. 4.

System Architecture

The general architecture of the proposed system is depicted in Fig. 1. As shown there the system consists of two main parts, which are described in the following sections.

Visual Segmentation

The content extraction of web images is based on textual information that exists in the same web document and refers to this image. In order to determine to which image the various text parts of a web page refer to, we use the visual cues which are connected to the outline and the presentation of the hosting web page. In order to obtain the set of visual segments that form a web page, we use the Visual Based Page Segmentation (VIPS) algorithm [START_REF] Cai | Vips: a vision based page segmentation algorithm[END_REF], which extracts the semantic structure of a web page based on its visual representation. It attempts to make full use of the page layout structure by extracting blocks from the DOM tree structure of the web page and locating separators among these blocks. Each web page is represented as a set of blocks that bare similar Degree of Coherence (DOC). With the permitted DOC (pDOC) set to its maximum value, we obtain a set of visual blocks that consist of visually indivisible contents. An example of a web page segmentation using pDOC = 10 (i.e. maximum allowed value) is illustrated in Fig. 2.

Text to Image Assignment

Each text block found in the web page has to be assigned to an image block. In other words, we attempt to determine to which image, each textual block refers How much does the Earth's gravity pull on me?

[edit] Block 1

It's easy to find your weight on Earth by using a scale. You have weight because the Earth's gravity pulls you towards its center. Normally, the ground or the floor get in the way, making you feel 'stuck' to them.

There are several kinds of scales:

1) Comparing of 2 masses (weights). You put the thing(s) you want to weigh on one pan (like some marbles), and then you put several "weights" on the other pan until the pointer shows that both pans have equal weights on them. Then you look at the pan with the known weights on it, and add them all up. The total is the mass of the thing(s) you want to weigh.

2) A spring balance usually has a hook on it, with a pan. You put the thing(s) you want to weigh on the pan, the spring is pulled, and the greater the weight, the further the spring is pulled. That distance, calibrated in pounds or kilogram (or whatever), is usually shown either on a dial or on a linear scale.

Block 2

Fig. 2. The results of VIPS algorithm on a fragment of a web page. Each visual block is marked with a rectangle region: dashed-red when the block is an image and continuousblue when it is text.

to. After this decision, the corresponding text blocks will be adequate to use for the extraction of the image information. The processing that takes place for this module is presented in Fig. 3 and each part is described in further details in the following paragraphs. Image/text Blocks Separation. The first task towards the assignment of each text block to the image block it refers to, is to determine whether a block contains image content or text. The use of the maximum pDOC during VIPS execution certifies that for a well formed HTML document, each block will either be an image or a text block indicating that no blocks with mixed content will be returned. The HTML source code that corresponds to each one of these blocks, is returned by the VIPS algorithm and it is used in order to classify them into two categories: (i) image blocks and (ii) text blocks.

Distance Calculation. Once the blocks are separated into these categories, the Euclidean distance between every image/text block pair is calculated making use of the Cartesian coordinates returned by the VIPS algorithm. The distance calculation in this case is not a trivial problem since the goal is to quantify the intuitive understanding of how close two visual blocks are. This understanding depends not only on the distance of the centres of the two visual blocks, but also on their shape, size and relative position. In order to solve the distance calculation problem several approaches were taken into consideration. The calculation of the distance between the closest edges of the blocks was found to offer the better representation of the block distance.

Clustering. After the distance calculation the web page has to be clustered into regions. Each region or cluster is defined by an image in its center and contains all the text blocks that have been found to refer to this image. For the clustering we took into consideration two different approaches. The first one is based only in visual cues and the location of the various blocks while the second approach mines semantic information and implements a k-means clustering.

Clustering Based on Visual Cues. In this approach, the text blocks are assigned to the corresponding cluster making use of the visual information that is available for them: the Euclidean coordinates which are returned be the VIPS algorithm are used for distance calculation and each text block is assigned to the cluster, whose center it is closest to. However, it is possible that one or more blocks of text do not refer to a certain image of the web page. For this reason, it is necessary to discard one or more text blocks from the calculated clusters. In order to determine which blocks of text are irrelevant to the image they are connected to, the blocks whose distance to the cluster center (i. e. corresponding web image) is longer than a defined threshold t, are discarded. In order to calculate this threshold, the distances d c i that appear in a cluster c are normalized to the maximum distance. Using the normalized values dc i the threshold t is calculated as t ed = t ′ + m eds ed , where t ′ is a static, predefined threshold (in our experiment is empirically set to 0.1), m ed is the mean value of the euclidean distances found in the cluster c and s ed is their standard deviation.

Clustering Based on Semantic Information. Semantic information, as obtained from the application of the Vector Space Model [START_REF] Salton | A vector space model for automatic indexing[END_REF], is used in the second approach in order to create a clustering which is based on the content of each block rather than its location on the web page. The web page is considered the corpus from which the vocabulary is extracted. Each text block tb i is one of the corpus' documents and it is expressed as a vector of term weights as

v i = [w 1i w 2i . . . w N i ]
T . The term w ti indicates the weight of text t in text block tb i and is calculated using the tf*idf [START_REF] Salton | A vector space model for automatic indexing[END_REF] statistic which expresses how important is each word for the representation of a text block. The k-means algorithm is then used on the vectors of term weights in order to cluster the text blocks into M regions. Since each text block may refer to any image but it may also be irrelevant to every image, the number of clusters M is equal to the total number of images found in the web page plus one for text blocks irrelevant to every image. Once the k-means algorithm is executed each text block is assigned to a specific cluster. However, it is not yet determined to which image each cluster refers to. To find the solution in this problem we considered the average distances among images and clusters as well as an initialization to the k-means algorithm based on the output of the first approach to the clustering procedure. The results presented in Sec. 3 are obtained using this initialization.

A set of manually labelled web images has been collected in order to create a corpus, based on which, the clustering of text blocks is evaluated. This corpus was obtained using an annotation tool that we designed in order to facilitate web image labelling. The annotator had to assign relevant text to each one of the images that exist in web pages which were rendered in the default browser. A dataset that consists of 40 web pages and their annotations was created in order to evaluate our method. Using the above described corpus and the evaluation measures Precision, Recall and F-score as defined in [START_REF] Alcic | Measuring performance of web image context extraction[END_REF], we obtained two different sets of results using the clustering methods described earlier. In the first set of results, the whole processing is based on visual information as obtained from the application of the VIPS algorithm. In the second set, the results are based on semantic information, as it is obtained from the Vector Space Model applied on the content of the web page. As shown in Table 1 there is a significant variation on the efficiency of the two approaches with the first approach having the highest average F-score. The execution of the proposed method yields an average F-score equal to 0.8610 for the total of the 131 annotated images. As shown in Fig. 4, more than 80% of the text blocks are identified with Recall and Precision values higher than 0.8, indicating that the system succeeds in retrieving most of the blocks that according to the annotation refer to a certain image. In this paper we presented an image indexing system that uses textual information in order to extract the concept of the images that are found in a web page. The method uses visual cues in order to identify the segments of the web page and calculates euclidean distances among these segments. It delivers a semantic or euclidean clustering of the contents of a web page in order to assign textual information to the existing images. During the experimenting and the evaluation it was found that a clustering based on semantic information of the contextual information delivers poor results compared to the clustering that is based on visual information. This stresses out the importance of understanding and processing the web pages as a structured visual document when it comes to web image indexing rather than an unstructured bag of words.

The weight vectors that were used for the k-means clustering are in general sparse vectors, since the length of the vocabulary is not proportional to the size of each text block. Moreover, when two neighbouring text blocks refer to the same image and contain similar semantic content, the authors usually select synonyms to express the same meaning. The Vector Space Model does not account such connections among different terms. It is therefore expected that the use of an ontology that describes the semantic distances and relations among different words and phrases will improve the results of the k-means clustering and this is the direction of our future study.

Fig. 1 .

 1 Fig. 1. The general architecture of the proposed system.

Fig. 3 .

 3 Fig. 3. The processing that takes place in the second part of the system: Text to Image Assignment

Fig. 4 .

 4 Fig. 4. Cumulative distribution function of F-measure.

Table 1 .

 1 The results for different Text Block Discarding Methods.

	Clustering Method Precision Recall F-score
	Visual	0.8610	0.8836 0.8962
	Semantic	0.3576	0.4528 0.4533