Evaggelos Spyrou
email: espyrou@iit.demokritos.gr

Phivos Mylonas
email: fmylonas@ionio.gr

A survey of geo-tagged multimedia content analysis within Flickr

Keywords: Flickr, tags, photos, multimedia, social networks

Our survey paper attempts to investigate how recent and undoubted emerge in enriched, geo-tagged social networks' multimedia content sharing works to the benefit of their users and whether it could be handled in a formal way, in order to capture the meaningful semantics rising from this newly introduced user experience. It further specializes its focus by providing an overview of current state-of-the-art techniques with respect to geo-tagged content access, processing and manipulation within the popular Flickr social network. In this manner it explores the role of information retrieval, integration and extraction from the technical point of view, coupled together with human social network activities, like, for instance, localization and recommendations based on preprocessed collaborative geo-tagged photos, resulting into more efficient, optimized search results.

Introduction

Current digital era is characterized by a single, yet very important observation: an extremely large amount of digital multimedia content is shared online every moment by people interacting within the so-called "social networks". This online social networking explosion shows no signs of abating, with almost twice as many Internet users having an online social profile than two years ago, helping to make Facebook3 the most viewed website in the European Union, according to recent research efforts. On top of that, more people are using the Internet to create their own multimedia content than ever before, with 73% of online users having a social networking profile [START_REF] Dugan | Social media update[END_REF], compared with 37% in 2008 [START_REF] Lenhart | Social media & mobile internet use among teens and young adults[END_REF].

Flickr 4 is an image and video hosting website created by a Vancouver-based company named "Ludicorp" back in 2004 and currently owned by Yahoo! 5 . What makes it special among other social networks is its aspect as an online community, within which users are able to interact by sharing comments about photography and create groups of particular interests. The Verge reported in March 2013 that "Flickr had a total of 87 million registered members and more than 3.5 million new photos uploaded daily" 6 . Each photo may contain metadata added by its photographer, such as tags that describe either its visual content or location, or a free text description. It also contains metadata added by the camera that has been used, such as date taken, camera settings, camera model, etc. Few Global Positioning System (GPS) enhanced cameras automatically geo-tag the photos they take, but in principal this is done manually, by the photographer. The vast majority of images uploaded to Flickr are taken by common users or amateur photographers. The textual metadata associated with the image often serves as a reminder of the context of the image for the photographer and his social circle [START_REF] Nov | Analysis of participation in an online photo-sharing community: A multidimensional perspective[END_REF], [START_REF] Van House | Flickr and public image-sharing: distant closeness and photo exhibition[END_REF].

In principle, every part of a photo may be tied to a geographic location, but in most typical applications, only the position of the photographer is associated with the entire digital photo. As the reader may imagine, this small detail implicates and significantly burdens most multimedia content search and retrieval tasks. In the most typical example, photos of a landmark may have been taken from very different positions apart and in order to identify all photos of this particular landmark within an image database, all photos taken within a reasonable circular distance from it must be considered. Now, when such geo-tagged photos are uploaded to online multimedia content sharing communities, such as Flickr, Panoramio7 or Instagram8 , that enable the construction of infinite connections among their users [START_REF] Van Dijck | Flickr and the culture of connectivity: Sharing views, experiences, memories[END_REF], a photo can be placed onto a map to view the location the photo was taken. In this way, social network users can browse photos from a map, search for photos from a given area, and find related photos of the same place from other users; these tasks are considered elementary in order to build additional, ad-hoc value-added digital services on top, like automated route/trip planning or like, to our most recent knowledge, the popular "NOW" app; the latter uses geo-tagged Instagram photos to find nearby events happening now 9 .

The act of automatically providing or calculating meaningful photo's geotags (the so-called "geo-tagging" process) opens a huge research topic for the researchers' community, mainly to the direction of being able to analyze them, to identify and determine social patterns amongst them. However, issues of credibility on the volunteered user-generated geo-tagging should become of broader research interest in various areas [START_REF] Flanagin | The credibility of volunteered geographic information[END_REF], [START_REF] Spielman | Spatial collective intelligence? credibility, accuracy, and volunteered geographic information[END_REF], motivating us to further investigate this topic in the following, focusing on the popular Flickr social network. At this point, it should be noted that our work differentiates from previous similar surveys, since they emphasized either on geotagged content without focusing specifically on user generated and manually geotagged photos of Flickr [START_REF] Luo | Geotagging in multimedia and computer visiona survey[END_REF], or in social media in general [START_REF] Wu | Social media research: A review[END_REF].

Multimedia Content Retrieval

The very first research community dealing with up-to-date multimedia community research challenges is the one depicted by Information Retrieval in general and multimedia content retrieval in particular. Since Flickr is mainly a photo sharing website, the fact that it attracted the interest of the image retrieval community is considered to be rather natural. The main approach followed by researchers is to use either textual metadata or visual properties of photos and often combine them in an effort to improve the accuracy of their respective algorithms. As depicted in the following, such research efforts vary from textual ones, to ones based on low level visual or even hybrid characteristics.

Aren't tags text, after all?

As discussed, the first approach to tackle the problem at hand is based solely on text retrieval, a branch of information retrieval where information is manipulated primarily in the form of text, having each photo represented solely by its textual features, i.e. the manually generated tags. Abbasi et al. [START_REF] Abbasi | Exploiting flickr tags and groups for finding landmark photos[END_REF] identified landmarks using tags and Flickr groups, without exploiting geospatial information. They used SVM classifiers trained on thematical Flickr groups, in order to find relevant landmark-related tags. Ahern et al. [START_REF] Ahern | World explorer: visualizing aggregate data from unstructured text in geo-referenced collections[END_REF] analyzed tags associated with geo-referenced Flickr images so as to generate knowledge. This knowledge was a set of the most "representative" tags for an area. They used a TF-IDF approach and presented a visualization tool, namely the World Explorer, which allowed users explore their results. Serdyukov et al. [START_REF] Serdyukov | Placing flickr photos on a map[END_REF] adopted a language model which lies on the user collected Flickr metadata and aimed to annotate an image based on these metadata. Their goal was to place photos on a map, i.e. provide an automatic alternative to manual geo-tagging. Venetis et al. [START_REF] Venetis | On the selection of tags for tag clouds[END_REF] examined techniques to create a "tag-cloud", i.e. a set of terms/tags able to provide a brief yet rich description of a large set of terms/tags. They presented and defined certain user models, metrics and algorithms aiming at this goal. Lerman et al. [START_REF] Lerman | Personalizing image search results on flickr[END_REF] aimed to personalize text-based search results by adding information about users' relations. Finally, Larson et al. [START_REF] Larson | Reading between the tags to predict realworld size-class for visually depicted objects in images[END_REF] tried to detect whether tags correspond to physical objects, and also the scale of these objects, using a natural language approach.

Shall we consider visual characteristics?

The second approach focuses on the visual aspects of multimedia content analysis. Research efforts in this area discard textual annotations and focus on lowlevel visual features. Wang et al. [START_REF] Wang | Learning image similarity from flickr groups using stochastic intersection kernel machines[END_REF] proposed a training algorithm and applied it on the problem of image similarity. They worked on a Flickr data set, under the assumption that two images are considered similar if they belong to the same group. Chatzilari et al. [START_REF] Chatzilari | Semi-supervised object recognition using flickr images[END_REF] used region level annotations and visual features, in an effort to recognize objects with a semi-supervised approach. They started from a set of Flickr photos that contain the same object. Philbin and Zisserman [START_REF] Philbin | Object mining using a matching graph on very large image collections[END_REF] created a graph based on visual features and tried to group similar Flickr photos, from a corpus of 1M photos. Avrithis et al. [START_REF] Avrithis | Retrieving landmark and nonlandmark images from community photo collections[END_REF] retrieved similar Flickr photos by using a 2-level clustering processing, both by means of geo-tags and visual features. Yanai et al. [START_REF] Yanai | A visual analysis of the relationship between word concepts and geographical locations[END_REF] focused on the relationship between words and locations. They used visual features and tried to associate them with certain locations, using an entropy based approach. Li et al. [START_REF] Li | Landmark classification in large-scale image collections[END_REF] used SVMs trained on visual features to classify a 30M data set. They observed that by incorporating temporal information, the accuracy of the results was significantly improved. Joshi and Luo [START_REF] Joshi | Inferring generic activities and events from image content and bags of geo-tags[END_REF] used visual detectors and incorporate bags-of-geotags within a probabilistic framework, in order to detect activities and events in photos. Yu and Luo [START_REF] Yu | Leveraging probabilistic season and location context models for scene understanding[END_REF] combined visual context with location information in order to detect concepts in photos. Luo et al. [START_REF] Luo | Event recognition: viewing the world with a third eye[END_REF] fused information extracted from both a Flickr data set and a set of satellite images, in order to detect events. Batko et al. [START_REF] Batko | Building a web-scale image similarity search system[END_REF] used MPEG-7 visual features and search into a set of over 50M photos from Flickr. Seah et al. [START_REF] Seah | Summarizing social image search results[END_REF] created visual summaries on the results of visual queries on a data set of Flickr images that in contrast to previous works, e.g., the one of [START_REF] Moëllic | Image clustering based on a shared nearest neighbors approach for tagged collections[END_REF], they attempted to generate concept-preserving summaries. Finally, Liu et al. [START_REF] Liu | Social-oriented visual image search[END_REF] incorporated the social aspect of photos, in order to re-rank search results more according to both social and visual relevances.

A little bit of both -the hybrid approach!

Since the visual content of images may provide a powerful description, many research efforts try to combine visual descriptions with textual metadata. Barrios et al. [START_REF] Barrios | Text-based and content-based image retrieval on flickr: Demo 30[END_REF] presented an image retrieval system that combines textual and visual content. They downloaded and stored locally images from Flickr and used simple color and texture visual descriptors, along with the title, description and tags, for each image. Crandall et al [30] used visual, temporal and geospatial information to automatically identify places and/or events in city and landmark level. They also added temporal metadata information to improve classification performance. With the same motivation, Quack et al. [START_REF] Quack | World-scale mining of objects and events from community photo collections[END_REF] divided the area of interest into non-overlapping, square tiles, then extracted and used visual, textual and geospatial features. They handled tags by a modified TF-IDF ranking and linked their results to Wikipedia10 . Gammeter et al. [START_REF] Gammeter | I know what you did last summer: object-level auto-annotation of holiday snaps[END_REF] overlaid a geospatial grid over earth and matched pairwise retrieved photos of each tile using visual features. Then they clustered photos into groups of images depicting the same scene. The metadata were used to label these clusters automatically, using a TF-IDF scheme. Moëllic et al [START_REF] Moëllic | Image clustering based on a shared nearest neighbors approach for tagged collections[END_REF] aimed to extract meaningful and representative clusters from large-scale image collections. They proposed a method based on a shared nearest neighbors approach that treats both visual features and tags. Li et al [START_REF] Li | Learning tag relevance by neighbor voting for social image retrieval[END_REF] proposed an algorithm that learns tag relevance by voting from visually similar neighbors. They did not use geospatial data, nor limited their approach on landmarks/places of interest and aimed to retrieve semantically similar images. Moxley et al. [START_REF] Moxley | Not all tags are created equal: Learning flickr tag semantics for global annotation[END_REF] classified mined geo-referenced tags as places, by extending [START_REF] Rattenbury | Towards automatic extraction of event and place semantics from flickr tags[END_REF], landmarks by clustering image datasets considering mutual information and prior knowledge from Wikipedia and visual terms using the mutual information between visual descriptors and tags. Ulges et al. [START_REF] Ulges | Learning visual contexts for image annotation from flickr groups[END_REF] adopted a context-based approach, assuming that users place semantically similar photos in Flickr groups. Fan et al. [START_REF] Fan | Justclick: Personalized image recommendation via exploratory search from large-scale flickr images[END_REF] proposed a system, namely JustClick which exploits both visual and textual information and after a search and retrieval process, it recommends photos using an interactive interface. Simon et al. [START_REF] Simon | Scene summarization for online image collections[END_REF] created visual summaries of large image data set based mainly on visual features, but also exploiting tags. Kennedy and Naaman [START_REF] Kennedy | Generating diverse and representative image search results for landmarks[END_REF] used visual features and tags, in order to extract the most representative tags and views for landmarks, working on a corpus of 110K Flickr photos from San Fransisco. Finally, Liu et al. [START_REF] Liu | Where your photo is taken: Geolocation prediction for social images[END_REF] were the first to consider user uploading patterns, geotagging behaviors, and the relationship between the temporal and the spatial gap of two photos from the same user.

Automatic Tag/Geo-tag Generation

In a slightly different approach, special attention has been given to methods exploiting the automatic generation of tags, a process often called "(tag-)recommendation", as well as the prediction of geo-tags, i.e., of the geographic coordinates where a photo has been taken, a process often referred to as "localization". In the following, we briefly present both approaches, summarizing most important research works in the fields.

Tag Recommendation

Initially and as expected, tag recommendation approaches often adopt traditional tag processing techniques. In this manner, Chen et al. [START_REF] Chen | Sheepdog: group and tag recommendation for flickr photos by automatic search-based learning[END_REF] proposed a system that automatically recommends tags for photos and also for adding photos into appropriate popular groups. For the latter case, they used SVM predictors in order to identify concepts and used these results so as to search for groups. Then, they used these groups to harvest more tags and attach them to their photos. Anderson et al. [START_REF] Anderson | Tagez: Flickr tag recommendation[END_REF] presented a system, namely TagEz which combined both textual and visual features, so as to recommend tags. Their results indicated that the use of textual metadata outperformed both visual and combined feautures. Chaundry et al. [START_REF] Chaudhry | Automated extraction and geographical structuring of flickr tags[END_REF] presented an approach for tag assignment to geographic areas, using a TF-IDF scheme and logistic regression, for various levels of detail. Hsieh and Hsu [START_REF] Hsieh | Search-based automatic image annotation via flickr photos using tag expansion[END_REF] exploited visual similarity and after a tag expansion process, aim to automatically annotate photos. Kennedy et al. [START_REF] Kennedy | How flickr helps us make sense of the world: context and content in community-contributed media collections[END_REF] selected representative tags from urban areas using a multimodal approach. Their results indicate that the use of visual features can drastically improve precision. Sigurbjörnsson and Van Zwol [START_REF] Sigurbjörnsson | Flickr tag recommendation based on collective knowledge[END_REF] extracted tag co-occurence statistics and tag aggregation algorithms, in order to recommend tags by investigating and evaluating four different strategies. Furthermore, they introduced a "promotion" function, whose role was to promote the most descriptive tags. Garg and Weber [START_REF] Garg | Personalized tag suggestion for flickr[END_REF], [START_REF] Garg | Personalized, interactive tag recommendation for flickr[END_REF] presented a system that while users tagged their photos, it dynamically suggested related tags by considering similar groups to user's preferences. Moxley et al [START_REF] Moxley | Spirittagger: a geo-aware tag suggestion tool mined from flickr[END_REF] presented SpiritTagger tool, in order to recommend tags for Flickr photos of urban regions, which is unaware of the user's tags and lies on visual properties and geographic distance, in order to select similar photos. Popescu and Moëllic [START_REF] Popescu | Monuanno: automatic annotation of georeferenced landmarks images[END_REF] presented Monuanno, a system that uses visual features to automatically annotate georeferenced landmark images. Kleban et al. [START_REF] Kleban | Global annotation on georeferenced photographs[END_REF] presented a world scale system for tag recommendation, based on geotags and visual features. Finally, Chen and Shin [START_REF] Chen | Tag recommendation by machine learning with textual and social features[END_REF] used both textual and social features of tags and a machine learning approach, in order to extract representative tags that can be related to the users favorite topics.

Content Localization

On the other hand, automatic geo-tag generation has gained huge research interest, mainly due to the vast available Flickr database of geo-tagged photos. Kelm et al. [START_REF] Kelm | A hierarchical, multi-modal approach for placing videos on the map using millions of flickr photographs[END_REF] adopted a hierarchical approach and tried to automatically predict geo-tags for Flickr videos. Their technique lies on both textual and visual features and also uses external resources, such as Geonames11 and Wikipedia. Van Laere et al. [START_REF] Van Laere | Combining multi-resolution evidence for georeferencing flickr images[END_REF] trained naive Bayes classifiers at different spatial resolutions. They used only textual features and worked at various spatial resolutions, for a set of 55 european cities. De Rouck et al [START_REF] De Rouck | Georeferencing wikipedia pages using language models from flickr[END_REF] used language probabilistic models that have been trained on Flickr photos, in order to geo-tag Wikipedia pages. Their approach outperformed Yahoo! Placemaker 12 and their results indicated that the increasing growth of tagged content in Flickr would continuously improve their accuracy. Friedland et al. [START_REF] Friedland | Multimodal location estimation on flickr videos[END_REF] combined textual and visual features and worked on a the MediaEval 2010 data set. They concluded that solely visual information proves inadequate for accurate geo-localization, but when combined with textual it can assist on the improvement of the accuracy. Hauff and Houben [START_REF] Hauff | Geo-location estimation of flickr images: social web based enrichment[END_REF] added information considering user's activities in Twitter 13 . However, even if their results were promising, the median location error was still far from usable. Van Laere et al. [START_REF] Van Laere | Towards automated georeferencing of flickr photos[END_REF] divided areas into disjoint regions and then used statistics and a Naive Bayes classifier. Van Laere et al. [START_REF] Van Laere | Finding locations of flickr resources using language models and similarity search[END_REF] proposed a tag-based approach that uses language models and similarity search, in order to estimate geo-tags based on a training set. Friedland et al. [START_REF] Friedland | Video2gps: a demo of multimodal location estimation on flickr videos[END_REF] worked on Flickr videos and used both textual and visual metadata. Their results may seem poor, however they were superior to all other contributions of MediaEval 201014 . Kalantidis et al. [START_REF] Kalantidis | Viral: Visual image retrieval and localization[END_REF] presented Viral, a system that aims to localize photos uploades by users, based on the visual similarity to geo-tagged images. They used a database of more than 2M photos taken from 40 cities. In previous work [START_REF] Spyrou | Placing user-generated photo metadata on a map[END_REF] we suggested a probabilistic framework which aimed to place Flickr data on a map based on their tags.

Joshi et al. [START_REF] Joshi | Exploring user image tags for geo-location inference[END_REF] proposed a probabilistic framework for tag-based localization. Their work was extended by Gallagher et al. [START_REF] Gallagher | Geo-location inference from image content and user tags[END_REF] who used a large geotagged corpus from Flickr, extracted several visual features and used location probability maps for tags. They integrated them and tried to localize photos. Hays and Efros [START_REF] Hays | Im2gps: estimating geographic information from a single image[END_REF] present IM2GPS, a system for image localization using visual features. It should be noted that they provided a probability distribution over the Earth. Kalogerakis et al. [START_REF] Kalogerakis | Image sequence geolocation with human travel priors[END_REF] extended this work by adding temporal information, in an effort to extract information about image sequences. O' Hare and Murdock [START_REF] Ohare | Modeling locations with social media[END_REF] presented a statistical language modeling approach, in order to identifying locations in arbitrary text. They investigated several ways to estimate models, based on the term and the user frequencies. To this goal. they used a set of public, geo-tagged photos in Flickr as ground truth. Hare et al. [START_REF] Hare | Gps estimation for places of interest from social users' uploaded photos[END_REF] estimated a continuous probability density function (PDF) over the Earth and combined textual with a number of weighted visual features. Their approach on tags differs from the others as they do not filter any of the tags, but they rather use them for evidence, i.e. certain words may be associated with certain countries. Finally, Li et al [69] removed "noisy" photos, i.e. photos that cannot contribute sufficiently to location estimation. They extracted both local and global features and instead of using the whole dataset, they performed clustering and use the resulting centroids, instead.

Conclusions

In this position paper we attempted to conduct a detailed survey and provide a brief summarization of current state-of-the-art techniques regarding geo-tagged Flickr content access, processing and manipulation issues. In this manner, we explored related research efforts mostly focused on information retrieval tasks. Our intention was to identify the trends in the surveyed area and organize them in a novel way that would integrate and add understanding to the work in the field with respect to the Flickr social network, so as for fellow researchers to be able to seek and reference related information efficiently. Among our future work is the extension of this survey to other Flickr application domains, other popular social networks and even other content types, such as text snippets according to the social network under investigation.

http://www.facebook.com

http://www.flickr.com

http://www.yahoo.com

http://en.wikipedia.org/wiki/Flickr#cite_note-4

http://www.panoramio.com

http://www.instagram.com

http://techcrunch.com/2013/01/11/now-app/

http://www.wikipedia.org

http://www.geonames.org/

http://www.programmableweb.com/api/yahoo-placemaker

http://www.twitter.com

http://www.multimediaeval.org/mediaeval2010/


Olga Politi

Iosif Mporas
email: 2imporas@upatras.gr

Vasileios Megalooikonomou

Comparative Evaluation of Feature Extraction Methods for Human Motion Detection

Keywords: Accelerometers, movement classification, human motion recognition

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

One of the most important tasks in pervasive computing is to provide accurate and opportune information on people's activities and behaviors. Applications in medicine, security and entertainment constitute examples of this effort. For instance, patients with obesity, diabetes or heart disease, are often required to fulfill a program of physical exercise that is integrated within their daily activities [START_REF] Jia | Diatetic and exercise therapy against diabetes mellitus[END_REF]. In computer vision, complex sensors such as cameras have been used to recognize human activities but their accuracy falls under a real-home setting, due to the high-level activities that take place in the natural environments, as well as the variable lighting or clutter [START_REF] Khan | Triaxial Accelerometer-Based Physical-Activity Recognition via Augmented-Signal Features and a Hierarchical Recognizer[END_REF]. As a result, body-attached accelerometers are commonly used as an alternative in order to assess variable daily living activities.

The human motion detection problem using accelerometers is an emerging area of research. Sensors embedded in objects or attached on the body are generally chosen to study movement patterns or human behavior. Accelerometers have been used extensively due to their low-power requirements, small size, non-intrusiveness and ability to provide data regarding human motion. In a common scenario, these data can be processed using signal processing and pattern recognition methods, in order to obtain a real-time recognition of human motion.

Several human activity recognition systems have been proposed in the past, which include the use of accelerometers. Some of them analyze and classify different kinds of activity using acceleration signals [START_REF] Mantyjarvi | Recognizing human motion with multiple acceleration sensors[END_REF], [START_REF] Sekine | Discrimination of walking patterns using wavelet-based fractal analysis[END_REF], while others apply them for recognizing a wide set of daily physical activities [START_REF] Ermes | Frequent Bit Pattern Mining Over Triaxial Accelerometer Data Streams for Recognizing Human Activities and Detecting Fall[END_REF], or describe a human activity recognition framework based on feature selection techniques [START_REF] Zhang | A Feature Selection-Based Framework for Human Activity Recognition Using Wearable Multimodal Sensors[END_REF]. Bernecker et al [START_REF] Bernecker | Activity Recognition on 3D Accelerometer Data[END_REF], proposed a reclassification step that increases accuracy of motion recognition. Karantonis et al. [START_REF] Karantonis | Implementation of a Real-Time Human Movement Classifier Using a Triaxial Accelerometer for Ambulatory Monitoring[END_REF] introduced an on-board processing technique for a real-time classification system, yielding results that demonstrate the feasibility of implementing an accelerometerbased, real-time movement classifier using embedded intelligence.

Khan et al. [START_REF] Khan | Triaxial Accelerometer-Based Physical-Activity Recognition via Augmented-Signal Features and a Hierarchical Recognizer[END_REF] propose a system that uses a hierarchical recognition scheme, i.e., the state recognition at the lower level using statistical features and the activity recognition at the upper level using the augmented feature vector followed by linear discriminant analysis. Several powerful machine learning algorithms have been proposed in the literature for the detection of human motion. The most widely used are the artificial neural networks [START_REF] Bernecker | Activity Recognition on 3D Accelerometer Data[END_REF][START_REF] Khan | Triaxial Accelerometer-Based Physical-Activity Recognition via Augmented-Signal Features and a Hierarchical Recognizer[END_REF][START_REF] Khan | Accelerometer's Position Free Human Activity Recognition Using A Hierarchical Recognition Model[END_REF], the naïve-Bayes [START_REF] Ermes | Frequent Bit Pattern Mining Over Triaxial Accelerometer Data Streams for Recognizing Human Activities and Detecting Fall[END_REF][START_REF] Ravi | Activity Recognition from Accelerometer Data[END_REF] and the support vector machines [START_REF] Zhang | A Feature Selection-Based Framework for Human Activity Recognition Using Wearable Multimodal Sensors[END_REF][START_REF] Ravi | Activity Recognition from Accelerometer Data[END_REF].

In this paper, a comparative evaluation of feature extraction methods for human motion detection is presented. The main contribution of the paper is the proposal of the best combination between preprocessing and feature extraction, for the human activity recognition task. For this purpose, we choose to evaluate different preprocessing and feature extraction combinations, formed after thoroughly examining the existing state-of-the-art methods. Although this study may seem quite simple, there is little known in bibliography regarding official comparison between methods that include preprocessing combined with feature extraction. Since most studies concentrate on preprocessing or feature extraction separately, focusing on the above combinations seems important not only for the acquisition of better results, but also for discovering meaningful data interpretations and features that "characterize" human motion.

The rest of this paper is organized as follows: In Section 2 we present the framework constructed for human motion detection. Section 3 offers details about the experimental setup and in Section 4 we present the achieved experimental results. Finally in Section 5 we conclude this work.

Framework for Comparative Evaluation

In the present framework for comparative evaluation, we assume that the input to the framework consists of 3-dimensional  

,, x y z signal streams, as illustrated in Fig. 1.

Each stream represents one movement direction in the sense of moving forward/backward, up/down and left/right. Preprocessing i consists of applying a sliding window W to the incoming streams, of constant length, resulting to i W frames, where 1 iI  . The time-shift between two successive frames is also constant and can result to frame sequences with or without overlap. After applying the sliding win-dow, each signal frame i W is either led directly to the feature extraction stage(Non- preprocessing method) or is previously processed using two different techniques. The utilization of these methods ensures that we get as much possible information from the data retrieved. In detail, each incoming representation of a signal frame is processed in parallel by each one of the feature extraction modules shown in Fig. 2. The estimated feature vectors could be mathematically represented as follows: the time-domain features

   

arg max ,

jk j f f k f d C F M 
, where j d is the motion class label assigned to the j -th test frame of the sensor data. After classification a post- processing algorithm could be applied on the automatically labeled frames of the test recording in order to fine-tune the detected human motion classes, but this is left for further studying.

3

Experimental Setup

Sensor data description

The USC-HAD database [START_REF] Zhang | USC-HAD: A Daily Activity Dataset for Ubiquitous Activity Recognition Using Wearable Sensors[END_REF], which is a freely available dataset provided by the University of Southern California was chosen in order to conduct experiments using the framework for comparative evaluation described in the previous section. The dataset corresponds to well-defined low-level daily activities appropriate for evaluation of algorithms for human motion in healthcare scenarios. The database consist of sensor measurements from 14 subjects, 7 male and 7 female, taken from a sensor placed at their front right hip, using sampling frequency equal to 100 Hz. The activities performed were: walking forward/left/right/upstairs/downstairs, jumping, sitting, sitting, standing, sleeping, and elevator up/down. MotionNode, the sensing platform used, integrates a 3-axis accelerometer, a 3-axis gyroscope, and a 3axis magnetometer. For the purpose of this paper, only the 3-axis accelerometer data were used. Subjects performed 14 different types of activities, including walking, running, sitting, standing, ascending and descending stairs. Ground truth was annotated by an observer while the experiments were being carried out.

Preprocessing and Feature Extraction

During preprocessing, each of the 3-dimensional  

,, x y z signal streams was frame blocked with a sliding window of 1 second length, with time-shift 0.5 seconds [START_REF] Karantonis | Implementation of a Real-Time Human Movement Classifier Using a Triaxial Accelerometer for Ambulatory Monitoring[END_REF]. Except the frame blocking three preprocessing methods were used [START_REF] Karantonis | Implementation of a Real-Time Human Movement Classifier Using a Triaxial Accelerometer for Ambulatory Monitoring[END_REF], namely: (i) the gravitational acceleration (GA) component of the signal, (ii) the body acceleration (BA) component of the signal and (iii) the Tilt sensing, defined as the angular position with respect to gravity per axis. Together with the baseline preprocessing (nonpreprocessing method), four preprocessing setups were created.

For each of the four preprocessed outputs per frame, statistical and physical features were extracted [START_REF] Zhang | A Feature Selection-Based Framework for Human Activity Recognition Using Wearable Multimodal Sensors[END_REF]. As far as feature analysis is concerned, statistical features are briefly divided into time-domain, frequency domain and time-frequency domain [START_REF] Avci | Activity Recognition Using Inertial Sensing for Healthcare, Wellbeing and Sports Applications: A Survey[END_REF]. Time-domain features include mean, median, variance, root mean square (RMS), standard deviation, skewness, kurtosis and interquartile range (25%, 50%, 75%). Frequency-domain features, which mainly represent the periodic structure of the signal, are the Fourier transform, the spectral entropy, the spectral energy and the 3rd order autoregressive-filter (AR) coefficients.

Time-frequency domain features are used to investigate both time and frequency characteristics of complex signals and in general employ wavelet techniques, such as wavelet coefficients or energy of wavelet coefficients. The physical features [START_REF] Zhang | A Feature Selection-Based Framework for Human Activity Recognition Using Wearable Multimodal Sensors[END_REF] in-clude the movement intensity (MI), the eigenvalues of dominant directions (EVA), correlation of acceleration along gravity and heading directions (CAGH) and the averaged velocity along each direction and the averaged acceleration energy (AAE).

Classification

For each of the four preprocessing methods the estimated feature vector has dimensionality equal to 77. Thus, the final feature vector per frame is of dimensionality 4×77=308. For the classification of the estimated feature vectors, we relied on the support vector machines (SVMs) implemented with the sequential minimal optimization method [START_REF] Keerthi | Improvements to Platt's SMO Algorithm for SVM Classifier Design[END_REF] using the polynomial kernel function (poly). The choice of the kernel-based algorithm is owed to the fact that SVMs do not suffer from the curse of dimensionality phenomenon [START_REF] Burges | A tutorial on Support Vector Machines for Pattern Recognition[END_REF]. The classifier was implemented using the WEKA machine learning toolkit software [START_REF] Keerthi | Improvements to Platt's SMO Algorithm for SVM Classifier Design[END_REF].

Experimental Results

The human motion detection framework presented in Section 2 was evaluated using the experimental setup described in Section 3. To avoid overlap between the training and test datasets we followed a 10-fold cross validation protocol. The experimental results, in percentages, for the four feature extraction methods and for each of the 14 subjects of the dataset are presented in Table 1. The best accuracy achieved for each subject is indicated in bold. The classification was carried out using individual classifiers, under a subject-dependent scenario. From Table 1 we can see that the best accuracy achieved is approximately 94%, in the non-preprocessed data representation and for Subject 11. The BA method is the one that performs the worst among feature extraction methods. Both the non preprocessed method and the tilt representation achieve very good results, with GA feature extraction method following shortly after the first two methods.

The experimental results may show that the non preprocessed method performs the best, so not using preprocessing can yield good accuracy in motion detection, but the interesting finding is that the Tilt method also performs as well. This is very important, due to the fact that by using Tilt sensing, we achieve to add a geometrical aspect to the previously linear acceleration signals. As a result, when it comes to motions that are characterized by changes in angular velocity or torque, i.e. turning or clockwise movements that may be followed by movements that are linearly translated, the Tilt sensing method is preferable.

Another observation is that the GA method also performs well, apart from the case of Subject 6. This outcome may be due to the fact that the database used for the experiments includes many motions that have a strong relation to gravity acceleration. To be more specific, five out of fourteen activities (walking upstairs/downstairs, elevator up/down and jumping) could be translated to changes only in the gravity dimension, not to mention the three static activities (lying, standing and sitting) which are also "characterized" by gravity acceleration.

As a result, to face the problem of variability in human motion, it seems more appropriate to deal with data preprocessing that involves mathematical or geometrical representation, rather than focus on the idea of applying filter-based signal processing. In addition, we investigated the importance of features used in the overall experimental process, in terms of feature influence in the classification process. The feature ranking was performed using the ReliefF algorithm [START_REF] Robnik-Sikonja | An adaptation of Relief for attribute estimation in regression[END_REF], and the top-3 features ranked by ReliefF are presented in Table 2. As can be seen in Table 2, within the most discriminative motion features for the task of automatic motion recognition are spectral entropy, the correlation between acceleration along gravity direction (CAGH), the correlation between gravity and heading direction, mean and median. At this point, it is evident that frequency domain features play an important role, as we meet spectral entropy and correlation at the top five ranked features. A physical feature (CAGH) is second in ranking, hence the importance of the addition of physical features in the feature extraction process. Attempting to interpret these ranking results, we could initially observe that spectral entropy is justifiably ranked first, since it helps differentiate between signals that have similar energy values but correspond to different activity patterns [START_REF] Brezmes | Activity recognition from accelerometer data on a mobile phone[END_REF]. The USC-HAD database includes movements with similar energy, i.e. sitting compared to standing and lying, or walking in different patterns (forward, left, right, upstairs and downstairs).

Following in rank is the CAGH feature that improves differentiation between activities that involve translation (in terms of acceleration intensity) in a single dimension, compared to the total acceleration intensity of the other two dimensions. Movements such as walking upstairs/ downstairs and walking left/right/forward fall in the previous category, hence the importance of using the CAGH feature. Correlation xz (between gravity and heading direction) also seems to play an important role in the classification results. Its interpretation is similar to CAGH, only in this case the comparison is inner-axial. Motions that have a strong xz correlation in the database are jumping, elevator up/down as well as static movements (lying, sitting, and standing).

Finally, statistical features like mean and median, despite their simplicity, could not be missing from the top-5 ranking, since they are known for the robustness and stability they provide to the classification process.

Conclusion

In the present work we present a comparative evaluation of feature extraction methods, based on 3-dimensional data acquired from inertial sensors worn by humans. The framework uses four different preprocessing methods and the motion signals are parameterized by statistical and physical features. The experimental results indicated an average performance of 90.89% for the best combination of preprocessing and feature extraction methods, for 14 everyday human motion activities. The highest performance achieved was approximately 94%. We deem the application of experiments constructed under a subject-independent scenario, in order to see if the present study responds well in intra-class errors, namely the errors introduces in the classification process when different people execute the same movement.

Fig. 1 .F

 1 Fig. 1. Block diagram of the preprocessing stage of the motion detection framework

Fig. 2 .M

 2 Fig. 2. Block diagram of the feature extraction and classification stage of the motion detection framework

Table 1 .

 1 Motion detection accuracy (%) per subject and feature extraction method

	Sub.	NP	BA	GA	Tilt
	01	91,80	57,66	91,88	89,72
	02	90,55	62,93	90,29	89,01
	03	90,74	61,92	90,63	89,80
	04	85,93	55,33	85,42	83,94
	05	86,23	59,00	86,74	85,10
	06	91,99	92,51	56,28	90,42
	07	92,46	60,56	92,72	90,55
	08	91,79	65,61	91,35	90,10
	09	92,07	69,24	91,24	91,08
	10	91,72	64,56	91,42	88,27
	11	93,91	58,54	93,89	91,37
	12	93,34	63,44	93,51	92,81
	13	92,57	57,61	92,33	90,81
	14	87,45	53,98	86,67	83,48

Table 2 .

 2 Top-5 features ranked by the ReliefF algorithm

	Ranking	Features
	1	Spectral entropy
	2	CAGH of x axis (gravity)
	3	Correlation xz
	4	Median
	5	Mean

Acknowledgement

The reported research was partially supported by the BioMedMine project (Thalis -University Of Patras -Mining Biomedical Data And Images: Development Of Algorithms and Applications), as well as the ARMOR Project (FP7-ICT-2011-5.1 -287720) "Advanced multi-paRametric Monitoring and analysis for diagnosis and Optimal management of epilepsy and Related brain disorders".


Dimitrios Triantafyllopoulos

Vasileios Megalooikonomou

Eye blink artifact removal in EEG using tensor decomposition

Keywords: eye blink, Tucker decomposition, wavelet transform, EEG

EEG data are usually contaminated with signals related to subject's activities, the so called artifacts, which degrade the information contained in recordings. The removal of this additional information is essential to the improvement of EEG signals' interpretation. The proposed method is based on the analysis, using Tucker decomposition, of a tensor constructed using continuous wavelet transform. Our contribution is an automatic method which processes simultaneously spatial, temporal and frequency information contained in EEG recordings in order to remove eye blink related information. The proposed method is compared with a matrix based removal method and shows promising results regarding reconstruction error and retaining the texture of the artifact free signal.

Introduction

Electroencephalographic (EEG) data are used in several cases where the brain's functionality needs to be analyzed. Their relative ease of use, their low power consumption, compact size and convenience as well as their temporal resolution, are some of the reasons that EEG recordings are preferred in several cases from their alternatives, such as functional Magnetic Resonance Imaging. Some of the applications where EEG data are used are Brain Computer Interaction (BCI) applications as well as medical tasks that require the supervision of a patient's brain functionality, like seizure monitoring.

Unfortunately, during EEG recordings except the underlying brain related activity, there are several signals captured also that degrade the information gathered. These signals are either related to external or internal reasons and are generally referred to as artifacts. External artifacts are often occurring due to technology's shortcomings such as electrical lines introducing a 50 Hz component, affecting the data recorded. The surrounding environment, such as walls and electrical devices, may cause additional noise. Those artifacts are usually counteracted with filtering techniques, for example the application of a Notch filter at 50 Hz.

Internal artifacts are related to the signals captured due to the subject's physical actions. For example eye blinking causes capturing of additional information in the frontal electrodes. Eye movement, as well as muscle movement, generate also undesired signals that are, of course, recorded. Avoiding these artifacts is one solution, but as showed in several studies [START_REF] Ochoa | P300 and blink instructions[END_REF][START_REF] Verleger | The instruction to refrain from blinking affects auditory P3 and N1 amplitudes[END_REF] , instructing subjects to avoid movement or any other activity that can cause an artifact affects the recorded data. On the other hand, the treatment of these artifacts is a challenging procedure and requires two steps. First the detection of their occurrence and then the handling of the detected events aim in the manipulation of these phenomena.

The detection of artifacts can be performed with visual inspection from a medical expert. But this procedure is, as expected, time consuming. Alternatively there are several automatic techniques that can be applied during or after recordings and detect the occurring artifacts. There are techniques, which are based in thresholds applied in time or frequency domain like in [START_REF] Delorme | Enhanced detection of artifacts in EEG data using higher-order statistics and independent component analysis[END_REF]. Some others are based in a supervised learning method using statistical [START_REF] Chadwick | Classifying eye and head movement artifacts in EEG Signals[END_REF] or autoregressive (AR) [START_REF] Lawhern | Detection and classification of subject-generated artifacts in EEG signals using autoregressive models[END_REF] features. Other methods, like [START_REF] Wu | An automated detection and correction method of EOG artifacts in EEG-based BCI[END_REF], use a combination of feature extraction and data driven thresholds.

Once the artifacted epochs are detected, there are two ways of manipulating these periods. The first one is rejecting the epoch that contains the artifacted signals. This procedure may solve the problem of artifacted epochs but results in substantial information loss. The second methodology is the removal of artifact related information and conservation of brain activity.

Removal methods are separated in two main categories. The first one aims in the removal of the EOG captured signal, which is captured with a separate sensor, with the use of linear combination and regression techniques [START_REF] Wu | An automated detection and correction method of EOG artifacts in EEG-based BCI[END_REF]. The second category of removal methods aims in the clarification of the components-sources that compose the recorded data and afterwards the identification of the artifact related ones. The most used decompositions are Principal Component Analysis (PCA), Canonical Correlation Analysis (CCA) and Independent Component Analysis (ICA). In [START_REF] Clercq | Canonical correlation analysis applied to remove muscle artifacts from the electroencephalogram[END_REF] and [START_REF] Vergult | Improving the interpretation of ictal scalp EEG: BSS-CCA algorithm for muscle artifact removal[END_REF] CCA is performed in order to remove muscle artifacts. Several methods report that ICA based removal methods outperform the methods based in other decompositions. In [START_REF] Halder | Online artifact removal for brain-computer interfaces using support vector machines and blind source separation[END_REF] an ICA based method discriminates the artifact related independent components, using an SVM classifier. In [START_REF] Mammone | Automatic artifact rejection from multichannel scalp EEG by wavelet ICA[END_REF] using high order statistics as criteria, wavelet decomposition in order to divide the recorded signal in frequency bands and ICA as the decomposition method, the proposed technique removes eye blink or muscle movement related artifacts from a pre specified epoch. In [START_REF] Sheng | A Novel Method to Remove Eye-Blink Artifacts Based on Correlation Using ICA[END_REF] the proposed ICA based method by using correlation coefficients between independent sources, specifies the independent component that most likely contains the artifact's signature. In [START_REF] Heute | Removing "cleaned" eye-blinking artifacts from EEG measurements[END_REF] a combination of ICA and Wiener filtering supress eye blink related artifacts.

In this study the effectiveness of a different decomposition method is examined. The decompositions in the methods mentioned so far are based οn two dimensional matrices, whose dimensions correspond to electrodes and time samples respectively. In order to examine, when necessary, the frequency information encapsulated in recordings the aforementioned methods, first extract spectral features and then classify the components as artifacts or not. By constructing a three-way array (tensor) whose first two modes correspond to electrode and time domain, while the third encloses the frequency information, it is possible to examine the three domains simultaneously. This concept was used previously in EEG signals in several studies [START_REF] Miwakeichi | Decomposing EEG data into space-time-frequency components using Parallel Factor Analysis[END_REF][START_REF] Mørup | Parallel Factor Analysis as an exploratory tool for wavelet transformed event-related EEG[END_REF][START_REF] Acar | Computational analysis of epileptic focus localization[END_REF]. In [START_REF] Acar | Multiway analysis of epilepsy tensors[END_REF] the same model was used in order to examine it's abilities in seizure localization as well as in artifact extraction. Regarding the latter the proposed method suggests multilinear subspace analysis with applying Tucker decomposition in the tensor created, in favor of removing the information that is correlated with the artifact. The main aim of our work is to examine the capability of tensors and more specifically Tucker decomposition's as a model to discriminate artifact related information from brain related activity. Our contribution is an automatic artifact removal method based on a similar tensor model with the one in [START_REF] Miwakeichi | Decomposing EEG data into space-time-frequency components using Parallel Factor Analysis[END_REF] and [START_REF] Acar | Multiway analysis of epilepsy tensors[END_REF]. The proposed method is compared with an ICA method, similar to the one proposed in [START_REF] Mammone | Automatic artifact rejection from multichannel scalp EEG by wavelet ICA[END_REF]. The result of the proposed modeling is a procedure which automatically removes artifact related information, while encapsulating simultaneously the spatial, temporal and frequency underlying structure of the recorded data.

In the following sections the theoretical background, the methodology followed as well as the experimental scheme is presented. In the end the conclusions and future work are discussed.

Background

Multilinear arrays

Multilinear arrays or tensors, as more than often are referred to, are a multidimensional generalization of vectors. A N-th order tensor is a product on N vector spaces having their own coordinate system [START_REF] Kolda | Tensor Decompositions and Applications[END_REF]. A matrix is a special case of tensors of order 2. It should be noted that the order of a tensor is the number of its dimensions. A general N-th order real tensor is notated as . As in the 2-dimensional case, where there exist several decomposition methods, like SVD, QR or NMF, there are high order generalizations of those methods that can be applied in high order tensors.

The most important of those decompositions are PARAFAC and Tucker decomposition. The former, introduced in [START_REF] Harshman | Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multimodal factor analysis[END_REF] and in [START_REF] Carroll | Analysis of individual differences in multidimensional scaling via an N-way generalization of "Eckart-Young" decomposition[END_REF] independently, decomposes the tensor in its rank-1 tensor components. The latter proposed in [START_REF] Tucker | Implications of factor analysis of three-way matrices for measurement of change[END_REF] is considered as the higher order generalization of SVD.

2.2

High order decomposition PARAFAC. Introduced in [START_REF] Harshman | Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multimodal factor analysis[END_REF] and in [START_REF] Carroll | Analysis of individual differences in multidimensional scaling via an N-way generalization of "Eckart-Young" decomposition[END_REF] independently, is based in the proposition of rank-1 decomposition of tensors in [START_REF] Hitchcock | The expression of a tensor or a polyadic as a sum of products[END_REF]. In [START_REF] Cattell | Parallel proportional profiles" and other principles for determining the choice of factors by rotation[END_REF] the same model was augmented, by introducing the parallel proportional profiles, whose aim was the estimation of sources ("source traits") that fit simultaneously in many profiles. After its third introduction, in 1970 by the two teams, the model is considered a very important tool in multilinear analysis. Its computation in [START_REF] Brett | MATLAB Tensor Toolbox Version 2.5[END_REF], the toolbox used in this research, is based in an Alternating least squares (ALS) algorithm. The 3-rd order tensor using PARAFAC is decomposed in rank-1 tensors as follows:

(

where , , , is the residual tensor and denotes the outer product.

Tucker. It is considered as the generalization of SVD, hence there are several cases where it is referred to as High Order SVD (HOSVD). A three way example of this decomposition is parted from three projection matrixes corresponding in each of the modes of the tensor, and one core tensor capturing the correlation of the different components. Since this interaction is allowed Tucker is considered as a more flexible model than PARAFAC. In [START_REF] Brett | MATLAB Tensor Toolbox Version 2.5[END_REF] Tucker is computed using a truncated SVD like method as an initialization step and then an ALS algorithm is followed. The Tucker decomposition of a 3-rd order tensor , with components in every mode is considered as follows:

(2) where is the core tensor, , , are the projection matrices and denotes the i-mode product (for more detail refer to [START_REF] Kolda | Tensor Decompositions and Applications[END_REF]).

Figure 1 shows the Tucker decomposition of the above scheme:

Methodology

In the proposed method the higher order decomposition used is Tucker decomposition. As described in [START_REF] Acar | Multiway analysis of epilepsy tensors[END_REF] and verified by our experiments, PARAFAC was unable to distinguish the artifact related information without removing also a large amount of useful information. This is due to the fact that generally artifact-related components correspond to components with large variance, therefore the removed information contains also great amounts of useful information. The more relaxed model of Tucker decomposition, due to the core tensor's non-diagonal structure, is more flexible to distinguish the artifact from clean signals' components.

The tensor constructed in the proposed method is the outcome of the application of continuous wavelet transform (CWT) in an EEG matrix with a variety of scales. The resulting tensor's modes are therefore , capturing the spatial, temporal and frequency information of the recorded data. The mother wavelet used in our experiments is Morlet wavelet due to its ability to reconstruct better an EEG signal. Before the application of CWT each channel is normalized using min-max normalization.

Once the decomposition of the tensor takes place the projection matrix corresponding to the second mode, which is the time domain, is examined for artifact related information. The criteria used in the identification of the artifact related components are the same as in [START_REF] Mammone | Automatic artifact rejection from multichannel scalp EEG by wavelet ICA[END_REF]. For each temporal signature kurtosis and entropy is extracted. The kurtosis criteria aims in the identification of peaky distributions, expected from an eye blink signature, while the entropy aims in artifacts of a noisy background. Then these features are separately normalized in order to have zero mean and unit variance. A threshold is applied in order to decide which signatures are artifact related. The latter ones are then removed and the reconstruction of the signal follows. Fig. 2 summarizes the process followed by the proposed method.

Fig. 2. Flowchart of the proposed method

Experiments

The data used in our experiments are from [START_REF] Clercq | Canonical correlation analysis applied to remove muscle artifacts from the electroencephalogram[END_REF]. The recorded data contain intentionally generated artifacts from a healthy 27-year-old male. From the EOG related recording an unmarked epoch was extracted as reference point of artifact free EEG period. A marked as an eye blink period was used as the reference point for artifact related information. The whole set of artifacted period's channels were added to the clean epoch's corresponding ones in order to create a test set with a ground truth data set.

Afterwards the ICA based method, as well as the tensor based one were used in order to remove the artifact related information. It should be mentioned that it is not the same as performing the removal methods in the artifact's period on it's own, since the artifact data contain their clean information also, which will be added to the clean epoch used as references. For better comparison of the two methods we plan to apply the two methods on the artifact epoch alone and then with the help of a medical expert having the results examined.

Once the clean epoch was extracted, one of the marked artifacts was added to the clean epoch's center. The clean epoch's duration was twice the size of the one of the artifact's epoch. The two methods were compared with different thresholds. Fig. 3 shows the average mean square error for both of these techniques. For every method and threshold value combination (thresholds belong in the set (1.2,1.3,1.4...1.8) five experiments of the removal procedure were performed. This error evaluation procedure was followed due to the fact that both methods rely on a random initialization. In more detail the ICA initializes the unmixing matrix randomly, while the Tucker decomposition initializes randomly the projection matrices. The mean square error is calculated based on the equation: [START_REF] Delorme | Enhanced detection of artifacts in EEG data using higher-order statistics and independent component analysis[END_REF] where CleanEpoch is the matrix of the clean epoch and ResultingEpoch is the matrix corresponding to the result of every method, n and m are the number of rows and columns of each matrix respectively. It is shown that the tensor based technique is more stable than the ICA based and has the lowest error for most of the thresholds. After a certain value, the ICA based method is clearly a better performer but the artifact signal in those thresholds is not cleared completely. It should be noted that for thresholds greater than 1.9 the suspected, for artifact containment, independent components were less than two. Therefore it was not possible to apply the ICA algorithm, leading in not removing the artifact at all. As noted in [START_REF] Mammone | Automatic artifact rejection from multichannel scalp EEG by wavelet ICA[END_REF] the preferred value of the threshold for the ICA method is 1.5. Applying the same threshold in the tensor based method had satisfactory results, as proven by our experinments, since the artifact was sufficiently removed without removing a great amount of the rest information in the signal. Fig. 4 and Fig. 5 show the two best results of the two methods. Each electrode's signal was normalized using min-max normalization, in order to show the relative amount removed from each method. The channels shown in this figure are part of the whole set of electrodes and are chosen to show each methods drawbacks. Fig. 4. The result of the tensor-based method. Every channel is affected, since the method is in essence a higher order PCA, but the texture of the reference signal is retained. It should be noted that the artifact exists only in the FP1 and FP2 channels, in the figures showed above. Any other information loss is unacceptable. The ICA based method retains most of the channels unfiltered. However, in the ones where a component was rejected, no texture was retained. By texture we mean the morphology of the clean signals. On the other hand the Tucker based method rejected information from all channels. But the texture of the signals is retained with a certain detail in almost every channel.

Discussion

Each method's usage has to be chosen depending on the application where the cleared data will be used on, and it's results. It should be noted that the criteria in the two methods were the same (kurtosis and entropy) and that these criteria play an important role in the results. Improved detection of artifact related components will certainly result in boosting the performance of both methods. For an application requiring the texture of the clean signal to be as less influenced as possible, the usage of the tensor based method is advised. In the case where the artifact suspected independent components are expected to be of small number compared to the amount of electrodes, ICA based methods should be preferred.

Applying spatial and frequency criteria could improve the detection of artifact related components. The creation of spatial, temporal or frequency profiles derived from tensor analysis could be also an interesting direction to be explored.

The goal of this work was to extend the tensor-based techniques for artifact removal by introducing automatic selection of artifact-related components. The results were promising and show that tensors can form the basis for an automatic removal method obtaining comparable results to the ICA based methods.

For future directions we aim to explore the possibilities of success of these methods on removing several other types of artifacts such as muscle artifacts. Using a medical expert's expertise it would be interesting also to see the result of applying these methods directly in an epoch, containing an artifact. The result will in this case be subjective but it will give a guideline of the ability of these methods to work efficiently in an artifact removing procedure during the recording.

Fig. 1 .

 1 Fig. 1. Tucker decomposition. PARAFAC is a special case of the Tucker decomposition where the core tensor is diagonal and the number of components in each mode are the same.

Fig. 3 .

 3 Fig. 3. Tensor based (waveTucker) in comparison with ICA based (awica) for several threshold values.

Fig. 5 .

 5 Fig. 5. The ICA-based method. Most of the electrode signals correctly remain intact but the artifact related electrodes, as well as the ones containing some sort of components that fulfilled the removal criteria, have lost the texture of the reference signal.

Acknowledgements

The reported research was partially supported by the ARMOR Project (FP7-ICT-2011-5.1 -287720) "Advanced multi-paRametric Monitoring and analysis for diagnosis and Optimal management of epilepsy and Related brain disorders".


Dimos Makris

Katia L Kermanidis

Ioannis Karydis
email: karydis@ionio.gr

The Greek Audio Dataset

Keywords: music, information retrieval, audio, Greek music, dataset

The Greek Audio Dataset (GAD), is a freely available collection of audio features and metadata for a thousand popular Greek tracks. In this work, the creation process of the dataset is described together with its contents. Following the methodology of existing datasets, the GAD dataset does not include the audio content of the respective data due to intellectual property rights but it includes MIR important features extracted directly from the content in addition to lyrics and manually annotated genre and mood for each audio track. Moreover, for each track a link to available audio content in YouTube is provided in order to support researchers that require the extraction of new featuresets, not included in the GAD. The selection of the features extracted has been based on the Million Song Dataset in order to ensure that researchers do not require new programming interfaces in order to take advantage of the GAD.

Introduction

Music Information Retrieval (MIR) research, in other words research on methods for Information Retrieval and Data Mining on musical data, has a number of requirements. Central to these requirements is the necessity to experiment with the methods on real musical data. This experimentation mainly achieves testing on the efficiency and effectiveness of the methods, as well as comparison of existing existing methods in order to show improvement.

In MIR research, the term musical data refers to sound recordings, sheet music as well as associated information to the musical content (i.e. metadata, social tags, etc). The process of accumulating a set of musical data (dataset) is very important for most scientific processed, including MIR. Datasets allow researchers to scientifically to compare and contrast their methods by testing these on commonly available collection of musical works.

Accordingly, MIR requires data for all kinds of music as results are not always intuitive due to the highly artistic nature of music. Although a number of widely used datasets do exist for the purposes of MIR research, most of these are collections of mainstream English language music. Local music has numerous differences from the these musical collections such us different instruments and rhythms. To the best of our knowledge, a standardized Greek audio dataset is not currently available.

Motivation and Contribution

To address the aforementioned requirements, we introduce the Greek Audio Dataset, a freely-available collection of Greek audio data for the purposes of MIR that, for each song it contains, offers:

audio features for immediate use in MIR tasks, the lyrics of the song, manually annotated mood and genre labels, a link to YouTube for further feature extraction, on the basis of MIR research processes.

The rest of the paper is organised as follows: Section 2 presents the related work on MIR available datasets, while Section 3 discuses the dataset, its creation processes as well as a detailed analysis of its content. Next, Section 4 details future directions concerning the dataset that could ameliorate is usability and further support MIR research. Finally the paper is concluded in Section 5.

Related Research

Since the early years of the MIR research, there have been numerous efforts to build datasets facilitating the work of MIR researchers.

The dataset titled RWC [START_REF] Goto | Rwc music database: Popular, classical, and jazz music databases[END_REF] is a copyright-cleared music database that is available to researchers as a common foundation for research. RWC was one of the first large-scale music database containing six original collections on different genres.

CAL500 [START_REF] Turnbull | Towards musical queryby-semantic-description using the cal500 data set[END_REF] is another widely used dataset, a corpus of 500 tracks of Western popular music each of which has been manually annotated by at least three human labelers with a total number of 1700 human-generated musical annotations.

Tzanetakis and Cook [START_REF] Tzanetakis | Musical genre classification of audio signals. Speech and Audio Processing[END_REF], introduced a dataset under the name GTZAN Genre Collection. The dataset consists of 1000 audio tracks, each 30 seconds long. It also contains 10 genres, each represented by 100 tracks and its size is approximately 1.2 Gb. The dataset was collected gradually with no titles from a variety of sources including personal CD collections or radio and microphone recordings, while it doesn't contain any music information or metadata.

USPOP2002 [START_REF] Berenzweig | A large-scale evaluation of acoustic and subjective music-similarity measures[END_REF] was introduced in order to perform comparison between different acoustic-based similarity measurements. 400 artists were chosen for popularity and for representation, while overall, the dataset contains 706 albums and 8764 tracks.

The Swat10k [START_REF] Tingle | Exploring automatic music annotation with "acoustically-objective" tags[END_REF] data set contains 10,870 songs that are weakly-labeled using a tag vocabulary of 475 acoustic tags and 153 genre tags. These tags have all been harvested from Pandora [START_REF]Pandora: A free personalized internet radio[END_REF] result from song annotations performed by expert musicologists involved with the Music Genome Project.

The Magnatagatune dataset features human annotations collected by Edith Law's TagATune game [START_REF] Law | Input-agreement: A new mechanism for collecting data using human computation games[END_REF] and corresponding sound clips encoded in 16 kHz/ 32kbps /mono/mp3 format. It also contains a detailed analysis from The Echo Nest of the track's structure and musical content, including rhythm, pitch and timbre.

Schedl et al. [START_REF] Schedl | A Professionally Annotated and Enriched Multimodal Data Set on Popular Music[END_REF], presented the MusiClef dataset, a multimodal data set of professionally annotated music. MusiClef contains editorial meta-data, audio features (generic and MIR oriented) are provided. Additionally MusiClef includes annotations such as collaboratively generated user tags, web pages about artists and albums, and the annotation labels provided by music experts.

Finally the Million Song Dataset (MSD) [START_REF] Bertin-Mahieux | The million song dataset[END_REF] stands out as the largest currently available for researchers with 1 million songs, 44,745 unique artists, 280 GB of data, 7,643 unique terms containing acoustic features such as pitch, timbre and loudness and much more.

Table 1 (adapted from [START_REF] Bertin-Mahieux | The million song dataset[END_REF]) collectively presents the aforementioned datasets by comparison of size and samples/audio availability.

The Dataset

The musical tradition of Greece is diverse and celebrated through its history. Greek music can be separated into two major categories: Greek traditional music and Byzantine music, with more eastern sounds [START_REF]Greek traditional music[END_REF]. Greek traditional (folk) music or "δηµoτ ική µoυσική", as it is most commonly referred to, is a combination of songs, tempos and rhythms from a litany of Greek regions. These compositions, in their vast majority, have been created by unknown authors, are more than a century old and are the basis for the Modern Greek traditional music scene. GAD contains many Greek traditional music songs, starting from the 60's and 70's as well as some current variants that are very popular nowadays.

Greek music contains special characteristics that are not easily to be found on the existing datasets. The unique instruments (bouzouki, lyra, laouto, etc) and the unique rhythms reflect on the audio feature extraction while the complexity of the Greek language can be an area of study for linguistic experts on lyric feature extraction.

Creation process

The audio data collection covers the whole range of Greek music, from traditional to modern. We made every effort to make the song and genre selection balanced in GAD as much as possible. The selection of the music tracks was made from personal CD collections, while some of the songs' recordings were available in live performances. The audio feature extraction process was applied on CD quality wave files (44,1KHz, 16 bit) using jAudio [START_REF] Mcennis | jaudio: An feature extraction library[END_REF] and AudioFeatureExtraction, a processing tool introduced by the MIR research team of Vienna University of Technology [START_REF]Audio feature extraction web service[END_REF].

For each song, the GAD additionally includes its lyrics. The lyrics included in the dataset have been retrieved among various sources, such as the web site stixoi.info [START_REF]stixoi info: Greek lyrics for songs and poetry[END_REF], and are used within for the purposes of academic and scientific research of MIR researchers. It should be noted that copyright of the lyrics remains with their respective copyright owners.

As far as genre classification is concerned, the following Greek musical culture oriented tags where used: Rembetiko, Laiko, Entexno, Modern Laiko, Rock, Hip Hop/R & B, Pop, Enallaktiko, that are described in the sequel (Section 3.2). Because of the different music styles that artists may adopt during their career, we performed listening tests to every song before choosing the appropriate genre tag. For mood annotation, the Thayer model [START_REF] Thayer | The biopsychology of mood and arousal[END_REF] has been adopted. To record the mood categories, we invited 5 annotators to listen and read the lyrics for one couple and refrain for each song. Then we computed a standard F-measure as a measure of the inter-annotator agreement [START_REF] Boisen | Annotating resources for information extraction[END_REF] and we found a value of 0.8 approximately. For clusters of mood with smaller F-measure, a discussion between the annotators was taken part in order to reduce controversy.

The dataset additionally includes a YouTube [22] link for each song that allows for access to the audio content. In order to identify the best match of the YouTube available alternative songs, for every song we attempted to identify the YouTube URL using the following criteria: number of views, number of responses, best audio quality and as close as possible performance as in the wave library where the features were extracted from.

The GAD is available as a download from the webpage of the Informatics in Humanistic and Social Sciences Lab of the Ionian University1 .

The content

The GAD contains audio and lyrics features and metadata for selected Greek Songs:

-1000 songs -1000 txt files with the lyrics -1000 best YouTube links -277 unique artists -8 Genres. We give the necessary explanations to distinguish the unique Greek genres.

• Pϵµπέτ ικo: 65 tracks. In urban centers with strong Greek presence, among other genres, it appeared a kind of folk called "Rembetiko". Major schools of Rembetiko were: Smyrnaiiki and the Piraeus school classic Rembetiko [START_REF] Ordoulidis | The greek laiko (popular) rhythms: Some problematic issues[END_REF]. "Zeibekiko", "karsilamas" and "hasapiko" are a few characteristic rhythms. • Λαικó: 186 tracks. As an evolution of Rembetiko are considered Greek Folk songs of decades of 1950-1960, which continues to evolve and sound today [START_REF] Liavas | The greek song: from 1821 to the 1950s[END_REF]. The transition to the so-called "Laiko" music is evident in the imposition of European instrument tuning, rhythms and now that the author can write songs with "harmony". • Eντ ϵχνo: 195 tracks. A complex musical work of art that combines Modern Greek music with poetry [START_REF] Sideras | The sung poetry[END_REF]. It differs from the Laiko mainly in verse, but also in music (instrumentation, style). • Moντ έρνo Λαικó: 175 tracks. The "Modern Laiko" is considered as the current evolution of popular music. Several pop elements, electronic sounds can be identified and is now the most common music heard in Greek live stages. The style and the rhythms have not changed much and the theme of the songs is adapted to the current daily problems. and "exciting" to "tired" and "serene" from 1-4. Valence values correspond to moods such as "sad" and "upset" to "happy" and "content" from A-D. Figure 1 shows the distribution of tracks by mood and we observe that the tracks with positive emotions (valence C, D), have variations in arousal and either are very calm (arousal 1) or intense stress (arousal 4). We also include all the lyrics for further feature extraction. The accumulated lyrics contain:

• 32024 lines • 143003 words • 1397244 characters
The data is available in two formats, HDF5 and CSV.

HDF5 is efficient for handling the heterogeneous types of information such as audio features in variable array lengths, names as strings, and easy for adding new types of features. Every song is described by a single file the contents of which are as shown in Figure 2. Each file has 2 groups: the folder "Audio Features" containing the three most important types of features, such as Timbral, Rhythm and Pitch [START_REF] Tzanetakis | Musical genre classification of audio signals. Speech and Audio Processing[END_REF] and the "Metadata" containing all the other information that is related to the song (i.e. title, artist, mood, genre and YouTube link). CSV is compatible for processing with Weka [START_REF] Holmes | Weka: a machine learning workbench[END_REF], RapidMiner [START_REF] Jungermann | Information extraction with rapidminer[END_REF] and other similar data mining platforms. The GAD provides the commonly used, on the discipline of MIR, audio feature sets in separate CSV files. Weka, besides the well-known options for preprocessing and classification, additionally offers attribute visualization. Figure 3 shows one such mapping between two timbral characteristics with sorting class the genre and more specifically, Rock and Entexno.

The main acoustic features are Timbral, Rhythm and Pitch. Most of the features were extracted using jAudio which also calculates derived features. These features are created by applying metafeatures to primary features.

-Timbral Texture Features: features used to differentiate mixture of sounds based on their instrumental compositions when the melody and the pitch components are similar. (251 features)

Future Direction of the Dataset

The GAD is not without issues that can be ameliorated in future versions. One of these issues pertains to the balance of the distributions of genre types in the data. In addition, classification of Greek music into genres can be confusing when use of only one tag is assumed. An extension of the dataset by addition of more data as well as a multi-label version of the dataset will eliminate those problems and it is already under development. Some of the main future actions that would greatly enhance the GAD, to name a few, are:

the inclusion of user generated tags (from tagging games or web-services), the collection of labels for mood and genre based on more users, the expansion of the number of songs (i.e. include latest top-chart songs), the refinement of genres by adding more detailed labels with descriptions, the balancing of moods and/or genres, the inclusion of scores for each song, the development of programming language wrappers.

Conclusion

The Greek Audio Dataset is, to the best of the authors' knowledge, the first complete attempt to create an annotated Greek audio dataset. Although its size is relatively small compared to latest datasets, it constitutes a good start for researchers who want to study how the Greek music reflects in the context of MIR. Following the methodology of the, ubiquitous in MIR research, Million Song Dataset, the GAD does not include the audio content and it uses the HDF5 format to store all the information. This makes it easy for any future enrichment with more features and musical information as well as for the utilisation of existing methodologies in order to access the data. In order to enhance further the dataset, it also contains lyrics information so as to provide for linguistic methods on MIR. In addition, the dataset is also available in CSV format that makes it very easy to explore it in data mining experiments using widely available software as WEKA and RapidMiner.

Fig. 1 .

 1 Fig. 1. Valence and Arousal on the GAD.

Fig. 2 .

 2 Fig. 2. View of a single track using HDFView.

Fig. 3 .

 3 Fig. 3. Visualizing Centroid and Rolloff for genres Rock and Entexno.

Table 1 .

 1 Comparison of existing datasets: size and samples/audio availability.

	dataset	# songs	includes samples/audio
	CAL500	500	No
	Greek Audio Dataset (GAD) 1000 No (YouTube links available)
	GZTAN genre	1000	Yes
	Magnatagatune	25863	Yes
	Million Song Dataset (MSD) 1000000	No
	MusiCLEF	1355	Yes
	RWC	465	Yes
	Swat10K	10870	No
	USPOP	8752	No

•

  Rock: 195 tracks. It also includes 80s Pop-Rock. • Hip Hop/R & B: 60 tracks. • Pop: 63 tracks. It also includes Dance-Club music style and older Greek disco hits. • Eναλλακτ ικó: 60 tracks. Although with the term "Enallaktiko" it is usually considered as "Alternative Rock" [14], we include tracks fusing Greek modern kinds of music. Pop Rock and Entexno elements can be found in this class. -16 Mood taxonomies: 2 dimensions, valence and arousal, which divide a 2-dimensional emotive plane into 4 parts by having positive/high and negative/low values respectively. Arousal and valence are linked to energy and tension, respectively. Arousal values correspond to moods such as "angry"

http://di.ionio.gr/hilab/gad