Dimitris Chousiadas
email: dim.chousiadas@gmail.com

Andreas Menychtas
email: amenychtas@unipi.gr

Panayiotis Tsanakas
email: tsanakas@grnet.gr

Ilias Maglogiannis

Advancing quantified-self applications utilizing visual data analytics and the Internet of Things

Keywords: Quantified-Self, Internet of Things, Visual Data Analytics, Personal Informatics, Sensors, Activity Tracking, eHealth, Containers

The exponential growth of the number and variety of IoT devices and applications for personal use, as well as the improvement of their quality and performance, facilitates the realization of intelligent eHealth concepts. Nowadays, it is easier than ever for individuals to monitor themselves, quantify and log their everyday activities in order to gain insights about their body performance and receive recommendations and incentives to improve it. Of course, in order for such systems to live up to the promise, given the treasure trove of data that is collected, machine learning techniques need to be integrated in the processing and analysis of the data. This systematic and automated quantification, logging and analysis of personal data, using IoT and AI technologies, has given birth to the phenomenon of Quantified-Self. This work proposes a prototype decentralized Quantified-Self application, built on top of a dedicated IoT gateway, that aggregates and analyses data from multiple sources, such as biosignal sensors and wearables, and performs analytics and visualization on it.

Introduction

The technological developments of the last decade contributed to an abundance of commercially available and affordable biomedical sensors, activity trackers and wearables. Moreover, these sensors come with specially developed smartphone applications, which enable the users to collect data about themselves, in a systematic and automated manner, and have access to reports and analyses, providing insights and valuable feedback.

The types of data that is collected in the context of Quantified-Self are virtually countless, including but not limited to: health, physical activity, nutrition, psychological and mental state or environment [START_REF] Swan | The quantified self: Fundamental disruption in big data science and biological discovery[END_REF]. For each of these categories, several applications exist that focus on the collection and analysis of relevant data, all of them aiming at highlighting the individual characteristics of the user's lifestyle, identifying his or her strong and weak points and triggering desirable behavior change. Machine learning and big data are technologies that are applied in this area to facilitate that analysis of the data and the creation of personalized knowledge for users and medical experts [START_REF] Fawcett | Mining the quantified self: personal knowledge discovery as a challenge for data science[END_REF].

According to research, the mere monitoring and logging of values, such as body weight, is effective in itself, by making the user conscious of its importance and in turn, leading to lifestyle changes [START_REF] Mcgrath | Wellness, fitness, and lifestyle sensing applications[END_REF]. The tracking of values over time, additionally, produces insights regarding the user's progress, highlighting the direct relation between their efforts and the results (e.g. systematically going to the gym leads to a decrease of body fat percentage [START_REF] Munson | Exploring goal-setting, rewards, self-monitoring, and sharing to motivate physical activity[END_REF]). State-of-the-art solutions include mobile applications for GPS tracking of cardio activities, websites for tracking body weight, fat percentage and bone mass using connected body scales, mobile apps for blood pressure monitoring, apps with integrated activity tracking [START_REF] Pcmag | The Best Fitness Trackers of 2017[END_REF].

Research has shown, however, that over 50 percent of people using similar applications have given up on them after a few months [START_REF] Ledger | Endeavour Partners Report: Inside Wearables: How the Science of Human Behavior Change Offers the Secret to Long-term Engagement[END_REF]. Some of the reasons include absence of a holistic approach to data collection and analysis, and lack of customization capabilities [START_REF] Menychtas | Automated integration of wireless biosignal collection devices for patient-centred decision-making in point-of-care systems[END_REF]. The constant emergence of new applications and corresponding wearable devices, usually incompatible with the previous ones, hardens the adoption of a vendor-free solution like the one proposed in this work. Furthermore, there exist privacy concerns in current Quantified-Self applications, since, typically, the collected data is permanently stored in the cloud, giving ownership and, thus, control of it to the application provider. The greater the sensitivity of the data, the greater the risk for the user, in case of data loss, data manipulation or hacker attacks. Moreover, a lot of providers, in order to maintain the free-of-charge status of their Quantified-Self services, sell the data to third parties (e.g. for advertising or marketing purposes) [START_REF] Leibenger | Privacy challenges in the quantified self movement-an EU perspective[END_REF].

The proposed Quantified-Self approach aims to provide the technical and functional background to support data collection from multiple sources, in the context of health and activity tracking, the analysis of that data and the visualization of the results, while taking into consideration the aforementioned shortcomings of the state-of-the-art applications. On the one hand, it allows the user to combine on the same platform several devices, resolving the compatibility issues by exploiting the advancements of Bluetooth communication technologies, and particularly of BLE [START_REF] Mackensen | Bluetooth Low Energy (BLE) based wireless sensors[END_REF]. On the other hand, being built on top of the innovative software stack of AGILE [1] and deployed on a Raspberry Pi, instead of the typical hybrid deployment in smartphone and cloud, it gives ownership of the data over to the user.

The rest of the paper is structured as follows: Section 2 describes the related work in the area of Quantified-Self while section 3 presents the proposed solution and the implementation of the system. The resulting application, with the data analysis and visualization features is presented in section 4. Finally, section 5 discusses the advancements of this approach compared to other approaches, concludes this work and highlights the future plans.

Related Work

A Quantified-Self application, such as the one presented here, aims at, not simply quantifying and gathering data, but also exploiting data analysis methodologies and tools to give feedback to the user and promote behavior change. F. Bentley et al. [START_REF] Bentley | Health Mashups: Presenting statistical patterns between wellbeing data and context in natural language to promote behavior change[END_REF] have studied the ways such change can occur, particularly in the context of health and wellbeing, highlighting the importance of the general (environmental and personal) context of the user's life, a conclusion that has also been reached by Choe et al. [START_REF] Choe | Understanding quantified-selfers' practices in collecting and exploring personal data[END_REF], who have shown that not tracking context and triggers leads to decreased chances to improve outcomes. This necessitates a holistic approach to data collection, in order to capture and quantify the whole spectrum of this context while in parallel, given the multifaceted nature of the user's everyday life, sophisticated data mining techniques need to be used, in order to uncover hidden, meaningful patterns in it. F. Bentley's et al. research, moreover, explores how users perceive and react to recommendations about behavior change, suggesting that the use of natural language, instead of charts or numbers, can sometimes be more effective.

With regards to the ways IoT data can be harvested, Chen, Feng, et al [START_REF] Chen | Data mining for the internet of things: literature review and challenges[END_REF] have presented an overview of the various data mining techniques, applied in the context of IoT. What distinguishes IoT systems, concerning data mining, is the requirement for realtime data processing, as well as processing at the edge of the network (Edge Analytics). As an example, such requirements exist in telemonitoring systems, such as the ones presented by C. Panagopoulos et al. [START_REF] Panagopoulos | Utilizing a Homecare Platform for Remote Monitoring of Patients with Idiopathic Pulmonary Fibrosis[END_REF] and K. Aguilar et al. [START_REF] Aguilar | Bringing care home: how telemonitoring can expand population health management beyond the hospital[END_REF], where doctors need to monitor patients and receive notifications in real time. Strongly connected with the above is the research of J. Li et al. [START_REF] Li | EHOPES: Data-centered Fog platform for smart living[END_REF], which investigates the potential IoT applications in a more general context than that of health and medical care. More concretely, through the lens of fog computing the researchers highlight the ways Smart Living can be advanced, in various aspects of everyday life (work, entertainment, energy consumption and, naturally, health) through the quantification and analysis of data.

Regarding the specific sensors that are used, as well as the architecture of the sensor network and the flow of collected data, the works of A. Kor et al. [START_REF] Kor | SMART-ITEM: ioT-enabled smart living[END_REF] and A.H.T.E. De Silva et al. [START_REF] De Silva | Development of a wearable tele-monitoring system with IoT for bio-medical applications[END_REF] make obvious the relevance and applicability of the Internet of Things and Quantified-Self concepts in healthcare and mHealth. Finally, B. D. Weinberg et al. [START_REF] Weinberg | Internet of Things: Convenience vs. privacy and secrecy[END_REF] have examined the privacy concerns that accompany the transition from Web 2.0 to Internet of Things. Conflicts of interests between users and organizations have elevated data ownership to a crucial issue. In the context of IoT, where data is generated automatically and continually, the question of who owns it is, indeed, of paramount importance.

System Architecture

The Quantified-Self concept is targeting data acquisition on aspects of a person's daily life in terms of inputs (e.g. food consumed, quality of surrounding air), states (e.g. mood, arousal, blood oxygen levels), and performance (mental and physical activities) through a modern, health centric, social and mobile enabled, communication platform that resides in the gateway (in terms of collecting and visualizing data). However, the ecosystem of biosignal sensors and activity trackers is very diverse from both business and technical perspectives. The vendors have also different business and market strategies, and in many cases these strategies are reflected in the devices themselves in technical and operational level. The open source designs for such devices that can be used for producing or communicating with them are uncommon. The hardware and firmware of the majority of the devices is proprietary and closed source, hence establishing communication with the device as a third-party required guidance from the vendors either by providing access to the communication protocols or through libraries / APIs. Therefore, what is usually happening today, is that the users are obliged to use each wearable and sensor with different, companion applications, which store the date to independent systems.

The unique characteristic of the proposed approach is that the Quantified-Self application is deployed on top of an IoT Gateway [START_REF] Menychtas | A Versatile Architecture for Building IoT Quantified-Self Applications[END_REF], which eliminates the need for additional applications or hardware. Wearable activity trackers and medical sensors automatically communicate with the gateway whenever within range, and offload the most recent data. However, for the wearables that the manufacturers provide only API libraries for Android and iOS, or specific applications for synchronizing the data with manufacturers' cloud platforms, neither of which is compatible with the proposed setup. In these cases, the activity data offloading is achieved via Internet and the public APIs of the cloud platform.

Fig. 1. Quantified-Self Concept

Fig. 1 presents the general architecture for the proposed Quantified-Self application. Each user connects the various activity tracking devices and biosignals sensors (such as oximeters, blood pressure monitors or glucometers), to monitor their daily physical activity and physical condition. The user is able to visualize and manage the locally stored data and create reports. In parallel, the data are processed and analyzed on the gateway, with personalized recommendations being sent to the users, encouraging them to reach their physical activity goals. In order to address the requirements for modularity and extensibility, the architectural model of the application is based on containers. More specifically, four main containers are deployed on the gateway to provide the required functionality of the application: a) IoT/Sensors for managing the communication with the various sensors and acquiring the biosignal measurements, b) Storage for storing the data locally, c) Analytics for the periodic and ad-hoc analysis of the data in order to create valuable knowledge for the users' health condition and activity and finally d) the Quantified-Self Web Application which coordinates the previous containers and provides a UI with modern reporting and data visualization functionality. All components of the application (the core application, database and Shimmer) run on a Raspberry Pi, on top of the AGILE Software Stack [START_REF] Menychtas | A Versatile Architecture for Building IoT Quantified-Self Applications[END_REF], on a Linux based operating system. The container-based deployment approach of AGILE was adopted, ensuring the virtual isolation of the components from each other and providing the required flexibility for the maintenance and extension of the system with additional features.

IoT Sensors Integration

The AGILE IoT Gateway software stack is a flexible architecture built for single board computers running independently or on the edge of the network. By providing developers ready, short distance networking abilities, such as Bluetooth or Zigbee, it allows acquisition from local sensors, view and local storing of data. For the integration of the Quantified-Self biosignal sensors and activity trackers, the respective components of AGILE were used. Using the harmonized REST API, the integration of any sensor is simplified in the cases where specific commands or operational workflows are required, custom device drivers are implemented. This approach follows a registration / publish / subscribe device model. Each device needs to be registered before use and the application can subscribe to the sensor data endpoints in order for published data to be streamed to the user. This model works well with BLE devices, which use GATT notifications [4] to stream data to the host.

Data Management

Attempting to combine on the same platform data from different medical devices, such as oximeters, glucometers, smart scales and blood pressure monitors, introduces significant complexities regarding the storage and manipulation of that data. Integrating, furthermore, data from the cloud APIs of Fitbit [9] and GoogleFit [10], only increases these complexities. The homogeneous and consistent manipulation of the collected data, however, is a necessary prerequisite for any meaningful processing to take place in the system. To this end, the tools developed by the open-source project Open mHealth [22] proved particularly useful. These tools include, a collection of schemata that define the structure of health data. Specifically, these schemata standardize the representation of health data following the syntactic rules of JSON, i.e. key-value pairs, defining the keys to be used for all types of health data. For the purposes of the proposed Quantified-Self application this standardization facilitates the easy and smooth integration of the data, regardless of the source, and simplifies the architectural choices for the database and the REST API. In the case of the medical sensors that communicate with the gateway through BLE, the conversion of the data to JSON objects that comply with the Open mHealth rules takes place during that communication.

Data Analytics

The amount of the activity and medical data that is collected gives ample opportunity for the analysis of that data and the discovery of patterns in it, providing valuable feedback to the user. The analytics component is based on the python tools of Scikit-learn [START_REF]Scikit-learn, machine learning in Python[END_REF], Matplotlib [START_REF]Matplotlib, plotting library for the Python[END_REF] and Numpy [START_REF]Numpy, package for scientific computing with Python[END_REF] which have been deployed on a specific container configured to communicate with the database and the web application. Scikit-learn, specifically, is a well-known machine-learning library for Python, featuring classification, regression and clustering algorithms, including the k-means algorithm that was extensively used in the proposed application. Numpy is numerical library for Python that provides, primarily, support for multi-dimensional arrays and matrices, designed to interoperate with Scikit-learn. The data visualization was realized utilizing Matplotlib, which is a plotting library for Python, along with the drawing capabilities of the JavaScript programming language. Since all data are stored locally, the data analytics component periodically, or upon user's request, processes the activity and biosignal data in order to create reports and aggregated information to be presented in the web app. In the current implementation data clustering and correlation techniques have been used, and the results are also exploited to provide motivational messages to the users. Regarding the clustering techniques, data cleaning and pre-processing was necessary. This included removing the outliers from the datasets, normalizing the measurements values, as well as adding potential missing values, either by replacing them with the mean value of the dataset or by filling in the respective values from the nearest data vector in the Euclidean space. Once the data cleaning was done, the data was aggregated, in most cases by day, and the k-means algorithm was applied on that aggregated data.

Quantified-Self Web Application

The user interacts with the system through a web application, which is also hosted on the IoT Gateway and is only accessible through the local home network and does not expose itself or the user's data to public networks or the Internet. Through the application, the users are able to manage the various IoT sensors (e.g. register and initialize them) and the stored data by communicating with the respective components of the system. The web application also visualizes the user's activity data, collected both from local sensors (if available) and the APIs of GoogleFit and/or Fitbit. The activity consists of the number of steps taken each day, the duration of physical activity every day, as well as the calories burned. Examples of such visualizations are shown in Fig. 2 and Fig. 3, where the user can see the number the steps taken during the last 7 days or the average duration of the user's physical activity for each day of week for the last year. Another category of data is the biosignals, namely heart rate, blood pressure, oxygen saturation, glucose levels and body weight. These are collected through specific sensors that communicate with the AGILE Gateway or directly, as user input. Similar visualization exists for that second category as well.

On the home page of the application, the user can find general results of the statistical analysis of the data, such as the ones shown in Fig. 4 and Fig. 5. Based on these, relevant messages appear to encourage and motivate the user. Furthermore, the system informs the user of how the overall community of users is doing, and he or she compares to it. Finally, there is also the option of setting goals, regarding the performance of the user. Ultimately, these features aim to maximize the user's commitment and bring about improved performance.

Experimentation

The proposed Quantified-Self application was provided to real users in order to assess the visual data analytics functionality. The biosignals data were analyzed using wellknown k-means algorithm. As an example of this, in particular, the heartbeat and oxygen saturation data have been clustered in three clusters. Specifically, each data point represents the average heartbeat and oxygen saturation measurements in a specific day and the choice of three clusters corresponds to the intuition that, for any given user, there are days with good measurements (green cluster), days with medium measurements (red cluster) or days with bad measurements (blue measurements). Furthermore, using the elbow method, the choice of three clusters was shown to be close to optimal, and it was, thus, selected due to its to intuitive interpretation. Once the clustering has been made, each day can be assigned to one cluster, informing, thus, the user, of the quality of his/her measurements that day. The results of the clustering can be presented to the user with a pie chart, as in Fig. 6 with the percentage of the user's measurements in each cluster.

Fig. 6. Percentage of measurements in the 3 clusters

Another example of how clustering can provide useful insights to the user is presented in Fig. 7. Specifically, each day of week is broken down in the percentages of the measurements of that day that belong to each one of the 3 clusters.

Fig. 7. Clusters per day of week

Moreover, a visualization is also included in the web application in order to provide valuable information to the user presenting the results of the clustering analysis as a time series. Concretely, to each one of the, say, last 30 days, a cluster 1 to 3 is assigned, with cluster 1 representing good measurements, cluster 2 medium measurements and cluster 3 bad measurements. It is possible to see, then, how the clustering changes or, in other words, how the quality of the user's measurements changes between days. In Fig. 8 the visualization of such an analysis can be seen. Specifically, the blue columns represent the cluster values for each day in the last month, whereas the orange ones present the delta of these values, namely, how small or big the change is between days. In this way, significant changes, i.e. anomalies can be traced.

Along with the clustering, the application computes the correlation between the two datasets, using the Pearson Correlation Coefficient, in order to indicate the strength of the linear relationship between the data (for the clustering example presented above, the coefficient is equal to -0.22). Generally, the correlation coefficient can be calculated for any combination of the data types in the application, enabling the user to receive feedback in the form of natural language phrases, such as: "On days when you run more than 20 minutes, your oxygen saturation level increases".

Discussion and Conclusions

The competitive advantages of the proposed Quantified-Self application are twofold. The main innovation, on the business level, is the fact that the data remain on the user's gateway, while data processing and analytics are provided locally. On the technical level, the infrastructure of the AGILE Gateway supports communication with and integration of several devices on the same platform. These two, combined, bring several benefits for the end-user, as well as for the developers' community. Furthermore, the visualization that the application employs provide valuable feedback to the user. The combination of different charts, shapes, coloring and phrasing of the results aim at enabling the users to intuitively understand and interpret their behavior and performance. The application proposes a fully automated solution that requires minor engagement of the user, making it suitable for people unfamiliar with technology. What further makes the solution user-friendly and advances its ease-of-use is the fact that the underlying IoT gateway can be built using affordable, commodity hardware. Moreover, the native support for modularity, extensibility and high customization, without the need to speed effort across the different layers and components of the gateway, is one of the most important benefits for the Quantified-Self application allowing for its continuous evolution and following the users' growing requirements and the new technological solutions and trends in the domain.

The application will be further extended by employing more sophisticated gamification techniques to advance the goal setting and data sharing features, as well as additional machine learning methods, like classification or regression, which would be used to predict value for the user, such as future body weight. Furthermore, there is also room for the integration of more interactive and informative visualization tools, which would build on top of the existing ones.

Fig. 2 .Fig. 3 .

 23 Fig. 2. Number of steps taken the last 7 days

Fig. 4 .Fig. 5 .

 45 Fig. 4. Information about today's activity presented in the Homepage

Fig. 8 .

 8 Fig. 8. Time series of the clusters of last 30 days: the blue bars represent the cluster to which each one of the last 30 days belongs, whereas the orange bars represent the change of clusters between days

Acknowledgements. Dr. A. Menychtas acknowledges the Greek State Scholarship Foundation (ΙΚΥ). This research was implemented with a Scholarship from IKY and was funded from the action "Reinforcement of postdoctoral researchers" of the programme "Development of Human Resources, Education and Lifelong Learning", with priority axes 6,8,9 and it was co-financed by the European Social Fund-ESF and the Greek State.


Juan Sánchez-González

Oriol Sallent

Jordi Pérez-Romero

Space/Time Traffic Fluctuations in a Cellular Network: Measurements' Analysis and Potential Applications

Keywords: Space/time traffic analysis, cellular networks, SON, Mobility Load Balancing, Coverage and Capacity Optimisation, energy saving

The characterization of the space/time traffic profiles in a cellular network can be of high interest for automating the operation of future networks, since the knowledge extracted from the traffic fluctuations in a cell and its neighbours can be effectively exploited by different optimisation functions. In this context, this paper takes as an input a set of real traffic measurements in a cellular network deployed in a large city and analyses, on a per cell basis, the traffic profile characteristics at different time scales (week, day, hour). Then, the analysis is extended to the space dimension by considering the traffic of one cell in relation to that of its neighbours. This allows identifying traffic complementarities between neighbour cells at different time scales that can be exploited by certain optimisation functions, as illustrated in the paper with specific examples.

Introduction

In the last years, cellular networks have witnessed an explosive increase in the traffic volume due to the massive penetration of new wireless devices (such as smartphones and tablets). It is envisaged that this trend will continue in the future with the emergence of new bandwidth-intensive applications such as high definition video, 3D, augmented reality/virtual reality, etc. and with the demand for wireless Machine-to-Machine communications. According to [START_REF]Cisco Visual Networking Index: Forecast and Methodology[END_REF], the worldwide mobile devices are expected to consume more than 48.3 Exabytes per month by 2021, which represents a sevenfold increase between 2016 and 2021. To cope with these challenging requirements, research and standardization work is nowadays focused on the development of the 5th Generation (5G). 5G networks are expected to be characterized by unprecedented flexibility to support a wide range of requirements and network functionalities associated to different application scenarios, going beyond the provision of enhanced Mobile BroadBand (eMBB) services to incorporate services associated to other sectors such as automotive, utilities, smart cities, etc. The resulting heterogeneity and the required flexibility will make the network planning and operation much more complex, thus claiming for an increase in the degree of automation of these processes.

Legacy systems such as 2G/3G/4G already started the path towards a higher degree of automation through the introduction of Self-Organizing Network (SON) functionalities [START_REF] Ramiro | Self-Organizing Networks. Self-planning, self-optimization and self-healing for GSM, UMTS and LTE[END_REF]. However, nowadays, the volume and variety of measurement data that can be obtained from the mobile network is overwhelming. Due to the recent developments of big data technologies [START_REF] Liu | On big data analytics for greener and softer RAN[END_REF], it is envisioned that 5G SON will better exploit the huge amount of data available by a Mobile Network Operator (MNO) [START_REF] Imran | Challenges in 5G: How to empower SON with big data for enabling 5G[END_REF] [START_REF] Pérez-Romero | Artificial Intelligence-based 5G Network capacity planning and operation[END_REF]. The target is to efficiently handle this big amount of data and turn it into value by gaining insight and understanding data structures and relationships, extracting exploitable knowledge and deriving successful decision-making.

Obtaining knowledge of space/time traffic profiles can be of high interest for network optimisation since the analysis of traffic spatio/temporal fluctuations in a given cell and its neighbours can be effectively exploited by different decision making processes associated to the planning and operation of the network. In this direction, several works have proposed methodologies to analyse the base stations traffic to obtain knowledge of the network usage to be used for network optimization. As an example, [START_REF] Zubair | Geospatial and temporal dynamics of application usage in cellular data networks[END_REF] provides a geospatial and temporal characterization of application usage in a real 3G cellular data network. In particular, the paper studies where and when users are running the different services (e-mail, web-browsing, etc.) in order to improve the radio resource provision in the network. Similarly, [START_REF] Ma | Modelling social characteristics of mobile radio networks[END_REF] proposes a methodology to identify different types of cells (i.e. cells deployed in residential areas, commercial areas, etc.) by evaluating several features such as traffic fluctuation, user mobility, temporal pattern of traffic generation and user service preferences.

Some works, such as [START_REF] Miao | MSFS: Multiple Spatio-Temporal Scales Traffic Forecasting in Mobile Cellular Network[END_REF][9], make use of real measurements to extract spatiotemporal traffic models that are used for traffic forecasting. The characterization and modelling of spatio-temporal fluctuations in the access network by analyzing real measurements in cellular networks has been addressed by [START_REF] Lee | Spatial Modeling of the Traffic Density in Cellular Networks[END_REF][11] [START_REF] Li | The prediction analysis of cellular radio access network traffic: From entropy theory to networking practice[END_REF]. In particular, according to [START_REF] Wang | An Approach for Spatialtemporal Traffic Modeling in Mobile Cellular Networks[END_REF], the spatial distribution of traffic can be assumed to be log-normal distributed around certain geographical areas with high traffic demands. However, the temporal evolution of traffic can be represented as the summation of different sinusoids with periods of 8, 12 and 24 hours to capture the user social behavior, because mobile users tend to have repetitive behaviors in periods of one day, half a day and working hours. Some works make use of Artificial Intelligence mechanisms in order to extract knowledge of the time traffic evolution. As an example, [START_REF] Hammami | Network planning tool based on network classification and load prediction[END_REF] compares Support Vector Machine and K-means methodologies for the classification of cells with high load levels during all the day, cells with a peak load level in the morning and cells with a peak load level in the evening. Classification algorithms are used in [START_REF] Pérez-Romero | On Learning and Exploiting Time Domain Traffic Patterns in Cellular Radio Access Networks[END_REF] to identify cells with periods of low traffic level and cells with peaks of traffic and apply this knowledge in different management processes such as energy saving or spectrum planning. Similarly, [START_REF] Kim | Traffic-aware energy-saving base station sleeping and clustering in cooperative networks[END_REF] proposes an algorithm that exploits knowledge of spatio-temporal traffic fluctuations to save energy by switching off certain base stations without reducing the network capacity.

While the above papers have addressed different strategies to analyze the traffic in the different cells, none of the previous works have focused on the identification of spatio-temporal traffic complementarities among neighbor cells and how to exploit them for optimizing the operation of the network. In this context, this paper provides two main contributions. First, it presents a statistical analysis of a bunch of realistic traffic measurements obtained from hundreds of cells in a large city to evaluate at what extent fluctuations exist in real traffic when considering, on a per cell basis, the traffic profile characteristics at different time scales (week, day, hour). Second, the analysis is extended to incorporate the spatial dimension by illustrating, through several examples, some existing traffic complementarities between neighbour cells at different time scales. For each example, the paper discusses possible network reconfiguration actions that can make use of the identified complementarities.

The rest of the paper is organised as follows. Section 2 presents the statistical analysis of the real traffic measurements in the time domain, while Section 3 presents examples of traffic complementarities between cells and possible ways to exploit them for the optimisation of the radio resource usage. Finally, Section 4 summarizes the main conclusions.

Analysis of traffic measurements in time

The available measurements have been obtained from 423 cells that belong to a real cellular network deployed in the city centre of one of the major cities in Spain. For each cell, the measurements have been recorded during one week in periods of 15 minutes. For the analysis of the time evolution of the cell traffic, different averaging periods (i.e. 2, 8 and 24 hours) have been considered. Due to confidentiality reasons, the presented traffic load has been normalised. In any case, this normalisation does not affect the analysis of the results, since we are only interested in relative traffic variations between cells.

Considering the measurements of all the cells, Fig. 1 shows the 10 th , 50 th and 90 th percentile of the normalised average daily traffic from Monday to Sunday (i.e. in the presented results, X th percentile represents the average daily traffic that is not exceeded by X% of the cells). It is observed that, over 24 hours period (i.e., on a per-day view) and over the whole scenario, traffic keeps at a rather constant level during weekdays and lowers during weekends. Nevertheless, the statistical analysis made at a shorter time scale provides further knowledge. As an example, Fig. 2 shows the 10 th , 50 th and 90 th percentile of the average traffic over 2 hours periods and over the whole scenario from Monday to Sunday. This view allows observing intra-day traffic variations, exhibiting higher traffic levels during working hours (i.e., from 8AM to 6PM).

In order to assess at what extent the different cells exhibit different traffic behaviours, the average daily traffic per cell from Monday to Friday is analysed. For each day, each cell is classified as having low (L), medium (M) or high (H) traffic. Based on this, it is found that 84% of the cells are classified in the same category (L, M or H) every day from Monday to Friday. Therefore, the other 16% of the cells exhibit significant daily traffic variations during weekdays. Fig. 3 depicts some examples of these cells. For example, Cell_264 has L traffic from Monday to Thursday and M traffic on Friday. In turn, Cell_93 has only H traffic on Thursday. In some cases, a reason to explain the traffic variation from one day to another can be found (e.g., Cell_93 is located in an area where there is a weekly street market on Thursday), while in other cases the source of the traffic fluctuation remains unclear. The average traffic over periods of 8 hours (i.e. 0h-8h, 8h-16h and 16h-24h) has been also analysed from Monday to Friday. Again, for each period of 8 hours, each cell is classified as having L, M or H traffic. Table 1 presents the percentage of cells classified in the same category every day from Monday to Friday and the percentage of cells classified differently in some days. It is observed that, at night, around 89% of the cells are classified equally (i.e. with L traffic) all days while more differences are observed in the evening. Fig. 4a and Fig. 4b show examples of cells with different classification on weekdays during the periods 8h-16h and 16h-24h, respectively. Note, for example, that Cell_264 has L traffic from Monday to Thursday while it has M traffic on Friday morning and H traffic on Friday evening. The reason of this behaviour is that Cell_264 is located in one of the most important entertainment and shopping areas of the city where the population density is higher especially on Friday evening. In turn, Cell_93 has a peak traffic on Thursday morning, while the traffic is M or L for the rest of the 8h periods of the week. The H traffic on Thursday was already identified in Fig. 3. A closer look allowed identifying that the H traffic is mostly during the working hours, as the traffic increase is due to the presence of the weekly street market.

Analysis of traffic measurements in time and space

Obtaining knowledge of spatio-temporal traffic fluctuations and traffic complementarities between neighbour cells will clearly be very valuable for the optimisation of the radio resource usage in the different cells. Specifically, the identification of traffic complementarities can be helpful as an input for certain SON functions that automate the operation of the network (see e.g. [2][16] for background reading about SON functions). In this respect, both self-optimization functions, which intend to tune certain network parameters according to specific performance targets, and selfplanning functions, which intend to automate the decisions of deploying new nodes in the network, can benefit from the analysis of traffic in the different cells and the identification of traffic complementarities. Depending on the time scale of the discovered complementarities, different kind of actions over the network may be done.

In the following, some examples of traffic complementarities found in the analysed scenario are presented and possible network reconfiguration actions are discussed.

3.1

Case study #1 among neighbour cells, it is possible to see in Fig. 5b, that the average traffic of neighbour Cell_1 is substantially lower than that of Cell_119 in all the considered time scales. As a result, the self-planning function can identify that no new cell is actually needed because the excess of traffic in Cell_119 can be absorbed by Cell_1. To achieve this, the network can rely on either the Coverage and Capacity Optimization (CCO) or the Mobility Load Balancing (MLB) self-optimization functions. Specifically, the CCO function can adjust the antenna tilts of the involved cells so that their coverage areas fit better their traffic demands. For example, the CCO function can decide an increase in the antenna tilt of Cell_119 and a reduction of the tilt of Cell_1. This would allow that more users in the area will tend to connect to Cell_1 and would reduce the overload situations of Cell_119. Alternatively, a similar effect can be obtained through the MLB function, which would adjust the cell reselection and handover thresholds to force that more users in the area connect through the less loaded Cell_1.

3.2

Case study #2

Fig. 6a presents the geographical location of several cells in an area in the city centre, while Fig. 6b shows the weekly and daily average traffic for Cell_202 and Cell_265. Although the weekly average traffic in both cells is very similar, the daily average traffic shows some complementarities between these two cells. As shown in Fig. 6b, Cell_265 has a relatively high load level on Friday and Saturday since this cell is located in an important shopping and entertainment street in the city, where people usually go shopping in the weekend. On the other hand, Cell_202 has higher traffic levels during the weekdays since it is located in a nearby business street, where people usually stay during working hours. Based on this, the system could rely on the CCO or MLB functions. For example, two different configurations could be considered for setting the antenna tilts and better adapt to the different traffic demands. On Friday and Saturday the selected configuration would consist in a higher tilt in Cell_265 and a lower one in Cell_202. This would allow shedding traffic from Cell_265 to Cell_202. For the rest of the days, the configuration would consist in a lower tilt in Cell_265 and a higher one in Cell_202 in order to shed traffic from Cell_202 to Cell_265.

3.3

Case study #3

Fig. 7a shows the geographical locations of Cell_36 and Cell_49, while Fig. 7b presents the normalised daily and 8-hour average traffic in both cells. Cell_36 is deployed in an entertainment area of the city with plenty of restaurants and shops, while Cell_49 provides coverage to some office buildings located in the harbour area. Although the daily average traffic from Monday to Thursday is very similar in Cell_36 and Cell_49, the traffic averaged over periods of 8 hours reveals that there is a clear complementarity between the morning and evening traffic served by Cell_36 and Cell_49 in the weekdays. In particular, Cell_49 exhibits a substantially high traffic during mornings, while at the same time the traffic in Cell_36 is lower. The situation reverses during evenings and nights and during the weekend, when the traffic in Cell_36 is higher than that of Cell_49. The identification of this complementarity suggests that the high traffic situations in either one or the other cell could be solved by shifting traffic from the highly loaded cell to the less loaded one. Again, this can be achieved through the CCO/MLB functions, so that traffic would be shed from Cell_49 to Cell_36 in the morning while traffic would be shed from Cell_36 to Cell_49 in the evening and during the weekends.

Conclusions

This paper has presented a statistical analysis of spatio-temporal traffic fluctuations and complementarities among neighbour cells by processing real measurements obtained from hundreds of cells deployed in a large city. Besides well expected observations, such as that the average traffic per day keeps at a rather constant level during weekdays and that higher traffic levels are observed during working hours in case of shorter averaging periods (e.g. 2h, 8h), the measurements have revealed that around 16% of the studied cells present significant inter-day traffic variations during weekdays. In some cases, an explanation can be found (e.g. a street market that occurs on a specific day of the week in the area of the cell, thus causing a traffic increase on that day), while in other cases the explanation remains unknown. The analysis has also jointly considered the traffic of a cell and its neighbours. This study has allowed the identification of some examples of areas exhibiting traffic complementarities among cells at different time scales. The detection of these complementarities can be valuable for supporting the operation of certain SON functions and optimizing the behaviour of the network. In particular, the presented examples illustrate situations in which the CCO/MLB functions can adjust the tilt or the handover and cell reselection thresholds to shift traffic from a highly loaded cell to a less loaded neighbour. The application of these actions avoids having to deploy a new cell to cope with the high load situations. Similarly, another example has illustrated the possibility of exploiting the traffic complementarities for energy saving purposes, identifying cells that can be switched-off during night periods.

Fig. 1 .Fig. 2 .

 12 Fig. 1. 10 th , 50 th and 90 th percentile of the average daily traffic.

Fig. 3 .

 3 Fig. 3. Traffic level evolution during weekdays.

Fig. 4 .

 4 Fig. 4. Traffic level during weekdays: a) in the period 8h-16h; b) in the period 16h-24h

Fig. 5a presentsFig. 5 .

 5 Fig. 5a presents a region with the geographical location of some cells of the network, while Fig. 5b shows the evolution of the daily and 2-hour averaged traffic during the week in Cell_119 and Cell_1. It is observed that Cell_119 presents a high level of traffic during morning and evening and lower traffic values at night. In turn, Cell_1 is located in an area with lower population density and, consequently, it has lower traffic levels. If the traffic was analysed separately on a per-cell basis, the high traffic level experienced by Cell_119 would suggest a planning decision involving the deployment of a new cell in its proximity or an increase in the capacity of Cell_119, e.g. by deploying a new carrier. Nevertheless, if the analysis considers the complementarities

Fig. 6 .

 6 (a) Considered scenario; (b) Normalised weekly and daily averaged traffic for Cell_202 and Cell_265.

Fig. 7 .

 7 (a) Considered scenario; (b) Normalised daily and 8-hour averaged traffic for Cell_36 and Cell_49.

FigFig. 8 .

 8 Fig. 8a presents the geographical location of several cells in another area of the city center, while Fig. 8b presents the daily and 8-hour average traffic in Cell_111 and

Table 1 .

 1 Percentage of cells classified under the same and under different categories in all the days from Monday to Friday.

	Period	% of cells with the same traffic	% of cells with different traf-
		level (H, M or L) in all days	fic levels in different days
	Night (0h-8h)	89%	11%
	Morning (8h-16h)	83%	17%
	Evening (16h-24h)	74 %	26%

Acknowledgements. This work has been supported by the EU funded H2020 5G-PPP project 5G ESSENCE under grant agreement 761592 and by the Spanish Research Council and FEDER funds under RAMSES and SONAR 5G grants (ref. TEC2013-41698-R and TEC2017-82651-R).

0.8 0.9 1 Mon_0h Mon_4h Mon_8h Mon_12h Mon_16h Mon_20h Tue_0h Tue_4h Tue_8h Tue_12h Tue_16h Tue_20h Wed_0h Wed_4h Wed_8h Wed_12h Wed_16h Wed_20h Thu_0h Thu_4h Thu_8h Thu_12h Thu_16h Thu_20h Fri_0h Fri_4h Fri_8h Fri_12h Fri_16h Fri_20h Sat_0h Sat_4h Sat_8h Sat_12h Sat_16h Sat_20h


Alaa Mohasseb
email: alaa.mohasseb@port.ac.uk

Mohamed Bader-El-Den
email: mohamed.bader@port.ac.uk

Mihaela Cocea
email: mihaela.cocea@port.ac.uk

Detecting Question Intention Using a K-Nearest Neighbor Based Approach

Keywords: Natural Language Processing, Question Classification, Machine Learning, Text Mining, Information Retrieval

The usage of question answering systems is increasing daily. People constantly use question answering systems in order to find the right answer for different kinds of information, but the abundance of available data has made the process of obtaining relevant information challenging in terms of processing and analyzing it. Many questions classification techniques have been proposed with the aim of helping in understanding the actual intent of the user's question. In this research, we have categorized different question types through introducing question type syntactical patterns for detecting question intention. In addition, a k-nearest neighbor based approach has been developed for question classification. Experiments show that our approach has a good level of accuracy in identifying different question types.

Introduction

The usage of question answering systems is increasing daily, people make frequent use of question answering systems in order to find the right answer for different kinds of information. The goal of question classification process is to accurately assign labels to questions based on expected answer type.

Many questions classification techniques have been proposed with the aim of helping in understanding the actual intent of the user's question but the abundance of available data has made the process of obtaining relevant information challenging in terms of processing and analyzing it.

Recent studies classified different type of questions by using different machine learning algorithms such as Support Vector Machine (SVM) [START_REF] Metzler | Analysis of statistical question classification for factbased questions[END_REF], [START_REF] Bullington | Open ended question classification using support vector machines[END_REF], [START_REF] Huang | Question classification using head words and their hypernyms[END_REF], [START_REF] Van-Tu | Improving question classification by feature extraction and selection[END_REF]. Other works like [START_REF] Zhang | Question classification using support vector machines[END_REF] and [START_REF] Mishra | Question classification using semantic, syntactic and lexical features[END_REF] used SVM in addition to other machine learning algorithms such as Naive Bayes, Nearest Neighbors and Decision Tree. Moreover, Neural Networks has been used as the machine learning algorithm in other works [START_REF] Sagara | Natural language neural network and its application to question-answering system[END_REF] and [START_REF] Ture | Simple and effective question answering with recurrent neural networks[END_REF].

Furthermore, other methods such as features selection have been applied to obtain an accurate question classifier [START_REF] Huang | Question classification using head words and their hypernyms[END_REF], [START_REF] Zhang | Question classification using support vector machines[END_REF], [START_REF] Li | Question classification using multiple classifiers[END_REF], [START_REF] Yen | A support vector machine-based context-ranking model for question answering[END_REF], [START_REF] Mishra | Question classification using semantic, syntactic and lexical features[END_REF] used bag-of-words, Other works like [START_REF] Yen | A support vector machine-based context-ranking model for question answering[END_REF] used semantic and syntactic features, Moreover, [START_REF] Huang | Question classification using head words and their hypernyms[END_REF] used uni-gram and word shape feature.

In this paper, we propose a method that automatically identifies and classifies users' questions intention using a k-nearest neighbor based approach based on the syntactical pattern of each type of question. In particular, we develop a framework which was adapted from [START_REF] Mohasseb | Web queries classification based on the syntactical patterns of search types[END_REF] and [START_REF] Mohasseb | Domain specific syntax based approach for text classification in machine learning context[END_REF] to test the performance of the proposed method. Experimental results show that our solution leads to accurate identification of different question types.

The rest of the paper is organized as follows: Section 2 outlines previous work on question classification. Section 3 provides a detailed description of the proposed question classification framework. Section 4 reports experimental results. Finally, Section 5 concludes the paper and outlines future work.

Related Work

In this section we outline previous work on questions classification methods and machine learning algorithms.

Recent studies classified different type of questions by using different machine learning algorithms. In [START_REF] Metzler | Analysis of statistical question classification for factbased questions[END_REF] a statistical classifier has been proposed which is based on SVM and uses prior knowledge about correlations between question words and types in order to learn question word specific classifiers. They have stated that under such a statistical framework, any data set, question ontology, or set of features can be used.

Other works like [START_REF] Zhang | Question classification using support vector machines[END_REF] and [START_REF] Mishra | Question classification using semantic, syntactic and lexical features[END_REF] used SVM in addition to other machine learning algorithms. [START_REF] Mishra | Question classification using semantic, syntactic and lexical features[END_REF] proposed an approach for question classification through using machine learning. In this work three different classifiers were used, which are; Nearest Neighbors (NN), Nave Bayes (NB), and SVM using two kinds of features: bag-of-words and bag-of n grams. In order to train the learning algorithm, a set of lexical, syntactic, and semantic features were used, among which are the question headword and hypernym. Similarly, in [START_REF] Zhang | Question classification using support vector machines[END_REF] five machine learning algorithms were used, which are; NN, NB, Decision Tree (DT), Sparse Network of Winnows (SNoW), and SVM using two kinds of features: bag-of-words and bag-of-ngrams.

In addition, authors in [START_REF] Van-Tu | Improving question classification by feature extraction and selection[END_REF] proposed a method of using a feature selection algorithm to determine appropriate features corresponding to different question types. Moreover, they design a new type of features, which is based on question patterns then applied a feature selection algorithm to determine the most appropriate feature set for each type of questions. The proposed approach was tested on the benchmark dataset TREC, using SVM for the classification algorithm.

SVM were also used in [START_REF] Bullington | Open ended question classification using support vector machines[END_REF] for the classification of open-ended questions. They have stated that SVM could be trained to recognize the occurrence of certain keywords or phrases in a question class and then, based on the recurrence of these same keywords, be able to correctly identify a question as belonging to that class.

Another classification has been proposed in [START_REF] Hacioglu | Question classification with support vector machines and error correcting codes[END_REF] using SVM. According to authors in this work enormous amount of time is required to create a rich collection of patterns and keywords for a good coverage of questions in an open-domain application, so they have used support vector machines for question classification. The goal is to replace the regular expression based classifier with a classifier that learns from a set of labeled questions and represented the questions as frequency weighted vectors of salient terms.

Moreover, works like [START_REF] Sagara | Natural language neural network and its application to question-answering system[END_REF] and [START_REF] Ture | Simple and effective question answering with recurrent neural networks[END_REF] used Neural Networks as the machine learning algorithm. [START_REF] Sagara | Natural language neural network and its application to question-answering system[END_REF] proposed a neural network for question answering system, they have stated that the proposed network can process many complicated sentences and can be used as an associative memory and a question-answering system. In addition, the proposed network is composed of three layers and one network which are, Sentence Layer, Knowledge Layer, Deep Case Layer and Dictionary Network. The input sentences are divided into knowledge units and stored in the Knowledge Layer.

The proposed approach in [START_REF] Ture | Simple and effective question answering with recurrent neural networks[END_REF] formulates the task as two machine learning problems which are, detecting the entities in the question, and classifying the question as one of the relation types in the knowledge base. Based on this assumption of the structure, this approach trains two recurrent neural networks and outperform state of the art by significant margins relative improvement.

Furthermore, other studies classified different type of question using different features selection like bag-of-words, semantic and syntactic features, and unigram and word shape feature.

Authors in [START_REF] Huang | Question classification using head words and their hypernyms[END_REF] proposed head word feature which used two approaches to augment semantic features of such head words using WordNet. In addition, other standard features were augmented as well such as wh-word, unigram feature, and word shape feature.

In [START_REF] Yen | A support vector machine-based context-ranking model for question answering[END_REF] a machine learning-based question-answering framework has been proposed, which integrates a question classifier with a simple document/passage retrievers, and proposed context-ranking models. This method provides flexible features to learners, such as word forms, syntactic features, and semantic word features. In addition, The proposed context-ranking model, which is based on the sequential labeling of tasks, this model combines rich features to predict whether the input passage is relevant to the question type.

Finally, works in [START_REF] Li | Question classification using multiple classifiers[END_REF] used machine learning approaches, namely, different classifiers and multiple classifier combination method by using composite statistic and rule classifiers, and by introducing dependency structure from Minipar and linguistic knowledge from Wordnet into question representation, in addition, features like the Dependency Structure, Wordnet Synsets, Bag-of-Word, and Bi-gram were used. Also a number of kernel functions were analysed and the influence of different ways of classifier combination, such as Voting, AdaBoost, ANN and TBL, on the precision of question classification.

Proposed Approach

In this section we introduce a K-nearest neighbor based approach using domain specific syntax information for question classification. The framework mainly relies on Question Type Syntactical Patterns. In the first part of this section question types that have been used are explained in detail; in the second section we introduce our syntactical patterns and in the third part we explain the structure of the proposed approach.

Question Types

Questions could be classified according to their intent into six categories; Factoid, Choice, Causal, Confirmation, Hypothetical and List; each of these types have their own structure and characteristics [START_REF] Mohasseb | Question categorization and classification using grammar based approach[END_REF].

1. Factoid: this type begins with a question word such as What, Where, Why, Who, Whose, When, Which, as well as How, how many, how often, how far, how much, how long, how old and any kind of information is expected as an answer. For example "what is a good blood pressure". 2. Choice: this type of question offers choices in the question. The question contains two (or more) presented options. These options are connected using the conjunction "OR". For example "Which is better iphone or samsung? and why?". 3. Causal: starts with How or Why and requires explanation. For Example, "why do earthquakes occur at destructive plate margins?". 4. Confirmation: this type of question begins with an auxiliary verb or linking verb for example "is Frankfurt a city in Germany?", in addition, the question could start with negative auxiliary verb or linking verb. For example, "wasn't Thomas Edison born in new jersey". The expected answer for this type of question is either Yes or No. 5. Hypothetical: hypothetical questions are asked to have a general idea of a certain situation. It is mainly What would you do if / What would happen if" type of questions. For example "what would you do if someone had a stroke?". 6. List: plural terms are a highly reliable indicator of this question, this type requires a list of entities or facts in answers. For example "What countries are in Africa?".

Question Types Syntactical Patterns

The proposed framework mainly relies on the question types and the characteristics of each type discussed in Section 3.1. Using these characteristics we propose the formulation of syntactical patterns for each question; thus, these give us domain-specific information. Each syntactical pattern is composed of a sequence of term categories. These categories of terms are described below. For the purpose of constructing Question Type Syntactical Patterns, a random set of 3, 000 questions has been selected from Yahoo Non-Factoid Question Dataset1 and TREC 2007 Question Answering Data2 .

The categorization of terms in our solution is mainly based on the seven major word classes in English: Verb (V), Noun (N), Determiner (D), Adjective (Adj), Adverb (Adv), Preposition (P) and Conjunction (Conj). In addition to that, we added a category for question words that contains the six main question words (QW): how, who, when, where, what and which. Some word classes like Nouns consists of subclasses, such as Common Nouns (CN), Proper Nouns (PN), Pronouns (Pron) and Numeral Nouns (NN). Also, the Verb class has subclasses, such as Action Verbs (AV), linking Verbs (LV) and Auxiliary Verbs (AuxV).

Furthermore, the syntactical patterns of each question types have been identified by tagging each term in the question to one of the main word classes mentioned above, and then a further tagging is done to assign each term in the question to one of the domain specific term categories. For example, in the question "who is Frida Kahlo", the terms will be tagged as follows: (a) "who" will be tagged to "QW", (b) "is" is tagged to "LV" and (c) "Frida Kahlo" is tagged to "PN".

Finally, after each term is tagged to one of the word classes, it will be tagged to the domain specific term category; the proposed categories are derived from the following topics; 1. Health: which includes specific terms related to health, medicine, beauty. 2. Sports: includes terms related to game and recreation, sports events, sports. 3. Arts and entertainment: consists of terms related to Entertainment, Celebrities Name, lyrics, Movies, Books, Authors. 4. Food and drinks: includes terms related to foods, drinks, recipe. 5. Animals: consists of terms related to Pets, wild animals. 6. Science and math: which includes specific terms related to Science, math. 7. Technology and internet: consists of terms related to Software and Applications, Site, Website, URL, Database and Servers. 8. Society and culture: includes terms related to Environment, Holidays, Months, history, political, Relationships, Family. 9. News and events: includes terms related to Newspapers, Magazines, Documents, events. 10. Job, Education and Reference: includes terms related to Careers, Institutions, Associations, Clubs, Parties, Foundations and Organizations. 11. Business and Finance: includes terms related to Money, company,products, Economy. 12. Travel and places: which includes specific terms related Geographical Areas, Transportation, Places and Buildings, Countries.

These categories help in the identification of the main topics that are found in most question answering systems. Terms categories have been created for the purpose of identifying the different type of questions. These terms have been constructed after the analysis of different datasets. For example, "QW" will be tagged to "Question Word Who" (QW W ho ); "LV" will not be tagged to any further categories and "PN" will be tagged to "Proper Noun Celebrity" (P N Celebrity ). This step is executed by using a database that contains more than 10,000 terms [START_REF] Mohasseb | Automated identification of web queries using search type patterns[END_REF].

Framework

To investigate the impact of using the domain specific syntax information on the classification performance, the following framework, shown in Figure 1, has been developed. The proposed framework involves automatic identification and classification of user's questions using KNN approach which is based on the patterns described in the previous section. We illustrate the framework by using the following examples of question: "is mercury a metal" and "what are the symptoms of diabetes". After parsing, each term in the question will be tagged to one of the terms category.

The final tagging will be:

Pattern Formulation

In this phase after tagging each term in the question, the pattern is formulated, as illustrated below for the two previously introduced questions.

Experimental Evaluation

To test the accuracy of our proposed approach 1, 160 questions were randomly selected from Yahoo Non-Factoid Question and TREC 2007 Question Answering Data. Their distribution is given in Table 1. K-Nearest Neighbor (KNN) was used as the machine learning algorithm for the automatic classification. The classification accuracy is obtained by using the implementation of the above algorithm from the Weka software [START_REF] Hall | The weka data mining software: an update[END_REF]. The effectiveness of the classification was evaluated based on Precision, Recall and F-Measure, i.e. typical metrics for the evaluation of classifiers, using 10-fold cross-validation and value of K=1. Furthermore, a comparison was done using different values of K to evaluated the classification accuracy when increasing the value of K from K=1 to 10. Table 2 presents the classification performance details (Precision, Recall and F-Measure) of the KNN classifier. Results show that KNN identified correctly (i.e. Recall) 82.8% of the questions when the value of K=1.

KNN classified correctly 93.1% (Recall) of the confirmation questions and 92.2% of the factoid questions, on the other hand, classification accuracy (Recall) for causal, choice, hypothetical and list questions were lower. KNN could correctly classified 32.3% of the causal questions, 25% of the choice questions, 28.6% of the hypothetical questions and 12.9% of the list questions. Figure 2 shows the impact of increasing the value of K, There is a marginal increase in the accuracy between k=1, 2 and 3 in which K=3 has the highest accuracy with 83.7% shown in table 3, however, in terms of Recall for certain question categories such as causal, choice, hypothetical and list K=1 performed better. In addition, when K=4, 5 and 6 the accuracy slightly decreased while when the value of K= 6 and 7 the accuracy increased again but decreased with K=9 and 10.

These results show that KNN deals well with confirmation and factoid questions. In addition, KNN could not distinguish between causal, choice, hypothetical and list type of questions and incorrectly classified most of them as confirmation and factoid questions. As shown in table 1 the question dataset suffers from imbalance between the labels, as the number of the instances that belongs to one more classes is outnumbered by the instances that belong to other classes. This is know by the class imbalance problem and it normally hinders the classifiers ability in correctly predicting the instances that belongs to the minority class [START_REF] Perry | Imbalanced classification using genetically optimized cost sensitive classifiers[END_REF]. However, the proposed algorithms has been able to perform reasonably well over the minority classes as shown in table 2 and table 3. Overall, the results validate that questions Type Syntactical Patterns is an effective method for question classification as well as for the distinction between different question types.

Conclusions

In this paper, we have proposed a method that automatically identifies and classifies different question types by using a domain specific syntax information, which is based on the syntactical pattern of each types of question. In particular, we developed a framework to test the performance of the proposed method and used a machine learning algorithm (KNN) to build a model for the identification of user's question type. The experiment shows that our solution led to a good performance in classifying questions.

As future work, we aim at examining and analyzing more questions from different data-sets and extending the analysis of the different types of questions. As mention earlier, the questions dataset suffers form imbalance between the label, we aim to investigate ensemble learning and other methods to deal with the class imbalance problem [START_REF] Bader-El-Den | Hierarchical classification for dealing with the class imbalance problem[END_REF], [START_REF] Bader-El-Den | Self-adaptive heterogeneous random forest[END_REF]. We are also planning to test other machine learning algorithms to classify the questions. In addition, we will test our dataset using other classification frameworks in order to evaluate our method and be able compare it with different approaches.

Fig. 1 .

 1 Fig. 1. Question Classification Framework

Question 1

 1 Pattern: LV + P N Science + D + CN Singular Question 2 Pattern: QW W hat + LV + D + CN P lural + P + CN Health 3. Question Classification In this step the system first attempts to match the question with the most appropriate Question Type Pattern to determine the Question type. For the given examples. Question 1 Type: Confirmation Question 2 Type: List Second, the system will match the question with the most appropriate Domain Category to determine the Question Domain Specific. This step is done by matching each words in the question with one of the Domain Specific Terms Categories 3.2 and the domain is selected based on the number of words related to a domain in each question. For the given examples. Question 1 Domain Category: Science and math Question 2 Domain Category: Health This will result in the final classification of each question in which Question 1 will be classified to Conf irmation Scienceandmath and Question 2 will be classified to List Health

  Question 1 Terms Tagging: is=LV , mercury= P N Science , a= D, metal= CN Singular , where P N Science is Proper Noun Science and CN Singular is Common Noun Singular Question 2 Terms Tagging: what= QW W hat , are=LV , the= D, symptoms=CN P lural , of= P , diabetes=CN Health , where CN P lural is Common Noun Plural and CN Health Common Noun Health.

Table 1 .

 1 Data distribution

	Question type Total
	Causal	31
	Choice	12
	Confirmation	321
	Factoid	688
	Hypothetical	7
	List	101

Table 2 .

 2 KNN classifier performance with value of K=1

	Question Types Precision Recall F-Measure
	Causal	0.714	0.323	0.444
	Choice	0.333	0.250	0.286
	Confirmation	0.849	0.931	0.889
	Factoid	0.853	0.922	0.886
	Hypothetical	0.667	0.286	0.400
	List	0.333	0.129	0.186
	Overall	0.797	0.828	0.805

Table 3 .

 3 KNN classifier performance with value of K=3

	Question Types Precision Recall F-Measure
	Causal	0.714	0.161	0.263
	Choice	1.000	0.083	0.154
	Confirmation	0.870	0.938	0.903
	Factoid	0.831	0.956	0.889
	Hypothetical	1.000	0.143	0.250
	List	0.385	0.050	0.088
	Overall	0.802	0.837	0.795

https://ciir.cs.umass.edu/downloads/nfL6/

http://trec.nist.gov/data/qa/t2007_qadata.html


Athanasios Alexopoulos

Andreas Kanavos
email: kanavos@ceid.upatras.gr

Konstantinos Giotopoulos

Alaa Mohasseb
email: alaa.mohasseb@port.ac.uk

Mohamed Bader-El-Den
email: mohamed.bader@port.ac.uk

Athanasios Tsakalidis

Incremental Learning for Large Scale Classification Systems

Keywords: Apache Spark, Apache MLlib, Big Data, Classification, Computing Performance, DataFrame, Spark SQL

One of the main characteristics of our time is the growth of the data volumes. We collect data literally from everywhere; smart phones, smart devices, social media and the health care system, which defines a small portion of the sources of the big data. The big data growth poses two main difficulties, storing and processing them. For the former, there are certain new technologies that enable us to store large amounts of data in a fast and reliable way. For the latter, new application frameworks have been developed. In this paper, we perform classification analysis using Apache Spark in one real dataset. The classification algorithms that we have used are multiclass, and we are going to examine the effect of the dataset size and input features on the classification results.

Introduction

Nowadays, one of the most frequently used term is Big Data, which is utilized so as to define the increase of the data volumes, as modern datasets are ever expanding both in size and complexity. But with the rapid growth of the stored data, difficulties have appeared [START_REF] Meng | Mllib: Machine learning in apache spark[END_REF]. The first obvious problem deals with where to store these huge amounts of data, and then how to process them and retrieve the information they hide. Researchers soon found out that even storing data can easily outperform the capabilities of a single computer, no matter how sophisticated that could be. So they decided not to opt for the popular scale up solution but came up with the idea of scaling out. Instead of continuously updating a single computer, spreading the data or the computation to a cluster of computers interconnected via network, would prove handy.

In order to solve the data storage problem, the idea of NoSQL, which stands for Not Only SQL, arose. Those database systems were originally supported and used by major Internet companies (Google, Amazon, Facebook), since they had difficulties in dealing with huge amounts of data with the conventional RDBMS solutions [START_REF] Moniruzzaman | Nosql database: New era of databases for big data analytics -classification, characteristics and comparison[END_REF]. Somewhat similar, researchers cope the need of processing those data, with the use of cluster programming models 1 . In order to unify, and thus simplify, the mode to process big data, Apache Spark [START_REF] Zaharia | Apache spark: a unified engine for big data processing[END_REF] project came up. This provides an efficient and scalable analysis of big data accordingly and can support a wide range of workloads as well.

Now that we have discussed how to store and process with big data, one query remains; what the origin of such data is. There is not a single answer to this question since we can collect data from everywhere and have ample forms. Ranging from a post or a tweet in social media, the graph that shows relations between persons who use the specific aspect of the media or share a trend, reviews of products, data collected from smart devices to even more sophisticated data like health records of patients.

In this paper we aim to perform classification analysis in a real dataset, using Apache Spark's MLlib, a machine learning library optimized for distributed systems. We investigate several classification models (Decision Trees, Random Forest, Logistic Regression) and the way they evolve along the size of the dataset.

The remainder of the paper is structured as follows: Section 2 presents the related work and cloud computing methodologies, while Section 3 presents the machine learning (e.g. classification) algorithms used in our proposed system. Section 4 presents the steps of training as well as the classification type, e.g. ternary. Moreover, Section 5 presents the evaluation experiments conducted and the results gathered. Ultimately, Section 6 presents conclusions and draws directions for future work.

Related Work

Data mining is an idea that came into existence during the 1990's and its main advantage is that it is suitable for discovering hidden patterns and other useful information from a known dataset. It provides the researcher with the tools to discover correlations that exist and where not obvious at first, and thus makes the task of Decision Making easier [START_REF] Hand | Principles of Data Mining[END_REF]. As Fayyad et al stated [START_REF] Fayyad | From data mining to knowledge discovery in databases[END_REF], the knowledge discovery has a wide range of application areas, with marketing, finance (especially investment) and fraud detection defining a portion of them. The process of knowledge discovery is structured in several stages, the first of which is data selection. The preprocessing of data is bound to follow and the transformation of it into the appropriate format with the final stage of Data Mining, in which a suitable algorithm (or technique in general) is applied in order to extract the hidden information, in a form appropriate for future use [START_REF] Witten | Data Mining: Practical machine learning tools and techniques[END_REF].

There are several tools that encapsulate those principles, with Weka [START_REF] Hall | The WEKA data mining software: an update[END_REF] being one of the most widely used and accepted in business and academia. Although its popularity, it has the limitation of the capabilities of a single computer, in terms of memory and computational power. Using the notion of distributed computation, researchers came up with a distributed framework or Weka, the Distributed Weka Spark [START_REF] Koliopoulos | A parallel distributed weka framework for big data mining using spark[END_REF], which is implemented on top of Apache Spark.

In addition, frameworks like Hadoop, Apache Spark, Apache Storm as well as distributed data storages like HDFS and HBase are increasingly becoming popular, as they are engineered in a way that makes the process of very large amounts of data almost effortless. Such systems are gaining much attention and consecutively libraries (like Apache Spark's MLlib), which make the use of Machine Learning techniques possible in the cloud, are introduced [START_REF] Fovet | Machine learning and data mining with apache spark[END_REF].

One of the first widely used framework that supports data processing in a distributed manner across clusters of computers is Apache Hadoop [START_REF] Shvachko | The hadoop distributed file system[END_REF], which makes use of the MapReduce paradigm. There were several efforts to use Hadoop for data mining problems, in [START_REF] Yang | An efficient data mining framework on hadoop using java persistence API[END_REF] researchers implemented KD-tree algorithm on Hadoop, in [START_REF] Zhao | Parallel K -means clustering based on mapreduce[END_REF] they developed a fast parallel k-means clustering algorithm based on MapReduce whereas in [START_REF] Kang | Big graph mining: algorithms and discoveries[END_REF] the researchers developed Pegasus, a big graph mining tool built on MapReduce.

In social media analysis, a cloud-based architecture was proposed in [START_REF] Baltas | An apache spark implementation for sentiment analysis on twitter data[END_REF] where authors aim at creating a sentiment analysis tool for Twitter data based on Apache Spark cloud framework. There, tweets are classified using supervised learning techniques and the classification algorithms are used for ternary classification. In addition, in [START_REF] Sioutas | Survey of machine learning algorithms on spark over dht-based structures[END_REF], a survey of machine learning algorithms implemented on Apache Spark over DHT-structures (Distributed Hash Table) is presented; authors experimented across a POS dataset that had been stored in Cassandra with some of the most influential algorithms that have been widely used in the machine learning community. Finally, a novel distributed framework implemented in Hadoop as well as in Spark for exploiting the hashtags and emoticons inside a tweet is introduced in [START_REF] Kanavos | Large scale implementations for twitter sentiment classification[END_REF]. Moreover, Bloom filters are also utilized in order to increase the performance of the proposed algorithm.

Distributed Computing

In this section we will briefly discuss the tools we used to perform the analysis. As above mentioned, the classification analysis will be performed with the use of Apache Spark. For clarity reasons though, a brief mentioning of Spark's predecessor, Hadoop MapReduce, is vital and of essential importance.

MapReduce Model

MapReduce [START_REF] Dean | Mapreduce: simplified data processing on large clusters[END_REF] is a programming technique used in distributed systems. Its main advantage being that it is easily scalable over a number of computing nodes. It consists of two main procedures, namely Map() and Reduce(), which is performed after the Map() job. A MapReduce program is executed in three stages the map, shuffle and reduce. In the map stage, the input data is split into a number of chunks and each chunk is sent to a mapper to execute the Map() algorithm. Its result is a set of < key, value > pairs which, during the shuffle stage are grouped by the key, and each key is fed into the corresponding reducer in which they execute the Reduce() function.

Apache Spark

Apache Spark2 [START_REF] Zaharia | Apache spark: a unified engine for big data processing[END_REF] was founded in 2009 at the University of California, Berkley. Although it shares the same principles as Hadoop, its philosophy differs. It uses the abstraction of "Resilient Distributed Dataset" (RDD's), which represent a fault-tolerant correlation of elements, distributed across many compute nodes that can be manipulated in parallel. Using them, a wide range of tasks, including SQL, streaming, machine learning and graph processing, in a unified manner, can be captured. Its main advantage over MapReduce paradigm is that we don't have to flush the intermediate data to the disk, just to read them at the reduce stage, since it can perform iterative computations in memory, which can have a positive impact on the performance [START_REF] Shi | Clash of the titans: Mapreduce vs. spark for large scale data analytics[END_REF].

MLlib

Spark ships with MLlib3 , a distributed machine learning library [START_REF] Meng | Mllib: Machine learning in apache spark[END_REF]. It consists of implementations of common machine learning algorithms, that can be used with scalable environments, for several types of analysis including classification, regression, clustering and collaborative filtering. MLlib also includes Java, Scala and Python APIs.

Classification Models

In this section we will discuss the classification methods that we utilized. We examined multiclass classification, while the focus of the research concerned the way that the size of the dataset affected the computation time needed to perform each as well as the way the metrics evolved. As a quick reminder, classification is an instance of supervised learning, and its purpose is, based on previous knowledge (set of observations and the output class of each observation) to identify in which output class a new observation belongs.

Decision Trees

Decision Trees is a classification algorithm, which can be represented in a tree form [START_REF] Kotsiantis | Supervised machine learning: A review of classification techniques[END_REF]. The nodes of the tree represent the feature of the dataset, and depending on each feature's value we navigate through the tree structure until we reach a leaf, which represents the output classes. The root of the tree is the feature that best divides the training data, by minimizing a loss function. The procedure in following is recursively executed to each partition of the data to form the subtrees, until the training data is divided into subsets of the same class.

Random Forest

Random Forest is a generalization of a Decision Tree classifier [START_REF] Breiman | Random forests[END_REF]. It consists of a set of Decision Tree classifiers, so in order to classify for a new input we insert it in each tree of the forest, ending in the "vote" of an output class. The class that will receive the most votes, is the output class the Random Forest will return. To construct each tree, we sample with replacement a number of cases from the original data, equal to the number of trees our forest has, and then we use a subset of the features of that specific selection in order for the size level of tree to be increased.

Logistic Regression

Logistic Regression is a regression model that can be used when the dependent value (output class) is categorical. It uses a logistic function to express the relationship between the dependent value and the independent (features of each class). Apache Spark supports both binomial and multinomial logistic regression, in which case, assuming we have K output classes, one class is chosen as pivot, K -1 models are created and the final result is the class with the largest probability among the K -1 models.

One-Vs-Rest

The One-Vs-Rest (or One-Vs-All) classifier, uses a base classifier, that can efficiently perform binary classification and trains a single classifier per output class (considering the portion of the data that belong to that class as positive and the rest as negative). In order to determine the label of the output class, it uses the classifier with the highest confidence score.

Multilayer Perceptron

A Multilayer Perceptron is an artificial neuron network model used to map a set of input data (input classes) to a set of output classes. It consists of layers of nodes fully connected, with a certain weight, to the ones of the next level. Using back-propagation, a technique that changes the connection weights of each node to the ones of the next layer in order to minimize the output error, we can train the Multilayer Perceptron for a given dataset.

Implementation

Our approach follows the proposal of [START_REF] Fayyad | From data mining to knowledge discovery in databases[END_REF], as presented in Section 2. Firstly, we need to discuss the framework on which the computation took place.

The overall architecture of the proposed system is depicted in Figure 1 taking into account the corresponding modules of our approach. Initially, a preprocessing step, as shown in the following subsection, is utilized and afterwards the classifiers for prediction scope are employed.

The creators of Apache Spark have also founded Databricks that supplies researchers with a web based platform in which they can store and analyse their data with Spark. It offers researchers a mini cluster with 6 Gb of RAM for their The realistic dataset is the PAMAP dataset [START_REF] Reiss | Introducing a new benchmarked dataset for activity monitoring[END_REF], also provided by UCL 4 . It contains 2.872.533 rows of data from 8 subjects and 12 different activities. The devices that were used to obtain those measurements were inertial measurement units in the ankle, chest and hand of the subject as well as a heart rate monitor. Since the dataset is realistic, some of the data were missing. There were two main reasons for that, data dropping due to using wireless sensors and problematic hardware setup. Also, there was another issue with the equipment used; the sampling frequency of the heart rate monitor was lower than the inertial measurement units. All those missing values were indicated as N aN .

Our first attempt to clean up the data was by withdrawing all the rows that contained at least one N aN value. As a result we had to withdraw 2.610.265 rows of data. To overcome this obstacle, we replaced all the intermediate N aN values of the heart rate monitor with the last known value. After that replacement, the dataset contained only 13.141 rows indicated as N aN , which we ignored in the final analysis.

In order to perform the analyses, we split the original data into segments of 5.000, 10.000, 25.000, 50.000, 75.000, 100.000, 200.000 rows. But, in order for the results to be comparable in each case, we had to keep the ratio of the number of instances that belong to a specific class to the total number of instances, stable. So, we calculated the fraction of each class in the original dataset, and at the examined cases we assured that each class had the same fraction of observations.

Analysis Cases

For each segment of both datasets we performed a series of classification analyses.

-Decision Trees, optimized regarding the depth of the tree, with values ranging between 2 and 30. -Random Forest, optimized regarding the number of trees in the forest, with values ranging between 1 and 60, with a step of two and maximum tree height equal to ten. -Logistic Regression, multinomial.

-Multilayer Perceptron with two, three and four levels of hidden layers. For the case of two levels, the numbers of nodes were 15-14, 20-15, 25-20 respectively, whereas for case of three levels the numbers of nodes were 25-20-15, 35-25-20, 40-30-20 and for the case of four levels 30-25-20-15, 35-30-25-20 and 40-35-30-25. Moreover, the number of epochs was set equal to 1000.

Evaluation

The results of our work are presented in the following Tables 1 to 5. Recall, Precision and F-Measure are used as the evaluation metrics of the different algorithms. Also, the time that was needed to train each classifier is presented, as well as information about the best settings.

In PAMAP Dataset, a series of classification analyses are conducted. In Table 1, the results of the Decision Tree classifier are presented. As the dataset increased in size, we kept getting higher scores, as expected. Worth noting is the fact that in this case, the height of the tree had the highest possible value in all cases, but its complexity increased since nodes kept increasing in number. Also, the computational time needed, as expected, kept rising with the size of the dataset, but not in a linear manner; when the size of the dataset increased by 40times (from 5.000 rows to 200.000), the time needed nearly doubled. Next, the Random Forest classifier (Table 2) is discussed. Again as the dataset size grew bigger, so did the metrics' scores. The only exception is the case of the 10.000 rows in which we had the best observed scores, but this might be caused by the split of the dataset. Overall, this classifier produced lower metrics compared with the Decision Tree classifier; the best F-Measure score is 76.7% while Decision Tree's was 90.0%. Furthermore, the training time of this classifier is overall higher than the Decision Tree one. When the size of the dataset increased 40 times, it needed 4 times more for the classifier to be trained. The next classifier is the Multinomial Logistic Regression (Table 3), where there was no parameter optimization performed. With few exceptions, we can argue that as the dataset size increases, so do the metrics, although some fluctuation exists. The metrics' score is higher than the Random Forest classifier, but did not perform as well as Decision Trees. As for the scaling of time regarding the size of the dataset, for 20 times bigger dataset, from 10.000 to 200.000 rows, the classifier's training time increased by 7.5 times. The last classifier examined is One-Vs-All (Table 4). As the base classifier, we have chosen Logistic Regression. Its performance was rather poor, both in time needed to train and in the metrics' score. Examining again F-Measure, it achieved 79.5% while the Logistic Regression achieved 80.8%. Considering time scaling, with regards to the dataset size, for 20 times bigger dataset, the classifier's training time has been increased by 3.23 times, and in every case the training was 7 to 11 times higher. Overall, it can be argued that the best choice in PAMAP dataset is the Decision Tree classifier, since it performed better in the metric scores from all other classifiers and its training time scaled well regarding the dataset size.

Conclusions and Future Work

In our work we delved into the performance of various classification algorithms in a distributed environment. Each classification method had strengths and limitations, depending on the dataset. The Decision Tree classifier outperformed all the other classification methods, which was a pleasant surprise.

As for future work, more datasets with the same classification methods to establish a better understanding, will be checked. Of course, the cluster on which we run the classification methods is another key aspect of the cloud computing in general, so checking the same or similar algorithms and dataset on different clusters may prove fundamental and reveal well-hidden secrets.

Fig. 1 .

 1 Fig. 1. Proposed System Architecture

Table 1 .

 1 Decision Tree

						Number of
	Dataset Size Precision Recall F-Measure Time Depth	Nodes
	5.000	0.862	0.859	0.860	0:15:25 17	731
	10.000	0.923	0.923	0.923	0:17:21 16	1007
	25.000	0.954	0.954	0.954	0:21:09 18	1625
	50.000	0.966	0.966	0.966	0:24:44 25	2441
	75.000	0.968	0.968	0.968	0:39:55 27	3363
	100.000	0.981	0.981	0.981	0:41:43 27	3139
	200.000	0.989	0.989	0.989	0:57:26 30	4083

Table 2 .

 2 Random Forest

						Number of	Number of
	Dataset Size Precision Recall F-Measure Time	Trees	Nodes
	5.000	0.907	0.907	0.907	0:29:27	60	28908
	10.000	0.927	0.925	0.926	0:36:01	50	28522
	25.000	0.945	0.945	0.945	0:49:12	44	31618
	50.000	0.953	0.953	0.953	1:04:11	42	35836
	75.000	0.950	0.949	0.949	2:01:05	44	38600
	100.000	0.958	0.958	0.958	2:28:11	48	45134
	200.000	0.951	0.950	0.950	3:42:04	56	56274

Table 3 .

 3 Logistic Regression

	Dataset Size Precision Recall F-Measure Time
	5.000	0.858	0.861	0.860	0:00:26
	10.000	0.861	0.862	0.861	0:00:26
	25.000	0.814	0.818	0.816	0:00:32
	50.000	0.803	0.807	0.804	0:00:42
	75.000	0.802	0.806	0.803	0:01:00
	100.000	0.825	0.828	0.826	0:01:14
	200.000	0.800	0.804	0.802	0:01:56

Table 4 .

 4 One-vs-All

	Dataset Size Precision Recall F-Measure Time
	5.000	0.755	0.755	0.749	0:02:03
	10.000	0.789	0.791	0.789	0:02:15
	25.000	0.788	0.792	0.789	0:02:29
	50.000	0.781	0.784	0.781	0:02:56
	75.000	0.770	0.773	0.771	0:04:56
	100.000	0.796	0.798	0.796	0:07:17
	200.000	0.774	0.778	0.775	0:11:16

Table 5 .

 5 Multilayer Perceptron

	Dataset Size Precision Recall F-Measure Time	Best Layer
	5.000	0.639	0.646	0.641	0:09:20 [31, 35, 25, 20, 12]
	10.000	0.691	0.708	0.697	0:14:27 [31, 35, 25, 20, 12]
	25.000	0.720	0.719	0.719	0:32:22 [31, 40, 30, 20, 12]
	50.000	0.707	0.732	0.721	1:01:55 [31, 40, 30, 20, 12]
	75.000	0.724	0.738	0.727	1:30:39 [31, 40, 30, 20, 12]
	100.000	0.755	0.776	0.759	1:55:59 [31, 35, 25, 20, 12]
	200.000	0.775	0.778	0.775	3:59:41 [31, 35, 25, 20, 12]

https://en.wikipedia.org/wiki/NoSQL

http://spark.apache.org/

http://spark.apache.org/mllib/

http://archive.ics.uci.edu/ml/datasets/PAMAP2+Physical+Activity+ Monitoring