Ilias Kalamaras
email: i.kalamaras11@imperial.ac.uk

Stavros Papadopoulos
email: s.papadopoulos11@imperial.ac.uk

Anastasios Drosou
email: drosou@iti.gr

Dimitrios Tzovaras
email: tzovaras@iti.gr

MoVA: A Visual Analytics tool providing insight in the Big Mobile Network Data

Keywords: Mobile network security, big data, visual analytics, information visualization, anomaly detection, hypothesis formulation and testing

Mobile networks have numerous exploitable vulnerabilities that enable malicious individuals to launch Denial of Service (DoS) attacks and affect network security and performance. The efficient detection and attribution of these anomalies are of major importance to the mobile network operators, especially since there is a vast amount of information collected, which renders the problem as a Big Data problem. Previous approaches focus on either anomaly detection methods, or visualization methods separately. In addition, they utilize solely either the signaling or the Call Detail Record (CDR) activity in the network. This paper presents MoVA (Mobile network Visual Analytics), a visual analytics tool for the detection and attribution of anomalies in mobile cellular networks which combines anomaly detection and visualization, and is applied on both signaling and CDR activity in the network. In order to address the large volume of the data, the proposed application starts with an aggregated overview of the whole network and allows the operator to gradually focus on smaller sets of data, using different levels of abstraction. The proposed visualization methods are able to differentiate between different user behaviors, and enable the analyst to have an insight in the mobile network operation and easily spot the anomalous mobile devices. Hypothesis formulation and validation methods are also provided, in order to enable the analyst to formulate network securityrelated hypotheses, and validate or reject them based on the results of the analysis.

Introduction

This work has been partially supported by the European Commission through project FP7-ICT-317888-NEMESYS funded by the 7th framework program. The With a steadily increasing amount of user devices, the problem of mobile network security, i.e. monitoring a mobile phone network and identifying abnormal and malicious behavior, is nowadays becoming even more challenging. The number of connected mobile devices is expected to increase even more in the next years, including diverse types of data, such as those originating from Internet of Things (IoT) devices. The vast number of mobile phone subscribers, communicating every day, results in a huge amount of signaling and billing records, containing multiple and diverse types of information. This constant flow of information from multiple sources renders the problem of mobile network security as a Big Data problem, posing the challenge of how to reduce the amount of information and focus on the useful aspects.

Visual analytics techniques can significantly assist in this direction. Visual analytics tools can assist the mobile network operator to have an overview of various aspects of the whole network, while allowing her/him to explore and focus on gradually smaller subsets of the data, until the desired information is reached and a decision is made. This paper presents MoVA (Mobile network Visual Analytics), a visual analytics application for the visualization and exploration of mobile phone networks. The application has been developed for the purposes of the NEMESYS project [START_REF]NEMESYS project[END_REF], and is designed with a focus to detecting anomalies in the mobile traffic, such as Denial of Service (DoS) attacks. The application consists of various tools, which allow the operator to have an overview of the network, to view similarities in the behavior of the mobile phone users and to formulate and validate various hypotheses regarding the mobile network data. To accomplish these tasks, the tools utilize graph-based visualization techniques as well as anomaly detection algorithms, based on signaling and Call Detail Record (CDR) data of the network. The complete application allows the mobile network operator to explore the mobile network and gradually focus on specific subsets of the initial large amounts of data, in order to finally detect anomalous behavior and trace its origins. The proposed application is designed in order to handle Big Data in a time-efficient manner, starting with a large-scale overview of the network and allowing the operator to gradually focus on the desired information and acquiring useful insights. The proposed application addresses modern challenges regarding the research about Big Data, as well as practical issues posed by telecommunication and mobile network security companies.

The rest of the paper is organized as follows. Related literature is presented in Section 2. The proposed visual analytics application and the tools comprising it are described in Section 3. Experimental results for the evaluation of the application are presented in Section 4, while Section 5 concludes the paper. opinions expressed in this paper are those of the authors and do not necessarily reflect the views of the European Commission.

Related work

The mobile network is comprised of millions of components and mobile devices, which exchange information at a large rate. The data collected from mobile networks are comprised of two types of information: 1) Signaling activity, and 2) CDR activity. The signaling activity is the control plane, and represents all the messages exchanged in order to activate/deactivate network services and establish connections. The CDR data represent the data plane, and contain information such as [START_REF] Petersen | The telecommunications illustrated dictionary[END_REF]: source, destination, time, duration, call type (e.g. voice, SMS, MMS etc). These two types of information contain the necessary information in order to detect anomalies in the network traffic, without using communication content and sacrificing user privacy. Signaling and CDR data are collected for each mobile subscriber, and for an extended period of time. This section presents the methods that are used for anomaly detection using these types of information.

With respect to anomaly detection using signaling data, most of the proposed approaches utilize this type of information in a aggregated format. Specifically, similarly to this paper, each signalling message is daggered over specific time periods, where the value in each time period represents the number of signals sent to the network. Using this type of information, Lee et al. [START_REF] Lee | On the detection of signaling DoS attacks on 3G wireless networks[END_REF] [4] utilized a cumulative sum (CUSUM) method for the identification of signaling attacks that the traditional detection systems cannot detect. The authors designed their method in such a way, so that it is difficult for the attackers to evade detection. They evaluated their approach on a novel DoS attack that affects the RNC and the Node-B in 3G and potentially WiMax networks. Alconzo et al. [START_REF] D'alconzo | A distribution-based approach to anomaly detection and application to 3G mobile traffic[END_REF] proposed the use of statistical techniques applied on time-series of unidirectional feature distributions. Coluccia et al. [START_REF] Coluccia | Distribution-based anomaly detection in network traffic[END_REF] proposed two distribution-based anomaly detection methods and provided enhancements on the method introduced in [START_REF] D'alconzo | A distribution-based approach to anomaly detection and application to 3G mobile traffic[END_REF].

With respect to anomaly detection using CDR data, Yan et al. [START_REF] Yan | Sms-watchdog: Profiling social behaviors of sms users for anomaly detection[END_REF] proposed SMS-Watchdog, a scheme that utilizes SMS for anomaly detection. Multiple information theoretic anomaly metrics have been used, in order to identify large deviations from the normal past behaviors. Kim et al. [START_REF] Kim | A detection mechanism for SMS flooding attacks in cellular networks[END_REF] utilized multiple statistical metrics for the identification of the mobile devices participating in an SMSflooding attack, and achieved better results than SMS-Watchdog. Murynets et al. [START_REF] Murynets | Anomaly detection in cellular Machine-to-Machine communications[END_REF] proposed two algorithms for the identification of anomalous SMS activities on mobile networks at different levels of abstraction, including aggregate, cluster, and individual device levels.

Apart from the analytical methods for anomaly detection in mobile networks, visualization-based methods have also been proposed in the literature. Eagle et al. [START_REF] Eagle | Reality mining: sensing complex social systems[END_REF] utilized entropy measures to characterize the CDR activity of each user. Afterwards, the authors utilized 2-dimensional plots to represent the change of entropy over time. Ye et al. [START_REF] Ye | Cell phone mini challenge award: Social network accuracyexploring temporal communication in mobile call graphs[END_REF] proposed a graph-based approach, for the visualization of CDR data. The vertices of the graph represent users and the edges call communication events. Shen et al. [START_REF] Shen | Mobivis: A visualization system for exploring mobile data[END_REF] proposed mobivis, another graph-based visualization approach for the analysis of CDR data. The vertices of the graph represent ontologies (e.g. users, locations), and the edges time dependent con- nections between ontologies. The authors also proposed circular histograms as well as 2-dimensional histograms for visualizing the activity of each user/group over time.

There are also commercial toolkits that offer visual analytics solutions and can be used by mobile network providers. SAS VA [START_REF]SAS Visual Analytics[END_REF] and Tableau [START_REF]Tableau Analytics[END_REF] are widely-used visual analytics applications which are able to visualize any kind of multi-dimensional data, using a variety of visualization techniques. However, being generic in nature, they are not targeted to mobile network data, losing much of their effectiveness in related scenarios. Certifuge Systems, [START_REF]Certifuge Systems[END_REF], another visual analytics application, is also generic in nature, but can also handle IP network data. However, it is still not focused on mobile network data.

Existing solutions that target mobile network security include tools that perform anomaly detection, but with no support from visualizations. Motive security guardian [START_REF]Alcatel Security-guardian[END_REF] was proposed by Alcatel for monitoring and analysis of signalingrelated data. This method can help network providers in pinpointing malware infections accurately and take action in real time. Thanks to the power of Motive security guardian network traffic analysis, the network operators can gain actionable insights into security threats and how to minimize their impact on the network and their subscribers. NSN mobile guard [START_REF]NSN mobile guard[END_REF] is a complete solution of network security proposed by Nokia. It provides anomaly detection methods that utilize signaling data, as well as an antivirus suite installed on mobile devices and protects subscribers against fraud. NSN mobile guard allows the network operators to see the security status of the devices on the network, while also enabling them to provide proactive support for infected subscribers. However, as already mentioned, both of the above tools do not provide visual analytics techniques, which would allow the operator to easily explore the available data. The proposed application aims at filling this gap in the existing applications, by utilizing visual analytics techniques, in conjunction with anomaly detection methods, in order to provide solutions for mobile network security.

Description of the proposed methods

As already mentioned, most of the previous work for the analysis of data collected in the mobile network, for the purposes of security, are very limited, and do not address Big Data. In addition, the majority of the aforementioned approaches focus only on signaling or CDR data, while none of these approaches combines visualization and anomaly detection into a common framework. The hereby proposed MoVA application aims at handling these limitations.

The proposed approach begins with an overview of the whole mobile network and provides different levels of abstraction, allowing the operator to gradually focus on smaller sets of data, which can be further explored and allow the detection and attribution of anomalies. A schematic representations of the proposed data pipeline is presented in Figure 1. Signaling and CDR data are collected from the mobile network. The signaling data are first visualized in an aggregated manner, which results in the selection of time periods which contain an anomaly. The CDR data of the selected time periods are afterwards used to extract multiple behavioral characteristics of the users and visualize their similarities. The network analyst can finally formulate attack related hypotheses, and validate or reject them using the results of the analysis and the visualization methods. Screenshots of MoVA are illustrated in Figure 2.

Visualization of the signaling activity in the network

The visual analytics methods presented in this section deal with the visualization of the signaling activity and the anomaly status over a long period of time. These methods provide an overview of the current and past network status to the analyst, and enable the selection of appropriate subsets of data which will be analyzed in depth with the rest of the methods integrated in the system.

As noted earlier, the signaling data are Big Data, collected from millions of devices and over an extended period of time. In order to address this issue, the total signaling activity in the network is aggregated over short periods (e.g. per hour). This aggregation results in multiple time series, one for each type of signal in the network, while each entry of the time series represents the total amount of signals sent for the specific time period. This is represented as a matrix S = {s ij |i ∈ 1 . . . K, j ∈ 1 . . . N }, where N is the total number of time periods, K is the total number of signals in the network, and s ij is the total number of signals belonging to the i th category, and sent to the network at the j th time period.

Thereafter, anomaly detection methods are applied on matrix S, in order to provide an overview of the anomaly status over the different time periods. Specifically, the Local Outlier Factor (LOF) [START_REF] Breunig | LOF: identifying densitybased local outliers[END_REF] is utilized on the multidimensional space defined by S, in which the position of each point are defined as s j = [s 1j , s 2j , ..., s Kj ]. The result of the LOF analysis is an anomaly score for each time period j ∈ {1, 2, .., N }.

The signaling activity represented by matrix S, and the LOF scores are subsequently visualized using stacked graph representations, as shown in Figure 3(a). The lower part of this figure visualizes the normalized volume of the signaling activity for each signal, over the different time periods. The upper part shows the LOF scores for each time period. An example of anomaly is manually highlighted in this figure, concerning a sudden increase of the Location Update Total (LU T) signal.

Figure 3(b) illustrates the visualization method that is utilized for presenting the aggregated signaling activity of a subset of users in the network. This subeset represents the top k mobile devices with the highest activity. The specific time period is selected using the stacked graphs representation shown in Figure 3(a). As shown in Figure 3(b), the components of the network and the mobile devices connected to them are positioned in different layers, namely, 1) UE (User Equipment): the actual mobile devices, 2) NodeB: the NodeBs that provides the mobile devices with wireless connectivity, 3) RNC: the RNCs (Radio Network Controller)which are responsible for controlling the NodeBs that are connected to them, and 4) Core Network: the componets of the core network.

The visualization of Figure 3(b) shows the distribution of the signaling messages to the different network components. This allows the mobile network operator to identify the most active users, as well as their activity distribution to the network. More details on the method presented in this section for the visualization of the signaling activity in the network can be found in [START_REF] Papadopoulos | Visual Analytics for Enhancing Supervised Attack Attribution in Mobile Networks[END_REF].

Visualization of common user behaviors

As a second stage of analysis, after focusing on a specific time period, the Multi-objective behavior visualization tool is provided to the operator, which visualizes groups of users with common behavior, utilizing multiple channels of CDR information. During mobile phone usage, multiple and diverse types of information are stored, such as the time of a call, the identity of the user starting the call and the recipient etc. Each of these attributes can be used to extract behavioral descriptors from the users, encoding certain characteristics of their behavior.

The descriptors are extracted by considering each attribute separately and constructing histograms of their values for each user, for the time period in consideration. These histograms are indicators of the behaviors of users so that a user with normal behavior has a histogram with a much different form than a user with abnormal behavior. Two types of descriptors are used, namely the Time Histogram Descriptor (THD), which is formed from the hours in a day that a user communicates with other users, and the Recipient Histogram Descriptor (RHD), which is formed from the IDs of the recipients of the user's communications.

The multiple descriptors extracted from a user can be used to visualize the similarities between users, with respect to the multiple recorded attributes. Although each descriptor could be used separately to visualize a specific behavioral aspect, considering all descriptors simultaneously allows the operator to view multiple aspects at once, and to have a more complete insight into the data, such that could not be gained by using each descriptor separately. For the purposes of the mobile network behavior visualization, the multi-objective visualization of [START_REF] Kalamaras | Multi-Objective Optimization for Multimodal Visualization[END_REF] has been used. Each descriptor is used to calculate unimodal distances between users. Subsequently, a distance graph is formed, where each user is considered as a vertex and there are edges between each pair of users, weighted by the distance between them. The Minimum Spanning Tree of the graph is extracted and is used to define a force-directed-based objective function [START_REF] Kalamaras | Multi-Objective Optimization for Multimodal Visualization[END_REF]. The minimization of this objective leads to a placement of the vertices on the two-dimensional plane, such that the connected vertices are kept close to each other.

Although large graph analysis is generally a computationally expensive task, the proposed approach deals with it effectively, in two ways: First, the overall top-down architecture of the approach is meant to use less complex methods, such as the stacked plots and the Mobile Network Graph on large amounts of data, and then to focus on smaller datasets, so that the more complex multiobjective method is run with few computational resources required. Second, by using the Minimum Spanning Tree, instead of the full graph or a neighborhood graph, only a few number of edges are kept, specifically equal to the number of vertices minus one. The few number of edges renders the minimization of the force-directed potential function computationally efficient.

Since there are multiple descriptors extracted from the users, multiple notions of distance are considered and multiple objective functions are defined. The simultaneous minimization of all the objective functions is normally infeasible; however, such problems are solved by multi-objective optimization methods, which result in a set of solutions, instead of one. The set of solutions, namely the Pareto set, contains those solutions which cannot be further improved without sacrificing one of the objectives. These solutions represent different optimal trade-offs among the multiple objectives and can provide insight into various aspects of the mobile network data and the user behaviors. More details can be found in [START_REF] Kalamaras | Multi-Objective Optimization for Multimodal Visualization[END_REF].

Hypothesis formulation and validation

Moving in even less abstract levels, MoVA allows the operator to formulate network security-related hypotheses for specific limited sets of data, and attempt to verify or reject them. The operator formulates a hypothesis by selecting a subset of the available CDR records, either manually or through one of the visualization tools. The hypothesis data are provided as input to the various visual analytics tools of the proposed system. Depending on the type of question submitted by the analyst, appropriate anomaly detection or visualization methods are employed in order to provide answers. The produced answers, either quantitative or visual, are presented to the analyst. Using this information as a guideline, the analyst can further explore the dataset by focusing on specific aspects of the data, or altering the question. The operator can submit one of the following three supported hypotheses.

Anomalous users hypothesis: The anomalous users hypothesis is testing if a subset of users exhibits abnormal behavior within a given time frame. Initially, the analyst selects a time frame of interest and then selects a subset of users which he/she desires to check if they have abnormal behavior or not. Then, the activity graphs are created for each user and provided as input to an anomaly detection algorithm, in order to identify if the selected users exhibit abnormal behavior. These activity graphs are directed graphs, in which the vertices represent users, and edges the communication between them. The weights of the edges represent the number of communication events between two users. Multiple features are then extracted from the graph created from the neighbors of each user, and are used for the supervised classification (using random forests [START_REF] Breiman | Random forests[END_REF]) of each user into two classes, normal and anomalous. The extracted features are: 1) number of edges, 2) Edge weight entropy, 3) Graph entropy, 4) Maximum LOF value of the edge weights, 5) Ratio of outwards towards inwards edge weights, 6) Average Outward/Inward Edge Weight, and 7) Number of Outward/Inward Edges with a specific weight. The result of anomaly detection is a classification of the selected users as anomalous or normal, which is utilized to calculate the probability that the original hypothesis is true.

Core network impact hypothesis: The core network impact hypothesis tests if a set of selected users exhibit abnormal behavior simultaneously, indicating a DoS attack which has a negative impact to the core network. The data available for the users are split into time windows of duration determined by the operator and anomaly detection is performed for each of the time frames. This results in a series of classification results, which are finally used to test the original hypothesis.

Botnet attack hypothesis: In this hypothesis, the analyst suspects that the detected attacks against the network were launched by a botnet, installed on multiple mobile devices. In order to validate or reject this hypothesis, the system searches in the past activity of the selected users for common patterns, which indicate 1) the botnet command and control interfaces, and 2) the infection methods that they use to spread and infect additional devices. To achieve this, the past CDR activity of each user is used to construct a symbolic time series, in which each symbol represents a specific communication event (e.g. Call towards user-X, or SMS from user-Y). Multiple symbolic time series are created, one for each user. These time series are provided as input to a Frequent Episode Mining algorithm (FEM) [START_REF] Ding | Efficient mining of closed repetitive gapped subsequences from a sequence database[END_REF], for the identification of common behaviors. The results are filtered based on a list of events which are known to be normal. The patterns that remain correspond to either the command and control interfaces, or the infection methods, or both.

It should be noted that the whole application can be used either with stored historical network data, in an off-line manner, as well as with streaming data, arriving at each time instance, in real time, in an on-line manner. The arriving signals can be initially visualized with the stacked graphs visualization approach, described in Section 3.1, where the new signals are appended to the right part of the plots. By selecting a time instance of the new data, the Mobile Network Graph can be visualized too. If a small time window around the arriving data is considered and selected by the operator, e.g. one day, behavioral features can be extracted from the CDR records and be used within the multi-objective behavior visualization. Using different durations for the time window, features can be extracted that correspond to smaller or larger time periods, thus allowing the operator to overview various temporal characteristics of the data and cope with the arriving data velocity. The various supported hypotheses can also be verified using this time window around the arriving data, for further analysis.

Experimental results

MoVA has been evaluated with synthetic data, in a scenario of Denial-of-Service (DoS) attack, caused by SMS-sending malware. In this section, the dataset used Fig. 4. The visualization of the signaling activity. The anomaly score on 14-Jan-2014 has suddenly increased when compared to the previous days. In addition, the volume of the T21 signaling message (i.e. the number of SMS received by the network subscribers) has also increased. The graph representation of the signaling activity is shown on the lower part, and shows that activity has uniform distribution with respect to the network topology, and thus, the attacking devices are not geographically constrained.

is described, followed by the evaluation of each of the visual analytics tools and methods presented in Section 3.

GEDIS dataset

GEDIS Studio [START_REF] Genielog | GEDIS Studio online[END_REF] is an online tool for generating Call Detail Record (CDR) data. For the purposes of evaluating the proposed MoVA application, CDR data corresponding to the SMS flood scenario of [START_REF] Kim | A detection mechanism for SMS flooding attacks in cellular networks[END_REF] have been produced. Six months (25 weeks) of data have been generated, for 4800 mobile phone users. The users are split into two groups. The first group consists of 4500 normal users with an SMS sending rate of 8 messages per day, while the rest 300 users are considered to be infected by SMS flooding malware, sending SMS messages with a rate of 64 messages per day, when the malware is active. The attacks are considered to occur from day 106 to 126 and from day 148 to 154, i.e. 4 weeks in total.

Results

Figure 4 presents the visualization of the signaling activity. As shown in the stacked graphs representation, the anomaly score on 14-Jan-2014 has suddenly increased by a large factor, when compared to the previous days. Along with this increase, there is also an increase in the T21 signaling message. This signaling message represents the number of SMS received by the network subscribers. The graph representation of the signaling activity on 14-Jan-2014 for a specific hour is shown on the lower part of Figure 4. The signaling activity has uniform distribution with respect to the network topology, and thus, the attacking devices are not geographically constrained. Selecting the 14-Jan-2014 as the time period for more detailed analysis, the analyst filters the CDR records and proceeds with the visualization of the CDR activity, using the multi-objective visualization tool. The THD and RHD descriptors were extracted and used as the multiple modalities. Figure 5(a) depicts the results of the visualization. This visualization corresponds to a solution from the center of the Pareto front. A small group of users appear to separate from the rest of the users, indicating that they have a much different, probably abnormal, behavior. The red/dark color in Figure 5(a) denotes the users selected by the operator for further analysis. By selecting this group, the operator wishes to test various hypotheses regarding these users.

Testing the anomalous users hypothesis is performed by running the anomaly detection method described in Section 3.3. The results of the anomaly detection are illustrated in Figure 5(b), where the red/dark color denotes the users detected as anomalous. It is apparent that most of the selected users of the small cluster are indeed identified as anomalous by the anomaly detection algorithm. The probability that the hypothesis is valid, i.e. that the selected users exhibit anomalous behavior, is 87.6%.

The core network impact hypothesis can be used by the operator to test if the selected users exhibit a behavior that has an impact to the core network, rather than to individual users. Figure 5(c) contains the results of testing the network impact hypothesis. From the leftmost figure, it is apparent that the number of anomalous users is maintained throughout the whole day, which is a first indication that the attack aims at the network malfunction. Furthermore, the network traffic of the selected users is close to 50% of the total traffic of all users, causing a significant negative impact to the core network.

Finally, the same users were utilized in order to run the botnet attack hypothesis. The past 14 days of CDR activity were utilized for the detection of common behaviors. The FEM algorithm identified multiple patterns which con- tained the normal event saf.gr . Removal of this event results in one dominant pattern, namely, mal.gr, 10.10.10.1 . This pattern shows that all the anomalous users visited the webpage mal.gr, from which they were infected by the botnet. Afterwards, the botnet communicated with the specific IP address 10.10.10.1 for the command and control interface. Compared to existing commercial solutions available, the proposed MoVA application has several advantages. Table 1 summarizes the differences between existing solutions and the proposed one. Widely used visualization applications, such as SAS VA [START_REF]SAS Visual Analytics[END_REF] and Tableau [START_REF]Tableau Analytics[END_REF], although they provide a variety of visualization techniques, are generic in nature. They can be used with any type of multi-dimensional data and they are not targeted to mobile network security. Certifuge Systems [START_REF]Certifuge Systems[END_REF], is further able to handle data related to IP network security, but it still lacks support for mobile network security data. On the other hand, applications that are targeted to mobile network security, such as Motive Security Guardian [START_REF]Alcatel Security-guardian[END_REF] and NSN Mobile Guard [START_REF]NSN mobile guard[END_REF], do not provide visualization tools that would allow the mobile network operator to have an insightful overview of the network operation. The proposed MoVA application is able to fill these gaps, by providing visualizations targeted at mobile network security, while still being able to be easily extended in order to handle IP network data, as well as any generic type of data.

Furthermore, existing mobile network-oriented applications utilize one kind of data, either signaling or CDR. Utilizing both signaling and CDR data for the analysis and visualizations of the proposed method has the advantage of allowing the implementation of the proposed top-down approach. The large amounts of constantly arriving signaling data are used in the initial stages of the visual analysis, utilizing more straightforward techniques, such as the stacked plots and the Mobile Network Graph, which present an overview of the large available data. Then, the more elaborate techniques, such as the multi-objective visualization and hypothesis testing are performed on smaller sets of data, utilizing the CDR records available for the users. This allows the application to handle big data, by allowing both a large-scale overview of the data and smaller-scale analysis and visualizations.

Finally, compared to methods that only utilize anomaly detection algorithms, the use of visual analytics, in the proposed application, has the advantage of data exploration. Visual analytics, by definition, involves the human operator in the visualization process, by allowing him/her to select data subsets, fine-tune parameters, compare different visualizations, etc. Combining the fast analytic processes of the automatic anomaly detection and analysis tools with the human perception, through visualization and interaction, leads to an effective means to explore the available data, to allow the operator to view different aspects of them and, ultimately, to gain more insight and arrive at a more accurate decision. Applications such as Motive Security Guardian [START_REF]Alcatel Security-guardian[END_REF] and NSN Mobile Guard [START_REF]NSN mobile guard[END_REF], although able to detect malware activity, do not provide the mobile network operator with visualizations that would allow him/her to instantly view the infected users, the origins of an attack, etc.

Conclusions

In this paper, a visual analytics application for the monitoring of large mobile phone networks, focused on network security and aiming at detecting traffic anomalies, namely MoVA, has been proposed. MoVA provides the mobile network operator with several tools allowing her/him to have an overview of the whole network and gradually focus on less abstract data that are candidates to be abnormal, thus gaining useful insights in the network operation. For an overview of the whole network, the Mobile Network Graph and stacked graphs visualizations are provided, which present a spatio-temporal view of the network. Focusing on a specific time period, the behavioral similarities of a set of users is analyzed through the multi-objective visualization tool, which aggregates information from many traffic attributes, in order to visually separate normal from abnormal behaviors. Finally, the operator can focus on a set of candidate anomalous users and use the hypothesis testing tool to verify specific hypotheses regarding the anomalous behavior of the users, its impact to the core network, and its root causes. Experimental evaluation of MoVA with a CDR dataset verifies the applicability of the proposed tools in visualizing large network traffic data and facilitating the operator. Future work includes adjusting and evaluating the proposed approach in various mobile network-related use cases.

Fig. 1 .

 1 Fig.1. The proposed data pipeline followed for the analysis of Big Data collected from mobile cellular networks.

Fig. 2 .

 2 Fig. 2. Overview of the proposed visual analytics tools integrated in MoVA. An overview of the network is provided to the analyst by visualizing aggregated signaling activity. Reducing the abstraction level, CDR data of specific users are used to visualize their behavioral similarities. Finally, the analyst can formulate and test specific security-related hypotheses.

Fig. 3 .

 3 Fig. 3. Visualization of the signaling activity in the network. (a) Stacked graphs visualization of the anomaly scores and signaling volume. (b) A scheme of the proposed layered layout of the Mobile Network Graph.

Fig. 5 .

 5 Fig. 5. The results of the hypothesis testing tool. (a) The result of the multi-objective visualization method. The red/dark color denotes the users selected by the operator. (b) The results of the anomalous users hypothesis. The red/dark color denotes the users identified as anomalous and the green/light color the normal users. (c) The results of the core network impact hypothesis.

Table 1 .

 1 Comparison to commercial mobile network security applications.

				IP	net-	Mobile
	Name		Generic	work		network	Techniques
				security	security
	SAS VA [13]		-		-	Line charts, pie charts, colored tables.
	Tableau [14]		-		-	Line charts, histograms, pie charts, colored tables, maps.
	Centrifuge Sys-tems [15]				-	Histograms, graphs, pie charts, colored tables.
	Motive Security Guardian [16]	-	-		Anomaly detection techniques.
	NSN Guard [17]	Mobile	-	-		Anomaly detection techniques.
						Stacked plots, graphs, k-partite
	MoVA					graphs, multi-objective visual-ization, anomaly detection algo-
						rithms, hypothesis testing.


Niki Sfika
email: sfika@ceid.upatras.gr

Konstantinos Manos
email: manosk@ceid.upatras.gr

Aigli Korfiati
email: korfiati@ceid.upatras.gr

Christos Alexakos
email: alexakos@ceid.upatras.gr

Spiridon Likothanassis
email: likothanassis@ceid.upatras.gr

Workflow Coordinated Resources Allocation for Big Data Analytics in the Cloud

Keywords: cloud computing, workflows, resources allocation, big data analytics

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Cloud computing involves a set of new technologies that permit the dynamic allocation of computational resources (storage, cpu, memory). This is extremely useful, especially when performing high demanding data analysis. In the modern world of information, there are a lot of sources (web, open data databases, IoT) that lead to the collection of huge amounts of data [START_REF] Gantz | The digital universe in 2020: Big data, bigger digital shadows, and biggest growth in the far east[END_REF]. Big Data Analytics is the business and informatics domain, responsible for processing such large capacity of data. Problems in this domain require, in most cases, a lot of computational resources, which can be provided by the cloud on demand [START_REF] Ji | Big data processing in cloud computing environments[END_REF]. Moreover, Big Data Analytics solutions are based on complex pipelines where several tasks must be performed by different systems in a specific order. Some tasks may also require parallelization and/or distribution on computer systems clusters in order to be executed. On the other hand, the notion of workflows is not new for the representation of complex jobs. Workflows are considered as powerful tools for enabling the composition and execution of complex analysis jobs in distributed environments [START_REF] Deelman | Workflows and e-Science: An overview of workflow system features and capabilities[END_REF]. Initially, workflows were used to represent in a formal way the business processes, but in the last years they have been widely used for the orchestration of the execution of complex processes in the field of integration of information systems [START_REF] Da Xu | Enterprise systems: state-of-the-art and future trends[END_REF]. Now, workflows are able to provide a reliable solution that encompasses all the steps of Big Data Analytics, from data access and filtering to data mining and processing for extracting knowledge [START_REF] Talia | Clouds for Scalable Big Data Analytics[END_REF].

Cloud computing is a relatively new and prominent set of technologies in the area of ICT and it is anticipated to play a key role in the modern information systems. In cloud computing, all available computational resources (such as networks, storage, applications, servers, etc.) are provided via web as utilities. The main difference of cloud computing against traditional approaches can be summarized to the on-demand access to the resources pool and the flexible and adaptable resources provision management by the cloud provider. In cloud computing, users can increase or decrease the capacity and consume as many resources as they need for their applications. Furthermore, the technologies composing cloud computing can be combined in order to provide highly scalable and adaptable applications owing to the fact that new applications and features can be deployed significantly faster. Another worth mentioning characteristic of cloud computing is that all the provided resources are hosted on remote server infrastructures, allowing them to be accessed remotely as long as there is Internet connection and users and systems can collaborate simultaneously on these resources [START_REF] Kagadis | Cloud computing in medical imaging[END_REF].

Big Data Analytics is an emerging field and it has been applied to various areas such as Business Intelligence and Analytics (BI&A), scientific problem solving and bioinformatics. Especially in the field of BI&A there are several proposed applications and case studies on a) ecommerce and market intelligence, b) e-government and politics, c) science and technology, d) smart health and well-being, and e) security and public safety [START_REF] Chen | Business Intelligence and Analytics: From Big Data to Big Impact[END_REF]. Additionally, in the field of bioinformatics, the large amount of biological data and information that has been gathered or generated the last decade is situated in different databases and thus, Big Data Analytics is considered to be able to give both a solution for the information integration as well in its analysis [START_REF] Howe | Big data: The future of biocuration[END_REF]. Variety, velocity, scaling, complexity, interpretation and security problems with Big Data raise challenges at all phases of the data analysis pipeline [START_REF] Demchenko | Addressing big data issues in scientific data infrastructure[END_REF]. Especially, in problems of data mining the analysis pipeline is complex including tasks for data collection and integration from different sources, data preprocess and cleaning and finally data analysis using various techniques as machine learning, statistics or heuristics methods [START_REF] Padhy | The survey of data mining applications and feature scope[END_REF].

In the current article, we are focusing on the problems of scaling and complexity in the Big Data analysis. The proposed architecture adds a level of calculation of the required computational resources based on the problem input and their reservation in a cloud infrastructure. The proposed system manages the creation/deletion of the Virtual Machines (VMs) that will finally execute the analysis and ensures that the minimum required resources in terms of performance and cost will be used. Moreover, the system employs workflows in order to face the challenges of the complexity of the Big Data analysis pipeline. Users can define with a visual tool the flow of the analysis tasks easily, decreasing the complexity for developing complex programs code.

The proposed approach is presented in detail in the rest of the paper. In Section 2 some significant technologies and related work in the area of cloud computing, workflows and Big Data Analytics are presented. Section 3 describes the basic functional components of the proposed architecture and Section 4 presents in details the utiliza-tion of workflows for a) resources allocation and b) Big Data Analytics processes execution. Finally, Section 5 concludes the paper.

2

State of the Art Technologies

Cloud computing and resources allocation

As mentioned above, Cloud Computing is a set of emerging technologies giving end-users access to all types of computational resources and the ability to consume them according to their demands. The efficient management of the actual hardware resources (storage, memory, processors and bandwidth) on virtualization environments concludes to a more efficient cost model on the cloud [START_REF] Iosup | Performance analysis of cloud computing services for many-tasks scientific computing[END_REF].

Cloud Computing is a general term denoting various levels of resources provision as services to the end-users. In an attempt to categorize the services provided by the cloud, the National Institute of Standards and Technology (NIST) of USA proposed three main delivery models [START_REF] Mell | The NIST definition of cloud computing[END_REF]:

1. Infrastructure-as-a-Service (IaaS) model offers on demand virtual machines with their own operating system, storage and network, 2. Platform-as-a-Service (PaaS) model offers functionalities on the computational resources as Application Programming Interfaces (APIs) for new applications development , 3. Software-as-a-Service (SaaS) model offers applications as a service. Users of SaaS applications are able to utilize their functionalities through a front-end web portal without additional hardware requirements in their local device.

The full installation of a cloud infrastructure requires the installation of software for managing the IaaS infrastructure. This software is often referred as cloud computing platform and its primary scope is the management of the virtualization infrastructure of the bare-metal servers. Some of the most commonly used open-source platforms are OpenNebula, Nimbus, Openstack and Cloudstack. Nevertheless, there are commercial IaaS vendors, such as Amazon WS (Amazon.com, Inc., WA, US), Google Cloud (Google Inc., CA, US), Rackspace (Rackspace Inc., TX, US) and Azure (Microsoft Corporation, WA, US).

In the proposed approach, the cloud platform used in order to provide cloud services infrastructure was Apache Cloudstack. Cloudstack is an open source cloud computing platform that controls and manages computational resources, storage and network to deploy a private or a public IaaS cloud [START_REF] Baset | Open source cloud technologies[END_REF]. It provides a flexible cloud orchestration for the developer. For the virtualization environment, our infrastructure is based on a Xen Hypervisor delivered from the open-source XenServer (Citrix Systems, Inc., FL, US). XenServer is a virtualization platform which practically includes a hypervisor providing the appropriate isolation between the running virtual machines (VMs) [START_REF] Williams | Virtualization with Xen (tm): Including XenEnterprise, XenServer, and XenExpress[END_REF]. Cloudstack offers a web graphical user interface (GUI) for both administrators and end-users to manage the cloud. Using the GUI, users can manage all provisioned virtual machines and networks. Another important functionality is Cloud-stack's HTTP API which can be used for the execution of any required operation by third-party applications. Following the basic architecture of most cloud computing platforms, CloudStack consists of two major components: a) the resources to be managed and b) the management server which must be informed about all these resources.

The management server orchestrates and allocates the declared resources in the cloud infrastructure. It allocates virtual machines to hosts and controls the assignment of storage and IP addresses to each virtual machine instance. It provides a series of options through the GUI and the API interfaces such as private/public networks, templates and snapshots management.

The host, being a single machine, provides its resources, such as memory, CPU, storage and network, in order for the virtual machines hosted on it to run. A hypervisor software (XenServer in our implementation) is installed in the host for managing the guest VMs and the virtual networks. If more capacity is needed for the guest VMs, a supplementary host can be added. What is important is that CloudStack has the ability to detect the amount of all the aforementioned resources automatically. Finally, the actual allocation of the resources on the hardware infrastructure is information completely unsighted to the end-users.

Workflow for services orchestration

A workflow is an orchestrated sequence of tasks describing the execution of a specific scientific or business process, and the exchange of information between them. Each task consists of a set of defined actions executed in a specified order. After a workflow starts operating, work and data pass through each step from start to finish until the process is complete. Usually, the creation and operation of a workflow is assigned to a Workflow Management System (WFMS) that handles the invocation and coordination of its various components [START_REF] Van Der Aalst | Workflow management: models, methods, and systems[END_REF]. Workflow applications have steadily increased in complexity and variation leading to the need of more flexible workflow systems able to handle and organize a large set of often incompatible resources [START_REF] Curcin | Scientific workflow systems-can one size fit all?[END_REF].

Workflows present a technology to achieve this supporting automation in the definition and execution of complex and integrated tasks orchestrating the invocation of web services (WSs) provided by different systems. The above approach, combining broadly accepted WSs and workflows, has mandated WS description in terms of workflow relevant standards. These include among others [START_REF] Ko | Business process management (BPM) standards: a survey[END_REF], the Web Services Business Process Execution Language (WS-BPEL) that follows the Business Process Model and Notation (BPMN) [START_REF] Chinosi | BPMN: An introduction to the standard[END_REF], and the Web Services Choreography Description Language (WS-CDL) [START_REF] Cambronero | A Petri net approach for the design and analysis of Web Services Choreographies[END_REF] The workflows depict processes that are being executed and coordinated by Workflow Management Systems. The last decade, workflows are widely used in the interoperability of information systems for integrating complex business processes. Thus, some of the well-known software vendors have introduced Workflow Management Systems (WMS), such as IBM BPM, Microsoft Windows Workflow Foundation, and SAP Business Workflow. Also, open-source projects have followed this trend, such as Apache ODE, jBPM, YAWL and Taverna. The later was used in the proposed approach because of its simplicity in installation and administration.

Taverna [START_REF] Wolstencroft | The Taverna workflow suite: designing and executing workflows of Web Services on the desktop, web or in the cloud[END_REF] is an open source and domain-independent Workflow Management System. It provides the infrastructure needed to design and run workflows, monitor the execution process, and manage the handling and coordination of a variety of different workflow services, while pipelining the data processing. Due to its flexibility, it can interact with a large variety of resources and tools, and can be used to create workflows that apply to a wide range of scientific fields. Among other things Taverna has been designed to offer automated communication and invocation of several components coming from different service providers, efficient handling of data exchange between different types of applications and easy data conversion in order to avoid compatibility issues between services. It also offers a better insight into the operating procedure through a detailed view of each component's interactions and results.

Big Data Analytics in the cloud

Applications for Big Data analytics are tightly connected to the process workload characteristics, such as scale, scope and nature. These characteristics provide the principal requirements to the future hardware and software [START_REF] Kambatla | Trends in big data analytics[END_REF]. Thus, the on demand provision of both hardware and software as a service from cloud providers makes cloud computing sufficient to be used as infrastructure for Big Data Analytics applications.

Scalable and distributed management of both process tasks and stored data for large amount of data has been a challenge for the research community for more than three decades. Today, the researchers focus on the design and development of systems that serve both intensive and ad-hoc analysis workloads [START_REF] Agrawal | Big data and cloud computing: current state and future opportunities[END_REF]. There are approaches in the direction of automatic scaling of cloud-based applications. Vaquero et al. [START_REF] Vaquero | Dynamically scaling applications in the cloud[END_REF] proposed a holistic approach for cloud applications scalability in both IaaS and PaaS models. Their approach is based on the overall management of collections of interrelated and context-dependent VMs by using policies and rules. Ming and Mao [START_REF] Mao | Auto-scaling to minimize cost and meet application deadlines in cloud workflows[END_REF] presented an approach for scheduling the execution of data analysis problems. In their approach the basic computing elements are VMs that are characterized by sizes and costs. The jobs are depicted as workflows and the final goal is to ensure all jobs are finished at minimum financial cost within their deadlines. User requirements are considered as deadlines. Another approach that incorporates the Handoop infrastructure for computer clustering and distributed analysis of Big Data is Starfish. Starfish is a self-tuning system for Big Data analytics, which automatically adapts to user needs and system workloads in a way transparent to the user [START_REF] Herodotou | Starfish: A Self-tuning System for Big Data Analytics[END_REF].

System Architecture

The proposed architecture aims to provide users with the flexibility of defining their pipeline analysis through a user-friendly workflow editor -the Taverna Workbenchand to automatically scale the execution of this workflow by calculating the required resources and managing the creation/deletion of the appropriate VMs. The system architecture consists of a series of components and it is based on the IaaS cloud service model. It is designed in such a way, in order to achieve high performance in terms of time and cost saving. The proposed architecture provides an automated process execution without user's interference, efficient communication between the components and an easily manageable graphical user interface for the users to set a problem's specifications and receive its results. Figure 1 depicts the functional components of the proposed architecture along with their interactions. The User portal is a web-based graphical user interface for the communication between the user and the system. It is developed following responsive design techniques and it is accessible from a variety of devices such as smartphones, tablets and PCs. Users can define new jobs by selecting the appropriate workflow from a list of predefined ones created off-line in the Taverna Workbench. The User portal also supports a mechanism for user authentication and permissions. Defined jobs, their parameters and user information and credentials are kept to a central relational Database (MySQL in our approach).

The Database is the module of the architecture that holds all the information such as the number of problems, their results and specifications, the number of users and their details etc. It can be accessed by the user portal, the workflow manager and the task monitor in order to store their information.

The CloudStack management server is the software module of Cloudstack platform with which the workflow manager and monitor communicate. It receives re-quests for VM administration tasks through an HTTP API. Its API is used from the other architecture's modules for either VMs creation/deletion or polling information for the running VMS.

The Workflow manager is responsible for the execution of the appropriate workflow when a new analysis job appears. It communicates with the database iteratively to receive details about new defined jobs. The actual execution of the workflow is performed on the Taverna Server that is part of the Workflow manager. The monitoring of the workflow's execution is performed by the HTTP API provided by the Taverna Server.

The Task Monitor manages the communication between the system and the VMs that are created to execute the actual analysis pipeline. More precisely, it receives a process's progress from the VMs through a series of messages. Upon a message arrival indicating the end of a process, it updates the database by marking this job as completed. The task monitor is based on the Advanced Message Queuing Protocol (AMQP) [START_REF] Bernstein | Blueprint for the intercloud-protocols and formats for cloud computing interoperability[END_REF].

4

Workflows for Resources Allocation and Data Analysis

Resources Allocation Workflow

The Resources Allocation Workflow contains the core workflow that depicts the initial steps that must be made in order to calculate the computational resources needed for the data analysis problem and the instantiation of the virtual infrastructure. The instantiation task contains all the appropriate steps for creating the VM instances and initializing them as depicted in Figure 2.

Fig. 2. The task of resources allocation and job analysis initialization

Initially, the workflow receives the data needed for both the orchestration process and the calculations within the VMs. The first step is the calculation of the appropriate computational resources (CPU speed, number of CPUs, RAM size etc.) required for each VM, depending on the size of the input data. This is achieved by running a python script that identifies the size of the problem and chooses the appropriate service offerings. An empirically specified rule that performs the mapping between the problem size and the necessary resources is employed. For example, when executing the problem addressed in [START_REF] Korfiati | IRaaS: A Cloud Implementation of an Interface Relaxation Method for the Solution of PDEs[END_REF] the resulting rule is depicted in Table 1: All python scripts, like this one, are run with the use of the Taverna "Tool" service which allows the execution of commands locally on the machine the workflow is run on (Workflow manager). The outcome of this procedure is the number of the VMs and the template (predefined data analysis software) and the service offerings (operat-ing system, memory, CPU and storage) for each VM. Figure 3 depicts the segment of the workflow that invokes the script for the calculation of the requested resources. The next step in the workflow is to invoke the appropriate HTTP methods of the Cloudstack Server API for creating the calculated VMs. Before a HTTP call is issued during the workflow execution, a python script must be run, with inputs from previous steps, to generate the URL required for that specific action, like deleting VMs. This applies to all HTTP API requests to Cloudstack. In order to ensure that the VMs are indeed running before moving to the next step a loop structure is used in the workflow that continuously requests new VMs status and stops when all the VMs are up and running. In Figure 4 the workflow loop structure is denoted with a double-line rectangular. Before starting the execution of the analysis workflow, it is necessary that the VMs have been booted and are ready to use. In this direction, each VM template is empowered with a REST service that checks if all the services needed from the analysis software are up. As above, a loop structure is used for requesting the status of the operating system and the analysis software that is installed in the new VM (Fig. 5). After the Workflow manager ensures that the VMs are booted and the software is ready, it executes the Analysis Workflow, which is presented in the next subsection. When the analysis pipeline is finished, the Task Monitor is responsible for updating the Database with the results and marking the analysis job as completed. Once again a script is used to communicate with the Database and wait until that final status change takes place. Then the VMs can be deleted. First, the VMs must be stopped. A workflow loop structure is also used in this case to check VMs' status until they shut down. After, a HTTP call is made to Cloudstack to delete the stopped VMs (Fig. 6).

Analysis Workflow

The analysis workflow is the segment of the entire workflow that is responsible for executing the analysis pipeline. Its structure depends on the complexity of the data analysis and the involved steps. As example, in Figure 7, we present a small analysis workflow with three VMs that are solving in parallel large-scale differential equations using the Interface Relaxation methodology [START_REF] Korfiati | Serial and Parallel Implementation of an Interface Relaxation Method[END_REF]. The computational resources and the number of VMs are tightly related to the size of the input. The resources allocation method that was used in the first step of the Resources Allocation Workflow has been tested in [START_REF] Korfiati | IRaaS: A Cloud Implementation of an Interface Relaxation Method for the Solution of PDEs[END_REF] providing significant performance optimization, especially in large datasets.

Workflow Execution

The overall workflow described in the previous paragraphs has been designed using the Taverna Workbench. The outcome is depicted in Figure 8. The Taverna workflow was stored in T2flow file format in order to be loaded in the Taverna Server that acts as the Workflow manager of the proposed architecture.

The execution of the workflow is handled by the Taverna Server. The Taverna Server is running on a VM on Cloudstack and can have local access to all VMs used during the problem execution. The whole communication with the Taverna Server is performed through REST API calls. More specifically there are three steps needed in order to initiate and run a workflow.  POSTing a T2flow document describing the desired workflow process. A workflow id is returned that allows us to handle further requests.  PUTing all input information and data needed by the workflow in their respective locations, as these are described by the workflow's input ports.  PUTing the value 'Operating' to the workflow's status.

After the execution of the workflow has begun, the Task Manager will periodically check the workflow's status through a HTTP request, until it has the value 'Finished'. This means that the execution of the problem has been completed and the VMs have been destroyed. Then, the job is also marked as 'Complete' in the database by the Task Monitor. When everything is done, the Task Manager checks the Database again for other pending jobs, and if there are, it chooses the next in the queue and repeats the process described above.

Conclusion and Future Work

One of the cornerstones of cloud computing is the elastic computational resources that it provides on demand. This enables users to quickly scale up and down the reserved resources according to user, business or performance requirements. The proposed architecture aims to achieve a certain level of dynamic scalability and flexibility in applications executing complex and highly demanding Big Data Analytics processes.

The proposed solution is based on two axes, the first one is the calculation of the required computational resources in terms of Virtual Machines and the second is the depiction of the whole process including virtual infrastructure preparation in workflow.

The proposed system introduces a step in the analysis pipeline where it calculates the number and size of the VMs that are needed on the cloud infrastructure for the execution of the analysis. The calculation is made taking account both the input data and the available resources in the cloud infrastructure.

For the execution of the analysis, a workflow with two segments is used. The first segment, Resources Allocation Workflow is composed of two steps in order to firstly identify the resources requirements and after to create the appropriate VMs for the actual analysis execution. The second segment is the Analysis Workflow that realizes the analysis pipeline in the cloud infrastructure (VMs and networks) that is dynamically created in the first segment of the workflow.

By the use of workflow coordinated resources allocation, the aforementioned approach can increase the performance in both terms of time and cost efficiency. Nevertheless, the proposed system is subject for further improvements. The authors have already started the research in the empowering of the calculation algorithm with rules that depict user or business requirements. Also, a self-improvement time-scheduling algorithm is going to be added for better allocation of resources in systems with high rate of analysis requests but with limited hardware resources.

Fig. 1 .

 1 Fig. 1. The proposed system architecture

Fig. 3 .

 3 Fig. 3. Workflow segment for calculating the required resources regarding input data

Fig. 4 .

 4 Fig. 4. Workflow segment for creating new VM

Fig. 5 .

 5 Fig. 5. Workflow segment for ensuring that VM's software is ready

Fig. 6 .

 6 Fig. 6. Workflow segment for stopping and destroying VMs

Fig. 7 .

 7 Fig. 7. Analysis workflow example

Fig. 8 .

 8 Fig. 8. The workflow for resource allocation and analysis execution as depicted in the Taverna Workbench

Table 1 .

 1 Empirical rule for the resources allocation for a specific test case

	Case	Problem size	Resources
	1	<= 75000	1 core, 2 GHz and 1 GB RAM
	2	(75000, 150000)	1 core, 2 GHz and 2 GB RAM
	3	>=150000	1 core, 2 GHz and 4 GB RAM


Dimitar Stanev
email: stanev@ece.upatras.gr

Panagiotis Moschonas
email: moschona@iti.gr

Konstantinos Votis
email: kvotis@iti.gr

Dimitrios Tzovaras
email: dimitrios.tzovaras@iti.grwww

Konstantinos Moustakas
email: moustakas@ece.upatras.gr

Simulation and Visual Analysis of Neuromusculoskeletal Models and Data

Keywords: neuromusculoskeletal simulation, big data, forward simulation, inverse simulation

This work presents a novel medical decision support system for diseases related to the upper body neuromusculature. The backbone of the system is a simulation engine able to perform both forward and inverse simulation of upper limb motions. In forward mode neural signals are fed to the muscles that perform the corresponding motion. In the inverse mode, a specified motion trajectory is used as input and the neural signals that are the root cause of this particular motion are estimated and investigated. Due to the vast amount of information that results from even simple simulations, the results are presented to the expert using visual analytics metaphors and in particular both embodied and symbolic visualizations. Several use cases are presented so as to demonstrate the analytics potential of the proposed system.

Introduction

Neuromusculoskeletal modelling and simulation has emerged as a potential tool for understanding how different neurological diseases have a direct impact on motor behaviour [START_REF] Buchanan | Neuromusculoskeletal Modeling: Estimation of Muscle Forces and Joint Moments and Movements From Measurements of Neural Command[END_REF]. Because of the diversity, complexity and the massive amount of data, presenting in studying the aforementioned problem ether measured or produced by the simulation schemes presents a grand challenge. New tools and methods must be developed both for the production of reliable results and means to organize and process them especially in a personalized and patient driven health care [START_REF] Swan | Emerging Patient-Driven Health Care Models: An Examination of Health Social Networks, Consumer Personalized Medicine and Quantified Self-Tracking[END_REF][START_REF] Nitesh | Bringing Big Data to Personalized Healthcare: A Patient-Centered Framework[END_REF]. By introducing these technologies in health care, will have a direct impact on the patients' quality of life, better management of the treatment and support in taking the best possible decisions for personalized diagnosis, reduction of the overall health cost, important contribution to cope with the growing demand of health services, new market areas and many benefits [START_REF] Chen | Business intelligence and analytics from big data to big impact[END_REF].

There is a great challenge when studying the massive amount of data handled by the biomechanics simulation process. Visual Analytics provide the suitable tools to cope with this matter as it is defined by the union of information visualisation and scientific visualisation that focuses on analytical reasoning facilitated by interactive visual interfaces [START_REF] Wong | Visual Analytics[END_REF]. Visual Analytics have been assembled successfully in the domains of brain activity simulation, as well as in the biomechanical motion analysis [START_REF] Keefe | Integrating Visualisation and interaction research to improve scientific workflows[END_REF]. Spurlock et al. [START_REF] Spurlock | Combining automated and interactive visual analysis of biomechanical motion data[END_REF] presented a framework for combining automated and interactive visual analysis techniques for use in high-resolution biomechanical data, analysing the complex motion of animals. Additionally, the study of human factors and ergonomics have benefited from advanced visualisation techniques in order to extract high level rules from the complex data space [START_REF] Drury | Human factors/ergonomics implications of big data analytics: Chartered Institute of Ergonomics and Human Factors annual lecture[END_REF]. Moreover, Visual Analytics provide the proper algorithms for biomedicine by analysing multi-scale biomedical data, especially when they derive from a variety of sources [START_REF] Vaquero | Visualisation and user interaction methods for multiscale biomedical data[END_REF]. Finally, it is worth mentioning that Visual Analytics are used extensively in drug discovery frameworks [START_REF] Huan | Systems biology Visualisation tools for drug target discovery[END_REF].

Analysis and Simulation

General Architecture

Fig. 1. The general architecture of the proposed system: the input data (colored orange), the model and simulation stages and the presentation of the results using visual analytics.

Figure 1 presents the general architecture of the system. The input to the model are the highly dimensional data, such as MRI, CT or ultrasound recordings, where model parameters are assessed and tuned per patient. Namely, each individual patient presents different geometric characteristics of the skeletal and muscular system, physical quantities: mass, inertia, length of the segments and muscular: muscle geometry, force capacity, length, point of action and many others. After the model is parametrized, simulation methods can be applied for studying different factors that have a direct impact on the motor behaviour. For the simulation, additional data are required, such as 3D motion recording, external forces, attached sensing devices and electromyography (EMG). The output produced by the simulation can be the state of the neuromuskuloskeletal system, e.g. joint torques, reaction forces, muscle lengthening, moment arm, individual forces, neural activation signals and many model dependent variables. Finally, the aforementioned results must be presented to the clinicians in an intuitive way, emphasizing the most important factors. The variations of the presented data and the quantity are an essential factor in utilizing big data techniques.

To understand the size and the volume of the data that are processed and produced, let's first assume that MRI, CT, Ultrasound and other recording used for model parametrization are roughly 2GB and approximately over 500x100 entries. 3D Motion capture and external measurements, can be at least 200M B for each recorded trial with the number of records depending on devices' frequency and the duration of the activity, typically 3000x50 entries. A single simulation can produce around 200M B with at least over 300 different variables and more than 10000 records, depending on the model and the hypothesis under study. So roughly speaking the volume of data is approximately

O(p • (2000 + 200 • n + 200 • m)) MB,
where p is the number of patients, n is the number of recorded trials and m is the number of simulation executions with different settings. As for the number of entries that must be processed O(p • (50000 + 150000 • n + 3000000 • m)) entries. Of course the above analysis can vary with respect to the model, the available data and the simulation resolution (e.g. microscopic of macroscopic).

Currently, we are employing visual analytics metaphors, such as scatter plots, 2D or 3D signal representations, and different visualization techniques for the assessment and the interpretation of the simulation results. Robust decision support systems, based on machine learning algorithms and statistical methods can be applied to discover different criteria for classifying what is the best treatment strategy per individual. Many different models exist with a variety of experimental recordings and simulation results, but unfortunately we lack the ability to organize and share them [START_REF] Hicks | Is my model good enough? Best practices for verification and validation of musculoskeletal models and simulations of human movement[END_REF]. The correct assessment of patient specific disease can be improved if there are available data sets ("Golden Standards") for the modelling and simulation tools to be enhanced and used for the improvement of health care [START_REF] Fregly | Grand challenge competition to predict in vivo knee loads[END_REF].

Neuromusculoskeletal Dynamics

Multibody mechanics is the field studying the classical mechanical properties, especially motion, of systems of bodies interconnected by joints, influenced by forces, and restricted by constraints. The key feature of a system that makes it suitable for multibody treatment is the observation that its motion is localized, that is, it is well-described as a set of independently identifiable parts which undergo large motion with respect to one another, but are themselves rigid or nearly rigid. Multibody mechanics, which has been used effectively to model skeletal motion and gait and many other systems relevant to a wide variety of scientific and engineering disciplines.

Movement in animals and humans is a result of a cascade of neurological and muscle physiological processes that lead to forces on bones that generate reaction forces and accelerate joints. Multibody dynamics plays a key role by providing a physical basis for transforming physiological forces to movement according to Newtons laws of motion. The dominant forces driving the skeletal system are musclulotendinous forces. Muscle forces arise from protein interactions that cause muscle-fibers to contract in response to the electrical state of the musclefiber. The electrical state of a muscle (its activation) is modulated by neural inputs from the central nervous system (CNS) also known as muscle excitations. We are interested in understanding the control of musculoskeletal dynamics to produce coordinated movement.

The musculoskeletal model is composed of the skeletal and the muscular dynamics. The skeletal dynamics can be successfully described from the equations of multibody theory, by modeling the degrees of freedom (DOFs) and the mass properties of the bony segments [START_REF] Pandy | Computer Modeling and Simulation of Human Movement[END_REF]. The equation of motion is defined as:

M(q) • q = F a + V(q, q) + G(q) + F(q, q) (1)
where M ∈ R N xN is the mass matrix of the system, N are the number of DOFs, {q, q, q} ∈ R N are the general coordinates position, velocity and acceleration respectively, F a ∈ R N are the general forces actuating the system, V ∈ R N are the centrifugal and Coriolis forces, G ∈ R N are the internal gravitational forces and F ∈ R N are the external forces.

Muscles are the main actuators of the body, their characteristics are highly non-linear, with many parameters because of the variety of the presented anatomical types. The muscle can be described by its activation and contraction dynamics [START_REF] Millard | Flexing Computational Muscle: Modeling and Simulation of Musculotendon Dynamics[END_REF]. The activation dynamics relate the calcium ion release from the sarcoplasmic reticulum with the firing of motor units which innervate the muscle. As for the contraction dynamics, they relate the force production in accordance with the state of the muscle (length, velocity). The muscle dynamics can be expressed by the following equations: ȧ = f (u, a, t act , t dact )

(2)

f M = g(a, l M , v M , F max , θ) (3)
where, u ∈ [0, 1] is the excitation of the muscle, a ∈ [0, 1] is the activation level, t act , t dact are activation and deactivation time constants, f M is the force produced by the muscle, which in terms is a function of the activation level, the muscle length l M , the muscle velocity v M , the maximum muscle force F max and other parameters θ depending on the muscle's type.

In order for the muscle forces to be coupled into the equation of motion (1), a description of the geometrical insertion sites of the muscles on the skeleton and their joint of action must be specified [START_REF] Erdemir | Model-based estimation of muscle forces exerted during movements[END_REF]. The equation of motion is transformed into the following form (4) by substitution of F a :

M(q) • q = R(q) • F M + V(q, q) + G(q) + F(q, q) (4)
where, R ∈ R N xM is the moment arm matrix, which relates the muscle forces F M ∈ R M to the acceleration of the degrees of freedom. It should be emphasized that based on the equation (4) a muscle can contribute to the acceleration of multiple joints [START_REF] Pandy | Computer Modeling and Simulation of Human Movement[END_REF].

Forward Simulation Pipeline

Forward dynamics simulation is a valuable scheme for estimating the impact of different neural commands to the motor behavior [START_REF] Buchanan | Neuromusculoskeletal Modeling: Estimation of Muscle Forces and Joint Moments and Movements From Measurements of Neural Command[END_REF]. During forward simulation the analysis begins with the cause and ends with the effect which is a motion trajectory Figure 2. The cause of the resultant motion can be a neural command orchestrating a group of muscles, activating them, thus producing forces which accelerate the joints of the model. The aforementioned process can be described mathematically by the equation of motion ( 4) which was derived previously, so accurate predictions can be made by solving for the general coordinates q.

q = M -1 (q) • {R(q) • F M + V(q, q) + G(q) + F(q, q)} ( 5)

The equation ( 5) can be numerically integrated twice and the coordinates of the system are predicted as a consequence of the forces F M generated by the neural command.

Fig. 2. The forward simulation pipeline where the cause is a neural excitation command to the muscles and the effect is an observed motion trajectory.

Inverse Simulation Pipeline

In contrast with the forward, the inverse simulation begins with the effect, which usually is a recorded motion Figure 3. The goal of these methods is to estimate the internal forces, muscle activations or neural commands that govern the musculoskeletal model to reproduce the recorded motion trajectories, since these signals are typically difficult to be measured or observed.

Usually, the analysis begins with a general model which is personalized and scaled accordingly to the experimental subject. Along with the experimentally measured kinematics the Inverse Kinematics (IK) method is used to translate the recorded motion to model space coordinates (typically joint angles). IK can be expressed as an optimization problem (6) of finding the general coordinates q of the model, so that for every instance of time the position of the virtual markers p(q j ) placed on the model to be as close as possible to the experimentally measured landmarks p j , subject to the mobility constraints b j of the joints. minimize q N j=1 w j • p j -p(q j ) 2 s.t. c j (q j ) ≤ b j , i = 1, . . . , N.

(6)

At this point, is worth mentioning that Inverse Dynamics (ID) can be used to estimate the joints' general forces using the result of the IK along with the experimentally measured external forces (e.g. ground reaction forces during gait activity) by solving the equation of motion (1) for F a . Alternatively, the muscles' neural excitations can be assessed by using different methods such as Computed Muscle Controls (CMC) [START_REF] Thelen | Using computed muscle control to generate forward dynamic simulations of human walking from experimental data[END_REF]. CMC solves the problem by tracking the experimental kinematics using a PD controller in a forward dynamic manner and an optimization scheme such as Static Optimization (SO) for estimating the redundant muscle forces [START_REF] Anderson | Static and dynamic optimization solutions for gait are practically equivalent[END_REF]. Figure 3 presents a method called Residual Reduction Algorithm (RRA), which can improve the results' quality by reducing the overall error introduced by noisy measurements, model assumptions and inconsistencies [START_REF] Delp | D.: OpenSim: Open-Source Software to Create and Analyze Dynamic Simulations of Movement[END_REF].

Visual Analytics

A simulation engine has been constructed for generating the motion of a 3D virtual upper limb arm. The motion characteristics are derived from the proposed neuromusculoskeletal methodology. The simulation engine receives as input gyroscopic data measured from real users and computes the activation signals of the respective muscles. The graphical user interface of the simulation engine is depicted in Figure 4. The tool can handle multimodal data information, such as gyroscopic, accelerometer measurements, forces, activation signals and simulation states. The computed data timelines are fully interactive and bound to the 3D model, so any user interaction with them is automatically transformed into a resultant motion of the model, thus allowing the researcher to better understand the motion factor characteristics. This tool has been created for studying the upper limb kinematics and dynamics of subjects with Parkinson's disease. An upper limb model Figure 5 of the right arm was developed based on the previous work of [START_REF] Holzbaur | A model of the upper extremity for simulating musculoskeletal surgery and analyzing neuromuscular control[END_REF][START_REF] Saul | Computer methods in biomechanics and biomedical engineering[END_REF] and is adjusted accordingly for the specific needs. The initial model has 15 degrees of freedom, but during simulations some of the DOFs were disabled in order to reduce the redundancy. The muscles were also reduced to 24 to simplify the number of control units (actuators) that the brain has to coordinate. It should be emphasized that the model is purposely simplified in order to study the main characteristics of coupled simulation between brain, peripheral nervous and the musculoskeletal systems for simple motion trajectories.

Preliminary Results

The potential of the proposed framework is examined by producing some preliminary results by employing inverse simulation techniques. More specifically, Figure 6 presents a comparison of the activation signals to the muscle for two subjects that were instructed to perform similar motion. First the recorded motion trajectory was translated to model space using IK, and then CMC was utilized for the assessment of the muscle activations. Figure 7 presents a comparison between the muscular force of the triceps for the two subjects. The results do not emphasise the usage of big data techniques, but if additional experimental trials were presented with more muscles and different motion trajectories the quantity of data will be significantly higher. Thus new methods must be applied for the assessment of qualitative results that can be used in decision support systems, such as machine learning or statistical analysis.

Conclusion

Due to the increased needs for patient specific health care, ICT approaches are expected to be used as a clinical decision support tool. Neuromusculoskeletal modelling and simulation has shown promising results in both accuracy and qualitative assessment of different observations that are hard to be measured or observed. The proposed approach integrates different simulation strategies in a unique medical decision support tool that also uses successful visual analytics metaphors to provide all the necessary information to the clinician in an intuitive manner. The proposed framework will be further developed so as to include brain computational models that will drive the musculoskeletal simulation. Thus, brain pathology will be directly linked to motor behaviour and vice versa.

Fig. 3 .

 3 Fig. 3. The inverse simulation pipeline for computing the muscle excitation (IK: Inverse Kinematics, RRA: Residual Reduction Algorithm, CMC: Computed Muscle Controls).

Fig. 4 .

 4 Fig. 4. Graphical user interface of the musculoskeletal upper limb model simulation. The user can study the motion by interacting with interface components. Muscle colouring is applied dynamically based on activation signal profiles.

Fig. 5 .

 5 Fig. 5. The developed right arm model with 15 DOFs and 24 muscles used during simulation.

Fig. 6 .

 6 Fig. 6. Comparing the muscle activations between two subjects for the same motion. Each figure depicts the activation signals for seven arm muscles. The user may interact with the signals and apply the motion state to the virtual humanoid.

Fig. 7 .

 7 Fig. 7. Comparing the forces generated by the same muscle (triceps) of two subjects; although the two subjects were instructed to perform the same motion.

Acknowledgement Partial funding for this work was provided by the EU co-funded project NoTremor: Virtual, Physiological and Computational Neuromuscular Models for the Predictive Treatment of Parkinson's Disease (EC FP7 Grant agreement no 610391).