R Martín-Vázquez

R Aler

EVANNAI I M Galván

Wind Energy Forecasting at Different Time Horizons with Individual and Global Models

Keywords: Wind Power Forecasting, Machine Learning, Forecasting Horizons

In this work two different machine learning approaches have been studied to predict wind power for different time horizons: individual and global models. The individual approach constructs a model for each horizon while the global approach obtains a single model that can be used for all horizons. Both approaches have advantages and disadvantages. Each individual model is trained with data pertaining to a single horizon, thus it can be specific for that horizon, but can use fewer data for training than the global model, which is constructed with data belonging to all horizons. Support Vector Machines have been used for constructing the individual and global models. This study has been tested on energy production data obtained from the Sotavento wind farm and meteorological data from the European Centre for Medium-Range Weather Forecasts, for a 5x5 grid around Sotavento. Also, given the large amount of variables involved, a feature selection algorithm (Sequential Forward Selection) has been used in order to improve the performance of the models. Experimental results show that the global model is more accurate than the individual ones, specially when feature selection is used.

Introduction

The correct forecast of the energy obtained from the wind is still one of the main challenges of renewable energies. To predict the wind power generated in a wind farm is a difficult task although there is a wide literature on this area. Currently, to forecast the power there are mainly three approaches: physical methods, statistical methods and methods based on artificial intelligence [START_REF] Tascikaraoglu | A review of combined approaches for prediction of short-term wind speed and power[END_REF][START_REF] Jung | Current status and future advances for wind speed and power forecasting[END_REF].

Physical methods collect values measured in the lower atmosphere to build mathematical models, named Numerical Weather Prediction (NWP), that are used to make predictions of the weather [START_REF] Roulston | Using medium-range weather forcasts to improve the value of wind energy production[END_REF]. These methods have the drawback that they require long operation time and a large amount of computational resources.

Statistical methods use historical data from wind farms to find a relationship between the input variables and the power. These models are faster than physical meth-ods but they have the disadvantage that they have large error for long horizons. Time series [START_REF] Croonenbroeck | A selection of time series models for short-to medium-term wind power forecasting[END_REF], Autoregressive Moving Average (ARMA) [START_REF] Liu | Comprehensive evaluation of ARMA-GARCH(-M) approaches for modeling the mean and volatility of wind speed[END_REF], Autoregressive Integrated Moving Average (ARIMA) [START_REF] Kavasseri | Day-ahead wind speed forecasting using f-ARIMA models[END_REF] or Modified Taylor Kriging (MTK) [START_REF] Liu | Prediction of wind speed time series using modified Taylor Kriging method[END_REF], are examples of these techniques.

Artificial intelligence methods use mainly, but not exclusively, machine learning techniques. They use information from NWP variables and/or historical data to predict power wind, trying to find the relationship between these variables. Some of these artificial intelligence techniques are Fuzzy Logic [START_REF] Zhu | A prediction model for wind farm power generation based on fuzzy modeling[END_REF], Genetic Algorithms [START_REF] Damousis | A Fuzzy Model for Wind Speed Prediction and Power Generation in Wind Parks Using Spatial Correlation[END_REF], Artificial Neural Networks (ANN) [START_REF] Mohandes | Support vector machines for wind speed prediction[END_REF], Support Vector Machines (SVM) [START_REF] Heinermann | Precise Wind Power Prediction with SVM Ensemble Regression[END_REF] or ensemble methods [START_REF] Ren | Ensemble methods for wind and solar power forecasting-A state-of-the-art review[END_REF]. Some of these approaches can be hybridized. The growing popularity of renewable energies has led to the search for new alternatives by combining existing methods to try to make more reliable predictions. Time series and boosting with historical data [START_REF] Jiang | Short-term wind power forecasting using hybrid method based on enhanced boosting algorithm[END_REF], ANN and time series with wind speed [START_REF] Doucoure | Time series prediction using artificial wavelet neural network and multi-resolution analysis: Application to wind speed data[END_REF] and Gaussian Process with NWP and historical data [START_REF] Chen | Wind Power Forecasts Using Gaussian Processes and Numerical Weather Prediction[END_REF] are some of the combinations that can be found.

In this work, machine learning methods using Support Vector Machine models (SVM) are proposed to predict wind power at different time horizons (or steps), from 3 hours until 15 hours, using meteorological variables (from a NWP model) in a grid centered at the Sotavento wind farm. With this purpose two approaches has been studied: individual models and global models. Individual models are made for working in a single time horizon, building up a different model for each horizon. The motivation for using individual models is to train them under the same situation that they will be used operationally. When used operationally, the only inputs available to the model are the meteorological variables forecasts. Therefore, for training the individual models, forecasts carried out in the past (historical forecasts) will also be used. This has the advantage that both training and operation use the same kind of data, but it has two drawbacks. First, forecasts contain prediction errors that might affect negatively the training process. Second, each individual model can only be trained with data belonging to its time horizon. Thus, a second approach is proposed, the global model, where a single model is trained using the data from all horizons put together. In this case, the model can be trained using reanalysis data, that contains observations / measurements, rather than forecast records. In operation mode, the same global model can be used for making predictions at any horizon.

On the other hand, using a grid of meteorological variables implies that the number of attributes can be very large, which can reduce the model generalization capability, specially for the individual models, which are trained with fewer data. In a previous work [START_REF] Martín-Vázquez | A Study on Feature Selection Methods for Wind Energy Prediction[END_REF] three different attribute selection methods were evaluated to estimate wind energy at Sotavento. It was concluded that Sequential Forward Selection (SFS) reduced significantly the number of attributes while obtaining very accurate results. In this work, SFS has also been used for both the individual and global models. In the case of the individual models, different attributes can be selected for different horizons.

The rest of the paper is organized as follows. In section 2 data sets are explained. Section 3 describes the individual and global approaches that are studied in the present work. Experimental results are presented in section 4. Finally, section 5 draws the conclusions learned from this study.

Data

As in the previous work [START_REF] Martín-Vázquez | A Study on Feature Selection Methods for Wind Energy Prediction[END_REF], data is obtained from a 5x5 grid, with a distance between coordinates of 0.125 degrees, centered at Sotavento experimental wind farm between 2005 and 2010. 19 meteorological variables are selected for each grid point, and they have been obtained from the ERA-20C dataset made available by the European Centre for Medium-Range Weather Forecasts (ECMWF) 1 .

This results in a total of 475 attributes (19*5*5). The 19 meteorological variables are shown in table 1. For the same period, there is a reanalysis data set and a forecasting data set. As its name indicates, forecasting datasets contain forecasts, while reanalysis datasets contain information obtained to observations. The forecast datasets provide predictions at different steps (or horizons). All the forecasts are made every day at 06:00UTC and predictions are provided for steps 3, 6, 9, 12, and 15 hours ahead. The reanalysis dataset contains a time series for each meteorological variable, with observations provided at 06:00UTC+3h, 06:00UTC+6h … 06:00UTC+15h. Generated wind power has been obtained from the Sotavento website2 . Table 2 displays the number of instances for each data set.

Individual and global approaches for energy forecasting

In this work, two different approaches are used for predicting wind energy: individual models and a global model. Individual models are built to be used for specific time horizons while the global model can be used for any time horizon. Next, the process used for training and using the individual and global models will be described:

Individual models follow Eq. 1, where t 0 is the time at which the predictions are made (06:00UTC), h = 3, 6, 9, 12, 15 are the steps, and v i (t 0 + h) are the forecasted meteorological variables. Each F h model is trained using the data for that particular step from the forecast dataset (see table 2, so that every individual model is specialized in a particular step.

𝐸𝐸 ℎ (0) (𝑡𝑡 0 + ℎ) = 𝐹𝐹 ℎ (𝑣𝑣 1 �(𝑡𝑡 0 + ℎ), … , 𝑣𝑣 𝑛𝑛 �(𝑡𝑡 0 + ℎ)) (1)
In order to study the influence on the forecast of the energy measured at prediction time t 0 , a new input variable is added (see Eq. 2) with the energy value at 06:00 UTC (E(t 0 )).

𝐸𝐸 ℎ (1) (𝑡𝑡 0 + ℎ) = 𝐹𝐹 ℎ (𝑣𝑣 1 �(𝑡𝑡 0 + ℎ), … , 𝑣𝑣 𝑛𝑛 �(𝑡𝑡 0 + ℎ), 𝐸𝐸(𝑡𝑡 0 )) (2)
The global model is trained using reanalysis data, according to Eq. 3, where v i (t) are the reanalysis meteorological variables. In this case, the data size is larger than the ones used with individual models (see table 2).

𝐸𝐸 (0) (𝑡𝑡) = 𝐹𝐹(𝑣𝑣 1 (𝑡𝑡), … , 𝑣𝑣 𝑛𝑛 (𝑡𝑡)) (3)
Eq. 4 shows how to use the global model for making predictions at time t 0 for different horizons (h = 3, 6, 9, 12, 15). In this case, forecasts 𝑣𝑣 𝚤𝚤 �(𝑡𝑡 0 + ℎ) must be used because observations are not available at prediction time. It is important to remark that, differently to individual models, the same global model is used for all steps.

𝐸𝐸 (0) (𝑡𝑡 0 + ℎ) = 𝐹𝐹(𝑣𝑣 1 �(𝑡𝑡 0 + ℎ), … , 𝑣𝑣 𝑛𝑛 �(𝑡𝑡 0 + ℎ)) (4)
Similarly to individual models, the influence of wind energy measured at prediction time (E(t 0 )) is studied, with models trained according to Eq. 5.

𝐸𝐸 (1) (𝑡𝑡) = 𝐹𝐹(𝑣𝑣 1 (𝑡𝑡), … , 𝑣𝑣 𝑛𝑛 (𝑡𝑡), 𝐸𝐸(𝑡𝑡 0 ))

Similarly to model E (0) , in order to use E (1) for making predictions, Eq. 6 must be used.

𝐸𝐸 (1)

(𝑡𝑡 0 + ℎ) = 𝐹𝐹(𝑣𝑣 1 �(𝑡𝑡 0 + ℎ), … , 𝑣𝑣 𝑛𝑛 �(𝑡𝑡 0 + ℎ), 𝐸𝐸(𝑡𝑡 0 )) (6)

Experimental Results

In this section the results obtained in the experiments are shown. To measure the performance the Mean Absolute Error (MAE) is used. A methodology similar to cross validation has been used to evaluate both individual and global models. Given that data is available for six years (2005 to 2010), each year is used as a test set, and the remaining five years are used to construct the model. That means that the process of model construction and model evaluation has been repeated six times (6 folds) and then the average number of inputs and test MAE have been obtained. Data reserved for model construction is further divided into training and validation datasets by choosing, every three days, the first two days for training and the third one for validation. The validation partition will be used for hyper-parameter tuning and to select the most relevant attributes with the SFS algorithm. The final model is trained joining the training and validation data, using the hyper-parameters and attributes selected previously. This model is finally evaluated on the test set.

SVM hyper-parameter tuning has been carried out by means of grid search, for Gamma={0, 0.01, 0.1, 0.1, 0.25, 0.5, 0.75, 1} and C={0.1, 1, 10, 100, 500}. Grid search returned the same parameter values for all folds: C=1 and Gamma=0.01 for individuals and global models.

Because of the large number of input attributes, and taking into account the results of the previous work, SFS has been used to reduce the number of input features. In the case of the global model, the 26 attributes selected are the same as in the previous work, where also the Sotavento reanalysis dataset was used [START_REF] Martín-Vázquez | A Study on Feature Selection Methods for Wind Energy Prediction[END_REF]. In the case of the individual models, SFS was run for each of the steps, so that the attributes selected are specialized for every step.

The average results of the 6 folds are shown in table 3. The first half of the table displays results for the individual models and the second half for the global one. For each of them, the MAE is provided for the model without feature selection (second column), with SFS selection (third column), and with SFS selection plus the E(t 0 ) input (last column). All results have been broken down by step (first column), but average and standard deviation results are also included. When SFS selection is used, the average number of inputs selected is also given. With respect to the individual models, it can be seen that using SFS (third column) reduces significantly the number of inputs (from 475 to 8.54 on average). MAE is also improved for all steps except the last one. Adding E(t 0 ) as input to the model (𝐸𝐸 ℎ (1) in the fourth column) does not improve results, except for the first step (3h) (see Fig. 1). For the global model, SFS selection improves MAE for all steps and adding E(t 0 ) as input decreases MAE for the two first steps (see Fig. 2). This should be expected, because the longer the horizon, the less relevant is the power measured at prediction time t 0 . Comparing the individual and global models, it can be seen in table 3 that the global models outperform the individual ones on average in the three cases studied: without feature selection, with SFS, and with SFS plus E(t 0 ). In the first case, the improvement of the global model is small (277.12 vs. 276.28), but the difference increases for the other two cases (274.07 vs. 256.50 and 295.83 vs. 257.72, respectively). In fact, for the last two cases, the global model is more accurate than the individual ones, not only on average, but for each step.

In order to assess the significance of the comparisons between global and individual models, the number of folds where each model is better will be given. E (0) global model is better in 5 out of 6 folds for every step than individual models. For E (1) models with SFS variables, results are better for global model than individual models in step 3 for 5 folds. In the following steps, the global model is better than the individual models in all of the folds.

In order to provide some baseline results, the persistence method has also been used. Persistence uses the current value of wind power as forecast for the next time step. The 6-fold average MAE obtained for each time horizon can be seen in table 4. It can be observed that the different approaches studied in this work are better than persistence for all the steps.

Conclusions

Two different approaches, individual and global, for predicting wind power in different time horizons have been studied and compared. Individual models are specific for each step. Therefore, they are trained with data belonging to the step and with forecasting dataset. The global model is trained with data from all steps and using the reanalysis dataset. This results in a unique model that can be used for making predictions in the different steps.

Although it could be expected that individual models performed better because they are specific for each horizon, the fact that fewer data is used to train each of them, and the need to use the forecasted input meteorological variables, rather than the reanalysis dataset, makes the global model perform more accurately, at least in this study.

Regarding the use of the SFS algorithm for feature selection, results show that both individual and global improve accuracy, but it is the global approach which specially takes advantage from it. Finally, adding the wind power measurement available at prediction time as input to the models, benefits mainly the global model and only the forecasting horizons close to prediction time. The improvement for short time horizons (steps 3 and 6) should be expected, because long time horizons depend much less on observations at prediction time.

Fig. 1 .

 1 Fig. 1. Individual Models with/without 𝐸𝐸(𝑡𝑡 0 )

Fig. 2 .

 2 Fig. 2. Global Model with/without 𝐸𝐸(𝑡𝑡 0 )

Table 1 .

 1 Meteorological variables used as input to the models.

	Variables

Table 2 .

 2 Number of instances for each data set.

	Data Set	Instances
	Forecast step 3 1445
	Forecast step 6 1446
	Forecast step 9 1543
	Forecast step 12 1550
	Forecast step 15 1468
	Reanalysis	8347

Table 3 .

 3 Results for the individual global approaches.

			Individual Models
		𝐸𝐸 ℎ (0) , 475 Inputs	𝐸𝐸 ℎ (0) , SFS Inputs	𝐸𝐸 ℎ (1) , SFS Inputs
	Step	MAE	MAE	Number Inputs	MAE
	Step 3	266.15	261.34	7.33	246.79
	Step 6	291.42	272.00	11.17	273.34
	Step 9	280.06	275.32	11.17	306.81
	Step 12	259.32	257.5	6.67	295.41
	Step 15	288.64	303.21	6.00	356.82
	Average	277.12	274.07	8.47	295.83
	Std	13.99	17.94	2.51	41.08
			Global Models
		𝐸𝐸 (0) , 475 Inputs	𝐸𝐸 (0) , SFS Inputs	𝐸𝐸 (1) , SFS Inputs
	Step	MAE	MAE	Number Inputs	MAE
	Step 3	268.86	256.61	26	238.80
	Step 6	282.88	259.74	26	251.30
	Step 9	273.42	247.71	26	252.46
	Step 12	260.53	237.13	26	242.60
	Step 15	295.72	281.33	26	303.43
	Average	276.28	256.50	26	257.72
	Std	13.54	16.43	0	26.20

Table 4 .

 4 Results for the persistence method.

	Step	MAE
	Step 3	305.85
	Step 6	395.20
	Step 9	427.57
	Step 12 450.34
	Step 15 494.20
	Average 414.63
	Std	25.8

https://www.ecmwf.int/en/forecasts/datasets/reanalysis-datasets/era-20c

http://www.sotaventogalicia.com/en/real-time-data/historical

Acknowledgements

The authors acknowledge financial support granted by the Spanish Ministry of Science under contract ENE2014-56126-C2-2-R.


Chao Jing

Ping Wei
email: pingwei@xjtu.edu.cn

Hongbin Sun

Ningnan Zheng

Spatial-Temporal Neural Networks for Action Recognition

Keywords: Action Recognition, Spatial-Temporal Structure, LSTM

Action recognition is an important yet challenging problem in many applications. Recently, neural network and deep learning approaches have been widely applied to action recognition and yielded impressive results. In this paper, we present a spatial-temporal neural network model to recognize human actions in videos. This network is composed of two connected structures. A two-stream-based network extracts appearance and optical flow features from video frames. This network characterizes spatial information of human actions in videos. A group of LSTM structures following the spatial network describe the temporal information of human actions. We test our model with data from two public datasets and the experimental results show that our method improves the action recognition accuracy compared to the baseline methods.

Introduction

Action recognition is to predict an action category label for an input video. It is an important problem in many applications, such as video search, security surveillance, and human-machine interaction.

Recognizing actions in daily-activity videos is a challenging problem. First, some different action categories have similar appearance and motion features. For example, the actions drinking and eating are very similar in motion features. Second, motion noise in videos increases the difficulty of action recognition. Third, the unrelated background or scene features often make the model unable to capture the key information of action recognition.

In this paper, we present a spatial-temporal neural network model to recognize human actions in videos. This network is composed of two connected structures -the spatial structure and the temporal structure, as shown in Figure 1. The spatial structure is a Two-Stream Network [START_REF] Simonyan | Two-stream convolutional networks for action recognition in videos[END_REF], which extracts appearance and optical flow features from video frames. Following the spatial network, the temporal structure is a group of LSTM networks [START_REF] Hochreiter | Long short-term memory[END_REF] which represent the temporal and transition information of human actions. With these two structures, our model can deeply mine and utilize the spatial and temporal features in videos for action recognition. We test our model with data from two challenging datasets -MSR DailyActivity 3D [START_REF] Wang | Learning actionlet ensemble for 3d human action recognition[END_REF] and UCF101 [START_REF] Soomro | Ucf101: A dataset of 101 human actions classes from videos in the wild[END_REF]. The experimental results show that our method improves the action recognition performance compared to other baseline methods.

Related Work

Traditional action recognition methods generally consist of two key parts: feature extraction and feature classification. For feature extraction, most approaches are based on appearance, geometric, or motion features of human bodies, such as skeleton features [START_REF] Fujiyoshi | Real-time human motion analysis by image skeletonization[END_REF][START_REF] Wei | Concurrent action detection with structural prediction[END_REF], optical flow features [START_REF] Chaudhry | Histograms of oriented optical flow and binet-cauchy kernels on nonlinear dynamical systems for the recognition of human actions[END_REF]. These methods extract features from human bodies and can achieve satisfactory results in most scenes. However, in complex scenes with cluttered backgrounds, it is difficult to compute the accurate positions of human body parts, the action recognition accuracy is drastically depressed. Similar to HOG (Histogram of oriented gradient) [START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF] and SIFT (Scale-invariant feature transform) [START_REF] Lowe | Object recognition from local scale-invariant features[END_REF] in images, multi-scale feature extraction algorithms with prior knowledge were proposed. For example, some approaches extracted action features around the spatial-temporal interest point [START_REF] Sch | Recognizing human actions: A local svm approach[END_REF][START_REF] Wang | Action recognition by dense trajectories[END_REF][START_REF] Klӓser | A spatio-temporal descriptor based on 3dgradients[END_REF]. In complex scenes or backgrounds, such kinds of methods have achieved impressive improvements in action recognition accuracy. With the extracted features, various classifiers are learned to recognize actions, such as Support Vector Machine (SVM) [START_REF] Cortes | Support-vector networks[END_REF]. Recently, neural networks and deep learning techniques [START_REF] Schmidhuber | Deep learning in neural networks: an overview[END_REF][START_REF] Goodfellow | Deep Learning[END_REF][START_REF] Lecun | Deep learning[END_REF] have been widely used in action recognition and achieved impressive performance [START_REF] Ji | 3d convolutional neural networks for human action recognition[END_REF][START_REF] Chen | Deep manifold learning combined with convolutional neural networks for action recognition[END_REF][START_REF] Li | End-to-end learning of deep convolutional neural network for 3d human action recognition[END_REF][START_REF] Rahmani | Learning a deep model for human action recognition from novel viewpoints[END_REF][START_REF] Husain | Action recognition based on efficient deep feature learning in the spatio-temporal domain[END_REF][21]. Compared with static image classification, the temporal components of videos provide additional and important recognition clues -motion information [START_REF] Simonyan | Two-stream convolutional networks for action recognition in videos[END_REF][START_REF] Papenberg | Highly accurate optic flow computation with theoretically justified warping[END_REF]. In the early stage, action recognition based on single CNNs model was adopted [START_REF] Ji | 3d convolutional neural networks for human action recognition[END_REF]. Although this method improves action recognition performance compared with traditional methods, the characteristics of time series was not deeply processed. Later, two-stream networks [START_REF] Simonyan | Two-stream convolutional networks for action recognition in videos[END_REF] which utilize appearance and optical flow CNNs have significantly improved action recognition performance compared with the single CNNs model. After that, the Long Short-Term Memory (LSTM) models [START_REF] Hochreiter | Long short-term memory[END_REF] and other Recurrent Neural Network (RNN) models are applied to action recognition tasks [START_REF] Donahue | Long-term recurrent convolutional networks for visual recognition and description[END_REF][START_REF] Baccouche | Sequential deep learning for human action recognition[END_REF][START_REF] Ng | Beyond short snippets: deep networks for video classification[END_REF]. LSTM and RNN models incorporate the temporal information of videos into spatial features and therefore remarkably improve the recognition accuracy compared with previous neural network architectures.

Inspired by those models, our spatial-temporal network model is a hybrid architecture of the two-stream network [START_REF] Simonyan | Two-stream convolutional networks for action recognition in videos[END_REF] and the LSTM network [START_REF] Hochreiter | Long short-term memory[END_REF][START_REF] Donahue | Long-term recurrent convolutional networks for visual recognition and description[END_REF][START_REF] Baccouche | Sequential deep learning for human action recognition[END_REF]. From data preprocessing to network structures, it extracts local and global features, combines multifeature learning, and is consistent with the sequential-data-based action recognition.

Spatial-Temporal Neural Network Model

A video frame containing appearance and geometric information of human actions is the smallest feature unit of the video sequence [START_REF] Donahue | Long-term recurrent convolutional networks for visual recognition and description[END_REF][START_REF] Baccouche | Sequential deep learning for human action recognition[END_REF][START_REF] Ng | Beyond short snippets: deep networks for video classification[END_REF][START_REF] Graves | Supervised sequence labelling with recurrent neural networks[END_REF][START_REF] Barbu | Video in sentences out[END_REF]. The temporal and motion information between successive frames is also essential for distinguishing different actions. Inspired by the previous convolutional network and LSTM methods [1, 23-25, 29], we present a spatial-temporal neural network model to jointly describe the spatial information in single frames and the temporal information between successive frames, as shown in Figure 1. This network is composed of two connected structures -the spatial structure and the temporal structure, as shown in Figure 1. The spatial structure is a two-stream network [START_REF] Simonyan | Two-stream convolutional networks for action recognition in videos[END_REF], which extracts appearance and optical flow features from video frames. Following the spatial network, the temporal structure is a group of LSTM networks [START_REF] Baccouche | Sequential deep learning for human action recognition[END_REF] which describe the temporal and motion information of human actions. With these two structures, our model can deeply mine and utilize the spatial and temporal features in videos for action recognition.

For a complete video, we first carry out the frame pre-processing (Section 3). One stream of the spatial structure extracts RGB features and another stream extracts optical flow features. The main structures of each stream are 6 convolutional layers and two dense layers [START_REF] Yuan | Feature extraction and image retrieval based on alexnet[END_REF]. The size of each channel input to the first convolutional layer (conv1) is 227*227*8. Through the feature extraction of the convolutional layer (conv1 to conv6), the dimension of our FC1 and FC2 layers is 4096. We use two layers of dense layer to prevent over fitting. Through this part, we obtain the feature in Fe+ layer with dimension of 4096*2L.

The temporal structure is a sequential 6 layers LSTM network. By dynamically inputting the obtained Fe+ features into the sequence learning module, it can learn the temporal feature of the video sequence.

Data Processing

Data preprocessing is the process to convert the original video sequence into the actual input data of the network model. In this section, we will address the problem of multi-frame inputs and introduce how to calculate multi-frame optical flows.

RGB Multi Frame Sequence Input Processing

We adopt a pre-processing method to take into account the feature expression of single frame RGB data and multi-frame RGB data. The basic idea is that for a complete video sequence, we take into account the connectivity of a video segment when we divide video into multiple segments. In the process, every 8 frames are used as a unit fragment. A sliding window is defined with 4 frames per step. It slides from the video start to the end. This process method is shown in algorithm 1.

Algorithm 1. Multi-frame preprocessing algorithm.

Multi-frame preprocessing algorithm.

Multi Optical Flow Calculation

In this section, we introduce how to calculate optical flow features in videos. Optical flow is an important feature of videos and it is widely used in the task of action recognition [START_REF] Papenberg | Highly accurate optic flow computation with theoretically justified warping[END_REF][START_REF] Baker | A database and evaluation methodology for optical flow[END_REF][START_REF] Simonyan | Two-stream convolutional networks for action recognition in videos[END_REF]. The optical flow contains the motion information of targets and is used to describes the video frame changes. In video sequence data, the instantaneous speeds of pixels can be used to characterize the correlations between pixel sequences in time domain, as shown in Figure 2.

We adopt the similar method with two-stream network [START_REF] Simonyan | Two-stream convolutional networks for action recognition in videos[END_REF] to calculate the optical flow featues in videos. For a video segment with L frames, we extract the optical flow information along the X and Y axes in each two adjacent frames. Then the optical flow feature of the video segment is an encapsulation of all the frame optical flow features. It is a vector with a dimension of w*h*2L [START_REF] Simonyan | Two-stream convolutional networks for action recognition in videos[END_REF], where w*h is the dimension of the single optical flow.

Experiments

We use the action recognition accuracy to evaluate the performance of different methods. The action recognition accuracy is defined as the ratio of correctly labeled video numbers to all testing video numbers. We train the model under the caffe framework [START_REF] Jia | Caffe: Convolutional architecture for fast feature embedding[END_REF] and use the hardware CUDA plus GPU to deal with the floating-point matrix operation of the network. We test the models on the MSR DailyActivity 3D dataset [START_REF] Wang | Learning actionlet ensemble for 3d human action recognition[END_REF] and the data samples from the UCF101 datase [START_REF] Soomro | Ucf101: A dataset of 101 human actions classes from videos in the wild[END_REF].

For the convolutional network component, we use video frames and optical flows to fine-tune a pre-trained AlexNet model [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF]. We set the learning batch size as 32. The learning rate starts at 0.001 and is divided by 10 after every 30k iterations. For all experimental settings, we set the dropout regularization ratio as 0.5 to reduce complex co-adaptations of neurons in nets.

For the LSTM part, the output of Fe+ is used as the input to the LSTM. The momentum and weight decay are set as 0.9 and 0.0005, respectively. The learning rate starts at 0.01 and is divided by 10 after every 30k iterations. The output dimension of the softmax layer is 16.

Action Recognition on MSR DailyActivity 3D Dataset

The MSR DailyActivity 3D dataset [START_REF] Wang | Learning actionlet ensemble for 3d human action recognition[END_REF] is captured using a Kinect camera. There are 16 action classes: drinking water, eating, reading, calling, writing on paper, using notebooks, vacuuming, waking up, sitting, throwing paper, playing games, lying on the sofa, walking, playing guitar, standing up, sitting down. There are ten subjects in total and two types of actions in each subject. One type action is at a standing position and one at a sitting position. The depth frames, the 3D skeletons of human bodies, and the RGB frames are recorded. We compare our method with seven other approaches: Dynamic Temporal Warping [START_REF] Müller | Motion templates for automatic classification and retrieval of motion capture data[END_REF], Actionlet Ensemble on Joint Features [START_REF] Wang | Learning actionlet ensemble for 3d human action recognition[END_REF], HDMM+3ConvEets [START_REF] Wang | Deep convolutional neural networks for action recognition using depth map sequences[END_REF], 4DH [START_REF] Wei | Modeling 4d human-object interactions for joint event segmentation, recognition, and object localization[END_REF], 4DHOI [START_REF] Wei | Modeling 4d human-object interactions for joint event segmentation, recognition, and object localization[END_REF], Proposed method with Spatial Structure, and Proposed method with Temporal Structure. Proposed method with Spatial Structure only uses the Two-Stream Network component, and Proposed method with Temporal Structure uses the LSTM component. Table 1shows the overall action recognition accuracy comparison, and Figure 3 (a) shows the accuracy of each action category.

Our method achieves an accuracy of 0.87, which outperforms other baseline approaches. Table 1 also shows that our method outperforms the spatial structure method and temporal structure method by a considerable margin, which proves the effectiveness of joint spatial-temporal network. We compare our approach with the spatial structure method of two-stream network and the temporal structure method of LSTM. Table 2 shows the accuracy comparison and Figure 3 (b) shows the accuracy of each action class. Our method achieves an accuracy of 0.85. The results show that our method outperforms the comparison methods by a large margin, which proves the strength and effectiveness of our method.

Conclusions

This paper presents a spatial-temporal neural network model to recognize human actions in videos. Our model jointly uses temporal and spatial dimension features of video sequences. With spatial and temporal structures, our model can deeply mine and utilize the spatial and temporal features in videos for action recognition. We test our model on two challenging datasets. The experimental results show that our methods improve the performance compared to other baseline methods.

Our future work will focus on complex neural network models on action recognition and video understanding.

Fig. 1 .

 1 Fig. 1. Illustration of our spatial-temporal network model. The left side is a two-stream model and the right side with LSTM describes the temporal information of human actions.

Fig. 2 .

 2 Fig. 2. Illustration of optical flow in human actions. (a) and (b) show a pair of successive video frames with human body motion. (c) shows the motion area.

Fig. 3 .

 3 Fig. 3. Accuracy of each action of MSR DailyActivity 3D and UCF101 recognition on the network.

Table 2 .

 2 Action recognition comparison on MSR DailyActivity 3D Dataset. Each category consists of 25 groups and each group has 4 videos, with a total of 100 videos per category. The videos from the same group may share some common features, such as similar background, similar viewpoint, etc.

	Method

Table 2 .

 2 Action recognition comparison with UCF101 data.

	Method

Acknowledgement

This work is supported by National Natural Science Foundation of China 61503297, National Key Research and Development Program of China 2016YFB1000903, National Natural Science Foundation of China 61790563, and the Open Project Program of State Key Laboratory of Mathematical Engineering and Advanced Computing.


Chao Chen

Gabriel Terejanu
email: terejanu@cse.sc.edu

Sub-Event Detection on Twitter Network

Keywords: Outlier detection, Time-series analysis, Social media mining 1

This work addresses the online detection of sub-events using Twitter stream data. We formulate the process of sub-event identification as an outlier detection problem using three statistical methods: Kalman Filter, Gaussian Process, and Probabilistic Principal Component Analysis. These methods are used to construct the probability distribution of percentage change in the number of tweets. Outliers are identified as future observations that do not fit these predicted probability distributions. Five real-world case studies are investigated to test the effectiveness of the methods. Finally, we discuss the limitations of the proposed framework and provide future directions for improvement.

Introduction

Launched in 2006, Twitter serves as a microblogging platform in which people can publish at most 140 character-long tweets or 10,000 character-long direct messages [START_REF] Meladianos | Degeneracy based real-time sub-event detection in twitter stream[END_REF]. Due to its popularity, portability, and ease of use, Twitter quickly has grown into a platform for people sharing daily life updates, chatting, and recording or spreading news. As of September 2015, Twitter announced that there were more than 320 million monthly active users worldwide1 1 . In comparison to conventional news sources, Twitter favors real-time content and breaking news, and it thus plays an important role as a dynamic information source for individuals, companies, and organizations [START_REF] Atefeh | A survey of techniques for event detection in twitter[END_REF].

Since its establishment, Twitter has generously opened a portion of its data to the public and has attracted extensive research in many areas [START_REF] Sakaki | Earthquake shakes twitter users: Real-time event detection by social sensors[END_REF][START_REF] Guo | Detecting non-personal and spam users on geo-tagged twitter network[END_REF][START_REF] Huang | Understanding us regional linguistic variation with twitter data analysis[END_REF]. In many studies, the primary task is to identify the event-related tweets and then exploit these tweets to build domain knowledge-related models for analysis. As defined by Atefeh [START_REF] Atefeh | A survey of techniques for event detection in twitter[END_REF], events are generally considered as "real-world occurrences that unfold over space and time". Compared to many data sources, tweets serve as a massive and timely collection of facts and controversial opinions related to specific events [START_REF] Atefeh | A survey of techniques for event detection in twitter[END_REF]. Furthermore, events discussed on Twitter vary in both scale and category, while some may reach to global audiences such as presidential election [START_REF] Wang | A system for real-time twitter sentiment analysis of 2012 u.s. presidential election cycle[END_REF], and others, such as wildfire [START_REF] Pohl | Automatic sub-event detection in emergency management using social media[END_REF][START_REF] Palen | Twitter-based information distribution during the 2009 red river valley ood threat[END_REF], appeal to local users. In general, studies of Twitter events can be categorized into natural events [START_REF] Sakaki | Earthquake shakes twitter users: Real-time event detection by social sensors[END_REF], political events [START_REF] Tumasjan | Election Forecasts With Twitter[END_REF], social events [START_REF] Lee | Discovery of unusual regional social activities using geo-tagged microblogs[END_REF], and others [START_REF] Mathioudakis | Twittermonitor: Trend detection over the twitter stream[END_REF].

Originated from the Topic Detection and Tracking (TDT) program, detection of retrospective or new events has been addressed over two decades from a collection of news stories [START_REF] Allan | In: Introduction to topic detection and tracking[END_REF]. Historically, there exist a number of systems developed to automatically detect events from online news [START_REF] Zhang | Geoburst: Real-time local event detection in geo-tagged tweet streams[END_REF][START_REF] Li | Tedas: A twitter-based event detection and analysis system[END_REF][START_REF] Ifrim | Event detection in twitter using aggressive filtering and hierarchical tweet clustering[END_REF].

An event usually consists of many sub-events, which can describe various facets of it [START_REF] Pohl | Automatic sub-event detection in emergency management using social media[END_REF]. Furthermore, users tend to post new statuses of an event to keep track of the dynamics of it. Within an event, some unexpected situations or results may occur and surprise users, such as the bombing during the Boston Marathon and the verdict moment of the Zimmerman trial. By building an intelligent system, we can identify these sub-events to quickly respond to them, thus avoiding crisis situations or maximizing marketing impact.

Background

Traditionally, unsupervised models and supervised models have been widely applied to detect events from news sources. Clustering methods have been a classic approach for both Retrospective Event Detection (RED) and New Event Detection (NED) since 1990s. According to Allan et al. [START_REF] Allan | In: Introduction to topic detection and tracking[END_REF], they designed a single pass clustering method with a threshold model to detect and track events from a collection of digitalized news sources. Chen and Roy [START_REF] Chen | Event detection from ickr data through wavelet-based spatial analysis[END_REF] also applied clustering approaches such as DBSCAN to identify events for other user-generated contents such as photos. Additionally, supervised algorithms such as naive Bayes, SVM, and gradient boosted decision trees, have been proposed for event detection. Becker et al. [START_REF] Becker | Beyond trending topics: Real-world event identification on twitter[END_REF] employed the Naive Bayes classifier to label the clustered tweets into event-tweets or nonevents tweets with derived temporal features, social features, topical features, and Twitter-centric features, while the tweets are grouped using an incremental clustering algorithm. Sakaki et al. [START_REF] Sakaki | Earthquake shakes twitter users: Real-time event detection by social sensors[END_REF] applied the Support Vector Machine to classify tweets into tweets related to target events or not with three key features. Subsequently, they designed a spatial-temporal model to estimate the center of an earthquake and forecast the trajectory of a hurricane using Kalman filtering and particle filtering. Popescu and Pennacchiotti [START_REF] Popescu | Detecting controversial events from twitter[END_REF] proposed a gradient boosted decision tree based model integrated with a number of custom features to detect controversial events from Twitter streams.

Furthermore, ensemble approaches are also employed to address the event detection problem. Sankaranarayanan et al. [START_REF] Sankaranarayanan | Twitterstand: News in tweets[END_REF] first employed a classification scheme to classify tweets into different groups, and then applied a clustering algorithm to identify events.

As argued by Meladianos et al. [START_REF] Meladianos | Degeneracy based real-time sub-event detection in twitter stream[END_REF], sub-event detection has been receiving more and more attention from the event research community. For the time being, there are a number of studies dealing with sub-event detection in an offline mode [START_REF] Zubiaga | Towards real-time summarization of scheduled events from twitter streams[END_REF]. Zhao et al. [START_REF] Zhao | Human as real-time sensors of social and physical events: A case study of twitter and sports games[END_REF] adopted a simple statistical approach to detect sub-events during NFL games when tweeting rate suddenly rose higher than a prior threshold. Chakrabarti and Punera [START_REF] Chakrabarti | Event Summarization Using Tweets[END_REF] developed a two-phase model with a modified Hidden Markov Model to identify sub-events and then derived a summary of the tweets stream. However, their approach has a severe deficiency because it fails to work properly under situations when unseen event types are involved. Zubiaga et al. [START_REF] Zubiaga | Towards real-time summarization of scheduled events from twitter streams[END_REF] compared two different approaches for sub-event detection. The first approach measured recent tweeting activities and identified sub-events if there was a sudden increase of the tweeting rate by at least 1.7 compared to the previous period. The second approach relied on all previous tweeting activities and detected sub-events if the tweeting rate within 60 seconds exceeded 90% of all previously tweeting rates. As claimed by the authors, the latter outlier-based approach outperformed the first increase-based approach since it neglected situations when there existed low tweeting rates preceded by even lower rates [START_REF] Zubiaga | Towards real-time summarization of scheduled events from twitter streams[END_REF].

Nichols et al. [START_REF] Nichols | Summarizing sporting events using twitter[END_REF] provided both an online approach and an offline approach to detect sub-events as well as summarizing important events moments by comparing slopes of statuses updates with a specific slope threshold, which was defined as the sum of the median and three times the standard deviation (median + 3*standard deviation) in their experiment. Shen et al. [START_REF] Shen | A participant-based approach for event summarization using twitter streams[END_REF] incorporated "burstiness" and "cohesiveness" properties of tweets into a participant-based sub-event detection framework, and developed a mixture model tuned by EM which yielded the identification of important moments of an event. Chierichetti et al. [START_REF] Chierichetti | Event detection via communication pattern analysis[END_REF] proposed a logistic regression classifier, to capture the new sub-events with the exploration of the tweet and retweet rates as the features.

In this study, we formalize sub-event detection as an outlier detection problem, where a set of statistical models, Kalman filter (KF), Gaussian process (GP), and probabilistic principle component analysis (PPCA), are used to construct the probability distribution of future observables. Outliers are identified as observations that do not fit these predicted probability distributions. Three real-world case studies (2013 Boston marathon, 2013 NBA AllStar, Zimmerman trial) are investigated to test the effectiveness of the methods. Finally, we discuss the limitations of the proposed framework and provide future directions for improvement.

Methodology

Our goal is to model the evolution of the probability distribution of the tweeting change rate (increase/decrease) from period t -1 to t as defined in the following equation. Each period t spans 30 minutes and #tweets represents the total number of tweets within that period and filtered for the particular event of interest.

𝑣 𝑡 = #𝑡𝑤𝑒𝑒𝑡𝑠 𝑡 -#𝑡𝑤𝑒𝑒𝑡𝑠 𝑡-1 #𝑡𝑤𝑒𝑒𝑡𝑠 𝑡-1 +1 (1)
Three methods (KF, GP, PPCA) described in the following subsections, are evaluated in constructing the probability density function 𝑝(𝑣 𝑡+1 |𝑣 1:𝑡 = {𝑣 𝑡 * … 𝑣 1 * }). All three approximate the target using a Gaussian density function. This probability distribution is then used to determine whether an observation 𝑣 𝑡+1 * is an outlier ( * denotes the actual observation of percentage change). An observation is labeled as unexpected sub-event when it is identified as an outlier at the 0.025 significance test for the onetail test.

𝑝(𝑣 𝑡+1 ≥ |𝑣 𝑡+1

* ||𝑣 1:𝑡 ) < 0.025

Kalman Filter (KF)

Kalman filter and its variants are widely applied in dynamic systems to estimate the state of a system [START_REF] Terejanu | Unscented Kalman Filter/Smoother for a CBRN Pu -Based Dispersion Model[END_REF][START_REF] Ozbek | Employing the extended Kalman _lter in measuring the output gap[END_REF]. In this study, we assume that a latent variable ℎ 𝑡 related to our quantity of interest -percentage change 𝑣 𝑡 , evolves with time using the following linear dynamical system.

ℎ 𝑡 = 𝐴ℎ 𝑡-1 + 𝜂 𝑡 ℎ ( 3
)
𝑣 𝑡 = 𝐵ℎ 𝑡 + 𝜂 𝑡 𝑣 (4)
ℎ 1 ~ 𝑁(𝜇 0 , 𝜎 0 2 ) (5)
Here, 𝜂 𝑡 ℎ is the process noise and 𝜂 𝑡 𝑣 is the measurement noise. They are as assumed to be independent of one another, temporally independent, and normally distributed according to 𝑁(0, Σ 𝐻 ) and 𝑁(0, Σ 𝑉 ) respectively. The model parameters 𝐴, 𝐵, Σ 𝐻 , Σ 𝑉 , 𝜇 0 , 𝜎 0 2 are learned from the data using the Expectation Maximization (EM) algorithm [START_REF] Fletcher | The kalman _lter explained[END_REF].

The initial mean 𝜇 0 and variance 𝜎 0 2 are obtained using data from a 12 hour window, and the EM is run on a 12 hour moving window to determine the rest of the parameters. After the parameters are obtained, we make a prediction for the next 30 minutes to compute the probability 𝑝(𝑣 𝑡+1 |𝑣 1:𝑡 = {𝑣 𝑡 * … 𝑣 1 * }) and test whether the next incoming observation is a sub-event.

Gaussian Process (GP)

To better capture the non-linearity in the data, we have also tested Gaussian processes. GP is a generalization of a multivariate Gaussian distribution to infinitely many variables [START_REF] Snelson | Variable noise and dimensionality reduction for sparse gaussian processes[END_REF]. Specifically, a GP defines a distribution over functions, 𝑝(𝑓), and 𝑓 is a mapping function. In this study we use GP to capture the nonlinear relation between several past observations of percentage change and future ones. Namely, we consider the following model.

𝑣 𝑡 = 𝑓(𝑣 𝑡-1 , 𝑣 𝑡-2 , 𝑣 𝑡-3 ) + 𝜖 𝑡 (6)
Here 𝑓(•) ∼ 𝐺𝑃(⋅ |0, 𝑘) and 𝜀 ∼ 𝑁(⋅ |0, 𝜎 2 ), where 𝑘(⋅,⋅) is the kernel function. Here, 𝐾 𝑁 represents the Gram matrix whose entries are given by the kernel function evaluated at the corresponding pairs of inputs in the training data. 𝐾 * 𝑁 is a row vector corresponding with kernel function evaluated between the new input 𝑥 * are all the training data points, and 𝐾 * * is kernel function evaluated at the new input point.

Probabilistic Principle Component Analysis (PPCA)

A third model is tested by simply approximating the joint distribution 𝑝(𝑣 𝑡 , 𝑣 𝑡-1 , 𝑣 𝑡-2 , 𝑣 𝑡-3 ) using a Gaussian distribution based on 48 samples corresponding to each 24-hour moving window. The prediction for the quantity of interest is obtained via conditioning the joint using the past three observations. Since we need to approximate the covariance in 4 dimensions using only 46 samples, we propose to use a more robust estimator such as PPCA than simply computing the sample covariance matrix.

PPCA model is defined as a linear relationship between the 4-dimensional observable [𝑣 𝑡 , 𝑣 𝑡-1 , 𝑣 𝑡-2 , 𝑣 𝑡-3 ] 𝑇 and the M-dimensional latent variable 𝑧 𝑛 which follows a zero-mean normal distribution with unit covariance matrix [START_REF] Tipping | Probabilistic principal component analysis[END_REF]. In this study we have set 𝑝 = 2.

[𝑣 𝑡 , 𝑣 𝑡-1 , 𝑣 𝑡-2 , 𝑣 𝑡-3 ] 𝑇 = 𝑊𝑧 𝑛 + 𝜇 + 𝜖 𝑛 [START_REF] Pohl | Automatic sub-event detection in emergency management using social media[END_REF] Here, 𝑊 is a 4 × 2 matrix, 𝜇 is the data offset, and 𝜖 is the projection error, which assumed to follow isotropic Gaussian distribution 𝜀 ∼ 𝑁(0, 𝜎 2 𝐼). We can then obtain the joint distribution of the features by integrating out the latent variables:

𝑝[𝑣 𝑡 , 𝑣 𝑡-1 , 𝑣 𝑡-2 , 𝑣 𝑡-3 ] 𝑇 ∼ 𝑁(𝜇, 𝐶) (8)
Here, the covariance matrix C = 𝑊𝑊 𝑇 + 𝜎 2 𝐼. The parameters W, 𝜇, and 𝜎 2 can be either estimated using the EM approach or by maximizing the following likelihood function [START_REF] Tipping | Probabilistic principal component analysis[END_REF].

L = - 𝑁 2 𝑑ln(2π) + ln|C| + tr(C -1 S) (9)
S = 1 𝑁 ∑ (𝑡 𝑛 -𝜇)(𝑡 𝑛 -𝜇) 𝑇 𝑁 𝑛=1 (10)

Experiments

Twitter data were collected from Jan. 2, 2013 to Oct. 7, 2014 using Twitter streaming APIs. Then we handpicked three national events during this period, including the 2013 Boston marathon event, the 2013 NBA AllStar event, and the Zimmerman trial event.

For these events, we filtered out relevant tweets with pre-specified keywords and hashtags, and provided basic summary of the events shown in Table 1. For two of the three events, we detected sub-events using data retrieved in one week. However, for the Zimmerman trial event, we missed partial data and thus used data collected in three days that were relevant to the event. Based upon the data, we developed an online detection system that could capture outliers. Figure 1 shows a daily pattern of the number of users and number of tweets for the collected tweets. As the figure indicates, there exist periodic patterns for both the number of tweets and the number of users.

Results

As shown in Figure 2-4, the upper sub-plot to the lower sub-plot are outliers identified by the KF, GP, and PPCA algorithms, respectively. Red color indicates actual percentage change of tweets, green color indicates the confidence interval, and cyan color indicates the identified outliers by each algorithm. For the Boston marathon event, as shown in Figure 2, there were 90, 4, and 7 sub-events detected by the KF, GP, and PPCA algorithms, respectively. 4 of the 90 sub-events identified by KF were labelled as real sub-events, 2 sub-events identified by GP were labelled as real sub-events, and 3 sub-events identified by PPCA were labelled as real sub-events. For this particular event, GP yielded the best precision and 2 of the 4 identified sub-event were real subevents. Meanwhile, KF achieved the best recall but with many false positives. Outliers of the Zimmerman trial event are visualized in Figure 3. In terms of the recall value, both KF and PPCA captured 3 of the 10 sub-events, but PPCA achieved a slightly better precision value. In contrast, GP achieved the best precision value. A summary of the three picked events are provided in Table 3. Overall, GP and PPCA yield similar F1 score, while GP achieves better recall value and PPCA achieves better precision. KF, in compared to the other two methods, yields the best recall value. This performance is most affected by the Boston event, in which many false positives are identified. More interestingly, we notice that GP provides robust estimates of the uncertainty while the other two methods yield higher uncertainty estimates for time windows after outliers. This observation can be illustrated by the large green confidence bounds after the two spikes in Figure 2 and Figure 3.

Conclusion

In this study, we explore of building an intelligent system for sub-event detection with three probabilistic models. The sub-events of a point-of-interest event is captured by the system if a new observation is out of the confidence bound of the predictive distribution. We demonstrate the proposed system could capture sub-events with varying performance. The KF model is able to produce slightly better recall, while the GP model is most robust to outliers and yields the best precision. Compared to these two models, PPCA achieves a balanced performance on recall and precision, yielding the best overall F1 score. Nevertheless, we need to interpret the aggregated evaluation with caution because the performance is affected by individualized events, outliers, and the proper choice of parameters. In the future study, we will further tune the parameters, incorporate robust distributions (e.g. t distribution), and take content features into considerations.

Fig. 1 .

 1 Fig. 1. Daily patterns of the collected tweets during 04/12/2013 and 04/18/2013.

Fig. 2 .

 2 Fig. 2. Predicted sub-events with KF, GP, and PPCA, for the 2013 Boston marathon event. The cyan color indicates the sub-events identified by each algorithm.

Fig. 3 .

 3 Fig. 3. Predicted sub-events with KF, GP, and PPCA, for the Zimmerman trial event. The cyan color indicates the sub-events identified by each algorithm.

Fig. 4 .

 4 Fig. 4. Predicted sub-events with KF, GP, and PPCA, for the 2013 NBA AllStar event. The cyan color indicates the sub-events identified by each algorithm.

Figure 4

 4 Figure4indicated the identified outliers of the NBA AllStar event. For this event, KF outperformed the other two methods and yielded slightly better recall and precision value. It captured 3 of the 12 sub-events and 3 of the 5 predicted sub-events were real.

  Common choices for kernel function include the squared exponential kernel function, polynomial kernel functions, and sigmoid kernel functions. In this work we have used cubic covariance function, parameters of which are determined using maximum likelihood estimation for each 24 hour moving window, where the training data consists of

	inputs {𝑋 = (𝑣 𝑡 * , 𝑣 𝑡-1 * , 𝑣 𝑡-2 * ) 𝑡=3…47 , 𝑌 = (𝑣 𝑡 * ) 𝑡=4…48 }.
	Once the training is completed, the probability density function corresponding to a
	new input 𝑥 * = (𝑣 𝑡 * , 𝑣 𝑡-1 * , 𝑣 𝑡-2 * ) is obtained via conditioning of the joint as follows. 2 ) 𝑝(𝑣 𝑡+1 |𝑥 * , 𝑋, 𝑦) = 𝑁(𝜇 * , 𝜎 * 𝜇 * = 𝐾 * 𝑁 (𝐾 𝑁 + 𝜎 2 𝐼) -1 𝑦 𝜎 * 2 = 𝐾 * * -𝐾 * 𝑁 (𝐾 𝑁 + 𝜎 2 𝐼) -1 𝐾 𝑁 * + 𝜎 2

Table 1 .

 1 Basic information for the picked events

	Event	Collection	Event Time	Collection	Key Words/Hashtags
		Starting Time		Ending Time
	2013 Boston	04/12/2013	04/15/2013	04/18/2013	marathon, #marathon
	marathon	00:00:00	14:49:00	23:59:59
	2013 NBA	02/14/2013	02/17/2013	02/20/2013	allstar, all-star
	AllStar	00:00:00	20:30:00	23:59:59
	Zimmerman	07/12/2013	07/13/2013	07/15/2013	trayvon, zimmerman
	trial	11:30:00	22:00:00	11:30:00

Table 2 .

 2 Evaluation metrics of the picked events

		Recall	Precision	F1
	KF	0.34	0.10	0.15
	GP	0.21	0.55	0.30
	PPCA	0.28	0.42	0.33

Table 3 .

 3 Evaluation metrics of the picked events

			Predicted
			sub-event	non sub-event
	KF	Real sub-event	10	19
		non sub-event	92	552
	GP	Real sub-event	6	23
		non sub-event	5	639
	PPCA Real sub-event	8	21
		non sub-event	11	633

https://about.twitter.com/company


Theodoros Iliou
email: tiliou@med.duth.gr

Georgia Konstantopoulou

Ioannis Stephanakis

Konstantinos Anastasopoulos

Dimitrios Lymberopoulos

George Anastassopoulos

Iliou Machine Learning Data Preprocessing Method for Stress Level Prediction

Keywords: Data preprocessing, machine learning, data mining, classification algorithms, stress, anxiety disorder, panic disorder

Data pre-processing is an important step in the data mining process. Data preparation and filtering steps can take considerable amount of processing time. Data pre-processing includes cleaning, normalization, transformation, feature extraction and selection. In this paper, Iliou and PCA data preprocessing methods evaluated in a data set of 103 students, aged 18-25, who were experiencing anxiety problems. The performance of Iliou and PCA data preprocessing methods was evaluated using the 10-fold cross validation method assessing seven classification algorithms, IB1, J48, Random Forest, MLP, SMO, JRip and FURIA, respectively. The classification results indicate that Iliou data preprocessing algorithm consistently and substantially outperforms PCA data preprocessing method, achieving 98.6% against 92.2% classification performance, respectively.

Introduction

Anxiety disorders are among the most prevalent mental disorders that may occur in the general population and they can lead to chronic characteristics, associated with significant percentages of mortality [START_REF] Armstrong | Gender differences in anxiety: An investigation of the symptoms, cognitions, and sensitivity towards anxiety in a nonclinical population[END_REF]. This paper is structured as follows: Section 2 describes the Diagnostic criteria, Section 3 presents the Cognitive models, Section 4 presents the Beck Anxiety Inventory (BAI), Section 5 describes our dataset, Section 6 presents the experimental results of this study, while Section 7 concludes this paper and describes future work.

Diagnostic Criteria

According to DSM-5, anxiety of separation, selective mutism, specific phobia, social anxiety disorder (or social phobia), panic disorder, agoraphobia, generalized anxiety disorder, substance-induced or drug-induced anxiety disorder, anxiety disorder due to another physical state, another predetermined anxiety disorder and unspecified anxiety disorder fall in the category of anxiety disorders [START_REF]Diagnostic and Statistical Manual of Mental Disorders (DSM -5)[END_REF].

Panic attack is a sudden onset of intense fear or intense discomfort that culminates in minutes, during which can occur at least four of the following symptoms (abrupt onset may occur from calm situation or stressful situation): palpitations, heart "pounding", or accelerated heart rate, transpiration, trembling or intense fear, breathlessness or feeling of suffocation, choking feeling, pain or discomfort in the chest, nausea or abdominal discomfort, dizziness, instability or fainting, chills or feeling of warmth, hallucinations (numbness or tingling), derealization (feeling unreal) or depersonalisation (feeling posting by itself), fear of losing control or oncoming madness, fear of death.

Generalized anxiety disorder is characterized from feelings of excessive anxiety and worry (fearful expectation), which are present quite often and during a period of at least six days for a number of events or activities (such as work and school performance). The person feels that it is difficult to control his worry, while stress and anxiety are associated with at least three of the following symptoms (with some of the symptoms to be present for more days during the last six months) [START_REF]Diagnostic and statistical manual of mental disorders[END_REF] [2] [10]: nervousness or anxiety or stress feeling, feeling of an unusual upset, difficulty in concentrating or feeling that the mind is emptied, irritability, muscle tension.

Cognitive Models

The theory of anxiety disorders through the cognitive-behavioral model has been the subject of extensive clinical research in recent years. Moreover, the use of this model in therapy has demonstrated so far promising indicators of high efficiency [START_REF] Bandura | Self-efficacy: toward a unifying theory of behavioural change[END_REF][START_REF] Beck | Anxiety disorders and phobias[END_REF][START_REF] Beck | An inventory for measuring clinical anxiety: psychometric properties[END_REF][START_REF] Blackburn | Cognitive therapy for depression and anxiety[END_REF][START_REF] Clark | Cognitive Approaches[END_REF][START_REF] Creamer | The Beck Anxiety Inventory in a nonclinical sample[END_REF][START_REF]Diagnostic and Statistical Manual of Mental Disorders (DSM -5)[END_REF][START_REF] Hoehn-Saric | Panic and Generalized Anxiety Disorders[END_REF][START_REF] Kendall | Anxious self-talk: development of the anxious self-statements questionnaire[END_REF][START_REF] Kendall | Self-referent speech and psychotherapy: the balance of positive and negative thinking[END_REF][START_REF] Kendall | The future for cognitive assessment of anxiety: Let's get specific[END_REF][START_REF]Anxiety and stress disorders: cognitivebehavioral assessment and treatment[END_REF]. [START_REF] Hoehn-Saric | Panic and Generalized Anxiety Disorders[END_REF], and also Freeman, in 1990 described stress as a globally known experience that function like a warning safety mechanism. This mechanism emits warning signals during hazardous conditions, associated with uncertainty, while it seems to malfunction in a number of cases, like for example when: (a) stress is too intense, (b) stress lasts also after the exposure to the risk (c) occurs in situations in which risk or threat are absent, or (d) occurs without any particular reason.

Hoehn-Saric & McLeod in 1988

Beck Anxiety Inventory

Beck Anxiety Inventory (BAI), created by Aaron T. Beck and his colleagues, is a 21-item, multiple-choice, self-report inventory that is used to measure severity of anxiety in children and in adults [START_REF] Beck | An inventory for measuring clinical anxiety: psychometric properties[END_REF]. BAI questions are associated with symptoms of anxiety that the subject experienced during the past week (including the day of the BAI), such as numbness and tingling, sweating, and fear that the worst is going to happen. It can be administered to individuals over 17 years old and it takes around 5 to 10 minutes in order to be completed. Several studies have found BAI to be an accurate measurement of anxiety symptoms in children and adults [START_REF] Raj | The Postural Tachycardia Syndrome (POTS): Pathophysiology, Diagnosis & Management[END_REF].

Somatic subscale is more emphasized on the BAI, as there are 15 out of 21 items measuring physiological symptoms, while the rest of the items concern cognitive, affective, and behavioral components of anxiety. Therefore, BAI functions more adequately in anxiety disorders with a high somatic component, like for example panic disorder. On the other hand, BAI is not appropriate for disorders, such as social phobia or obsessive-compulsive disorder, which have a stronger cognitive or behavioral component. Many questions of the Beck Anxiety Inventory include physiological symptoms, such as palpitations, indigestion, and trouble breathing. Because of this, it has been shown to elevate anxiety measures in those with physical illnesses like postural orthostatic tachycardia syndrome, when other measures did not [START_REF] Pyle | Data Preparation for Data Mining[END_REF].

The Beck Anxiety Inventory (BAI) and the Anxiety Disorders Interview Schedule (ADIS-IV) were administered to 193 adults with a primary diagnosis of generalized anxiety disorder (GAD), specific or social phobia, panic disorder with or without agoraphobia, obsessive-compulsive disorder (OCD), and no psychiatric diagnosis, at a major Midwestern university recruited from an anxiety research and treatment center. The results of this study support previous findings that the strongest quality of the BAI is its ability to assess panic symptomatology and can be used as an efficient screening tool for distinguishing between individuals with and without panic disorder [START_REF] Leyfer | Examination of the utility of the Beck Anxiety Inventory and its factors as a screener for anxiety disorders[END_REF]. The study of Steer RA, Kumar G, Ranieri WF (1995), administered the Beck Anxiety Inventory to 105 outpatients between 13 and 17 years old who were diagnosed with various types of psychiatric disorders and the results are supporting the use of the inventory for evaluating self-reported anxiety in outpatient adolescents [START_REF] Powers | Evaluation: From Precision, Recall and F-Measure to ROC, Informedness, Markedness & Correlation[END_REF].

The Beck Anxiety Inventory (BAI), created by Aaron T. Beck and other colleagues, is a 21-question multiple-choice self-report inventory that is used for measuring the severity of anxiety in children and adults. It is a reliable tool. The BAI contains 21 questions, each answer being scored on a scale value of 0 (not at all) to 3 (severely).

Data Collection

Questionnaires were completed by 103 students, aged 18-25, who were experiencing anxiety problems and had visited the Office of the Special Consulting Health Services, of Patras University, from the September 2014 until the June 2016.

A result between 0-21 indicates low levels of stress. Although this is usually positive, someone should consider whether the result is realistic or if the person is in refusal concerning its problems. A result between 22-35 indicates moderate stress levels, meaning that the body is trying to tell something. The person should look for patterns in relation to the appearance of symptoms. For example, whether symptoms appear before a specific activity (e.g., a business meeting) and must learn to manage the stress before this activity. No panic is needed, but certainly some technical manager stress will help. A result over 36 points shows high levels of stress and is certainly a reason to ask for supply management. Remember that a high level of anxiety is not a sign of weakness or personal failure. It is something that should be addressed in order to prevent the impact to the mental and physical level of someone [START_REF] Creamer | The Beck Anxiety Inventory in a nonclinical sample[END_REF]. Questionnaires were completed by 103 students, aged 18-25, who were experiencing anxiety problems and had visited the Office of the Special Consulting Health Services, of Patras University, from the September 2014 until the June 2016. It was performed by one person, the psychologist of the office. Though the BAI was developed to minimize its overlap with the depression scale as measured by the Beck Depression Inventory, a correlation of r=.66 (p<.01) between the BAI and BDI-II was seen among psychiatric outpatients, suggesting that the BAI and the BDI-II equally discriminate between anxiety and depression. Another study indicates that, in primary care patients with different anxiety disorders including social phobia, panic disorder, panic disorder with or without agoraphobia, agoraphobia, or generalized anxiety disorder, the BAI seemed to measure the severity of depression. This suggests that perhaps the BAI cannot adequately differentiate between depression and anxiety in a primary care population. We have uploaded the original data as well.

Data preprocessing

The set of techniques used prior to the application of a data mining method are named as data preprocessing for data mining [START_REF] Balasubramanian | The Isomap Algorithm and Topological Stability[END_REF] and it is known to be one of the most meaningful issues within the famous Knowledge Discovery from Data process [START_REF] Hall | Correlation-based feature selection for machine learning[END_REF]. Since real world data are generally imperfect, incomplete (lacking attribute values, lacking certain attributes of interest, or containing only aggregate data), noisy (containing errors or outliers), inconsistent (containing discrepancies in codes or names) and contain redundancies, is not directly applicable for a starting a data mining process. We must also mention the fast growing of data generation rates and their size in business, industrial, academic and science applications. The bigger amounts of data collected require more sophisticated mechanisms to analyze it. Data preprocessing is able to adapt the data to the requirements posed by each data mining algorithm, enabling to process data that would be unfeasible otherwise. Data preprocessing mainly includes: (ii) Data integration: using multiple databases or files.

(iii) Data transformation: normalization and aggregation.

(iv) Data reduction: reducing the features of the initial dataset but producing the same or similar analytical results.

(v) Data discretization: part of data reduction, replacing numerical attributes with nominal ones.

Indicatively, we describe the below data preprocessing methods:

 Principal component analysis (PCA) tries to find a rotation such that the set of possibly correlated features transforms into a set of linearly uncorrelated features. The columns used in this orthogonal transformation are called principal components [START_REF] Vafaie | Feature selection methods: genetic algorithms vs. greedy-like search[END_REF]. This method is also designed for matrices with a low number of features.

 Iliou Preprocessing method transforms the initial dataset into a completely new dataset with 4*m attributes (in big data, Iliou method achieves feature reduction), where m is the number of classes of the dataset output. In this paper we modified our method in order to achieve better results and less complexity.

Iliou method uses Statistics and Linear Algebra methods like the dot (inner) product of dataset vectors in conjunction with some descriptive statistics [START_REF] Iliou | A Novel Machine Learning Data Preprocessing Method for Enhancing Classification Algorithms Performance[END_REF].

The new dataset achieves better classification performance comparing to other preprocessing methods. We used Iliou method for classification or prediction problems [START_REF] Iliou | A Novel Machine Learning Data Preprocessing Method for Enhancing Classification Algorithms Performance[END_REF][START_REF] Kohavi | A study of cross-validation and bootstrap for accuracy estimation and model selection[END_REF]. Below follows the detailed description of the method:

Step 1

Let's assume that a dataset of a machine learning problem named dataset1 is chosen, with n instances (rows), k variables (columns) and m classes. Then, a new set (called from now-on Basic-Set or BS) is created randomly selecting 10% of data from dataset1, consisting of d instances and m classes. The remaining 90% of dataset1 is called Rest-Set.

Step 2

Afterwards, inner (dot) product of every Basic Set instance (row) with the remaining rows of the Basic Set is computed (An inner product is a generalization of the dot product. In a vector space, it is a way to multiply vectors together, with the result of this multiplication being a scalar). Then, follows the calculation of mean and median values of the inner product result for every instance of each class with the rest instances of its class (Mean_classm_rowx and Median_classm_rowx respectively, see equation 1 and 2), producing totally m+m=2m new variables, namely Total_Mean1, Total_Mean2,…, Total_Meanm and Total_Median1, Total_Median2, …, Total_Medianm. For each one of these variables, the method produces d values.

The Mean_classm _rowx and Median_classm_rowx values are calculated as shown in equation ( 1) and equation ( 2) respectively, where m is the name of the class, x (ranges from 1 to d) is the row of the BS and m1, m2 ….mk is the first, second…and last row of m class of the BS. Step 3

Assuming that Rest-Set from step 2 has r instances (rows) and m classes, a similar to step 2 approach follows. Specifically, inner (dot) product of every single Rest-Set row with every single row of the Basic Set is performed. Then, the mean and median values of the inner (dot) product result of every row for each class are calculated (RS_Mean_classm _rowj and RS_Median_classm_rowj respectively), producing new m+m=2m variables for every row of the Rest Set. As a result, we have r values for RS_Mean_classm_rowj, and r values for RS_Median_classm_rowj.

Similarly to step 2, we compute mean and medial values (RS_Mean_classm_rowx and RS_Median_classm_ rowx respectively) for every class.

Apart from the above, the Final_Meanm_rowj and Final_Medianm_rowj values are also calculated as shown in equation ( 5) and equation ( 6) respectively (m is the name of the class and j (from 1 to r) is the row of the Rest set.

Eq. 5

Eq. 6

Finally, m Final_Meanm_rowj and Final_Medianm_rowj values result, one for every class m and every row j of the Rest set.

Step 4

The rows (variables) RS_Mean_classm_rowj, RS_Median_classm_rowj, Final_Meanm_rowj and Final_Medianm_rowj for every class are selected from previous step and then are placed in a new table.

The method ends with the transposition of the Table we described in previous step and the final dataset is now ready to be forwarded in any classification schema. Concluding the description of the proposed method, it is evident that the final dataset consists of 4 variables, namely RS_Mean_classm _rowj, RS_Median_classm _rowj, Final_Meanm_rowj and Final_Medianm_rowj for every class of the initial dataset. Thus, if the original dataset has m classes, the final dataset will have 4*m variables.

Experimental results

The experiments conducted using seven classification schemes (Table 1): IB1 (Nearestneighbour classifier), J48 (C4.5 algorithm implementation), Random Forest, MLP (Multilayer Perceptron), SMO (Support Vector Machines), JRip (Repeated Incremental Pruning to Produce Error Reduction) and FURIA (Fuzzy Unordered Rule Induction Algorithm), respectively. In order to estimate how accurately the above predictive models will perform in practice and to assess how the results will generalize to an independent data set, we used the repeated 10-fold cross validation technique [START_REF]Waikato Environment for Knowledge Analysis, Data Mining Software in Java[END_REF]. The experiments conducted using WEKA 3.8 data mining software [START_REF] Matthews | Comparison of the predicted and observed secondary structure of T4 phage lysozyme[END_REF] by their default WEKA parameters. We evaluated the performance of the classification schemes using Precision, Recall, Kappa statistics, Weighted Avg ROC area, Weighted Avg Precision Recall Curve (PRC) area, Matthews correlation coefficient (MCC) and Root mean squared error metrics (Table 1) [START_REF] Lehmann | Theory of Point Estimation[END_REF][START_REF] Powers | Evaluation: From Precision, Recall and F-Measure to ROC, Informedness, Markedness & Correlation[END_REF][START_REF] Steer | Use of the Beck Anxiety Inventory with adolescent psychiatric outpatients[END_REF][START_REF] Smeeton | Early History of the Kappa Statistic[END_REF]. Hence, we evaluated the performance of Iliou preprocessing method by comparing against Principal Component Analysis (PCA) and initial data without preprocessing.

In literature, Galatzer-Levy, Isaac R. et al. ( 2014) they did experiments classifying trauma survivors according to their stress achieving classification performance 76-80% using SVM, AdaBoost and Random Forest classification algorithms [START_REF] Galatzer-Levy | Quantitative forecasting of PTSD from early trauma responses: A Machine Learning application[END_REF]. Purnendu Shekhar Panday (2017) based on heart beat tried to predict whether a person is in stress or not achieving 68% accuracy in classification process [START_REF] Shekhar | Machine Learning and IOT for Prediction and Detection of Stress[END_REF]. In [34] a real-life, unconstrained study carried out with 30 employees within two organisations and the classification results using ensemble approach increased the accuracy by ≈10% to 71.58% compared to not using any transfer learning technique.

In equations 7 and 8 the formulas for Precision and Recall metrics are presented. The True Positive (TP) is the number of items correctly labeled as belonging to the positive class. The items that correctly labeled as not belonging to the positive class, they are called True Negative (TN). In the case that items which were not labeled as belonging to the positive class by the classifier but should have been, they are called False Negatives (FN). Finally, the items incorrectly labeled as belonging to the class, they are called False Positives (FP). Thus, the number of true positives, true negatives, false negatives and false positives add up to 100% of the set. Precision = (true positives) / (true positives + false positives) Eq. 7

Recall = (true positives) / (true positives + false negatives) Eq. 8

As we can observe in Table 1, initial (raw) data have achieved the best classification results 92% with Support Vector Machines (SMO) algorithm, PCA transformed data achieved the best classification results 90% and 89% with FURIA and JRip algorithm respectively. Finally, Iliou method achieved 98% classification performance with SMO and IB1 algorithm.

Conclusions

In this paper we focused on stress level prediction using initial dataset without preprocessing and datasets being preprocessed by PCA and Iliou data preprocessing methods, respectively. Our experimental evaluation has shown that the Iliou data preprocessing algorithm consistently and substantially outperforms PCA data preprocessing method in classification performance. Moreover, table 1 reveals that Iliou method achieved better classification results than initial dataset. In our point of view, Iliou method can be used for significantly improve classification algorithms performance in every stress level dataset.

In future work, it would be preferable to make the same experiments in larger datasets using more classifiers. In addition, Iliou data preprocessing method could be extended to classification algorithm.

Figure 1 .

 1 Figure 1. Data Preprocessing techniques

  above, the Total_Mean and Total_Median values are calculated based on equation (3) and (4) respectively, where m is the name of the class and d is the sum of rows in the BS. Finally, m total_Mean and m total_MEDIAN values are generated, one for every class of the BS.

Figure 2 .

 2 Figure 2. Experimental Results

Table 1 .

 1 34. AlbanMaxhuni, PabloHernandez, Leal., EnriqueSucar, Venet Osmani, Eduardo Morales, Oscar Mayora. "Stress modelling and prediction in presence of scarce data". In: Ambient Intelligence for Health, Springer International Publishing, Puerto Varas, Chile, pp. 224-336 (2015). Classification Results

					Original data
		Pre %	Rec %	k	ROC	PRC Area	TP Rate	FP Rate	MCC	Root MSE
	IB1	82.1 81.55	0.7	0.852 0.749 0.816 0.105 0.706 0.3455
	J48	80.6 80.6 0.681 0.847 0.715 0.806 0.105 0.701 0.3493
	Ran	82.3 82.5 0.709 0.954 0.905 0.825 0.114 0.716 0.277
	For
	MLP	85.5 86.4 0.772 0.975 0.939 0.864 0.069 0.798 0.2388
	SMO	92.4 92.2 0.872 0.955 0.888 0.922 0.034 0.887 0.3022
	Jrip	73.5 71.8	0.54 0.796 0.705 0.718 0.165 0.555 0.4015
	FURIA 67.1	68	0.45 0.788 0.664 0.680 0.222 0.468 0.4186
					PCA
		Pre	Rec	k	ROC	PRC Area	TP Rate	FP Rate	MCC	Root MSE
	IB1	73.4 73.8	0.56 0.768 0.639 0.738 0.179 0.568	0.41
	J48	87.3 87.4	0.79 0.901 0.836 0.874 0.077 0.797	0.27
	Ran	88.4 88.3	0.80 0.969 0.935 0.883 0.079 0.818	0.26
	For
	MLP	87.6 87.4 0.793 0.964 0.917 0.874 0.057 0.815	0.27
	SMO	89.8 87.4	0.78 0.937 0.835 0.874 0.088 0.811 0.3196
	Jrip	89.4 89.3	0.82 0.917 0.850 0.893 0.064 0.829	0.25
	FURIA 90.5 90.3	0.84 0.953 0.906 0.903 0.056 0.843 0.2325
					ILIOU method
		Pre	Rec	k	ROC	PRC Area	TP Rate	FP Rate	MCC	Root MSE
	IB1	98.7 98.6	0.97 0.990 0.977 0.986 0.002 0.981	0.09
	J48	93.4 93.1	0.88 0.947 0.897 0.931 0.037 0.890	0.21
	Ran	94.5 94.4	0.9	0.998 0.996 0.944 0.035 0.908	0.14
	For
	MLP	97.4 97.2	0.95	1	1	0.972 0.014 0.957	0.09
	SMO	98.7 98.6	0.97 0.993 0.979 0.986 0.002 0.981	0.27
	Jrip	92.5 91.7	0.86 0.959 0.918 0.917 0.051 0.873	0.23
	FURIA 93.2 93.1	0.88 0.985 0.972 0.931 0.047 0.887	0.17