Izwan Nizal
email: nizal@uum.edu.my

Mohd Shaharanee

Jastini Mohd Jamil
email: jastini@uum.edu.my

A Framework for Interestingness Measures for Association Rules with Discrete and Continuous Attributes Based on Statistical Validity

Keywords: data mining, interesting rules, statistical analysis

Assessing rules with interestingness measures is the pillar of successful application of association rules discovery. However, association rules discovered are large in number, some of which are not considered as interesting or significant for the application at hand. In this paper, we present a systematic approach to ascertain the discovered rules, and provide a precise statistical approach supporting this framework. Furthermore, considering that many interestingness measures exist, we propose and compare two established approaches in selecting relevant attributes for the rules prior to rule generation. The proposed strategy combines data mining and statistical measurement techniques, including redundancy analysis, sampling and multivariate statistical analysis, to discard the non-significant rules. In addition to that, we consider real world datasets which are characterized by the uniform and non-uniform data/items distribution with mixture of measurement level throughout the data/items. The proposed unified framework is applied on these datasets to demonstrate its effectiveness in discarding many of the redundant or nonsignificant rules, while still preserving the high accuracy of the rule set as a whole.

Introduction

Data mining or knowledge discovery from data (KDD) is known for its capabilities in offering systematic ways in acquiring useful rules and patterns from large quantities of data. The rules derived from data mining application are considered interesting and useful if they are comprehensible, valid on tests and new data with some degree of certainty, potentially useful, actionable, and novel [START_REF] Han | Data mining : concepts and techniques[END_REF]. [START_REF] Mcgarry | A survey of interestingness measures for knowledge discovery[END_REF] claims that the majority of data mining/machine learning type patterns are rule based in nature with a well defined structure, such as rules derived from decision trees and association rules. The most common patterns that can be evaluated by interestingness measures include association rules, classification rules, and summaries [START_REF] Geng | Interestingness measures for data mining: A survey[END_REF]. Association rule mining is one of the most popular data mining techniques widely used for discovering interesting associations and correlations between data elements in a diverse range of applications [START_REF] Zhang | On the discovery of significant statistical quantitative rules[END_REF]. The association rule mining techniques are different from each other, but a commonality that remains is that all the frequent patterns are first extracted and then association rules are formed from such patterns. Frequent pattern extraction plays an important part in generating good and interesting rules, and is considered the most difficult and complex task. Different methods have been proposed for discovering interesting rules from data and have been categorized into three main classes, namely objective, subjective and semantic measures [START_REF] Han | Data mining : concepts and techniques[END_REF][START_REF] Mcgarry | A survey of interestingness measures for knowledge discovery[END_REF][START_REF] Geng | Interestingness measures for data mining: A survey[END_REF].

Our work in the area of rules interestingness measures is motivated by the objective interestingness measures which are based on probability theory, statistics and information theory. Various objective interestingness criteria have been used to limit the nature of rules extracted, as explained in [START_REF] Geng | Interestingness measures for data mining: A survey[END_REF][START_REF] Shaharanee | Interestingness of Association Rules Using Symmetrical Tau and Logistic Regression[END_REF]. Works such as [START_REF] Bing | Mining association rules with multiple minimum supports[END_REF][START_REF] Yun | Mining association rules on significant rare data using relative support[END_REF] have proposed and successfully developed two approaches, namely multiple support and relative support for generating rules for significant rare data that appears infrequent in the database but is highly associated with specific data. Mutual Information and J-Measures are common information theory approaches in objective interestingness measure [START_REF] Tan | Selecting the right interestingness measure for association patterns[END_REF]. A number of researchers have anticipated an assessment on pattern discovery by applying a statistical significance test as discussed in [START_REF] Shaharanee | Interestingness of Association Rules Using Symmetrical Tau and Logistic Regression[END_REF].

Assessing whether a rule satisfies a particular constraint is accompanied by a risk that the rule will satisfy the constraint with respect to the sample data but not with respect to the whole data distribution [START_REF] Webb | Discovering Significant Patterns[END_REF]. As such, the rules may not reflect the "real" association between the underlying attributes. The hypotheses reflected in the generated rules must be validated by a statistical methodology for them to be useful in practice, because the nature of data mining techniques is data driven [START_REF] Goodman | Data Analysis in the 21st Century[END_REF]. However, even if the rules satisfy appropriate statistical tests, it can still be the case that the underlying association is caused purely by a statistical coincidence [START_REF] Aumann | A Statistical Theory for Quantitative Association Rules[END_REF].

The contributions of the work presented in this paper, is in developing systematic ways to verify the usefulness of rules obtained from association rules mining using statistical analysis. A unified framework is proposed, that combines several techniques to access the quality of rules, and remove any redundant and unnecessary rules. Initial ideas and preliminary results were presented earlier in [START_REF] Shaharanee | Interestingness of Association Rules Using Symmetrical Tau and Logistic Regression[END_REF][START_REF] Shaharanee | Ascertaining Association Rules Using Statistical Analysis[END_REF]. Several extensions and refinements took place in regards to the method being applicable to more realistic datasets including complex data types, infrequent items and uneven attribute value distribution. Furthermore, a comparison of the statistical measure used in our framework with the popular Mutual Information measure is included. The rest of the paper is organized as follows. Section II, briefly overviews some related works and defines the problem of ascertaining the discovered rules. In Section III, we describe our proposed framework. The framework is evaluated using real world datasets and some experimental findings and explanation are given in Section IV. Section V concludes the paper and describes our ongoing work in this field of study.

Related Works

Association rule mining in its most fundamental structure is to discover interesting relationships among items in a given dataset under minimum support and confidence conditions. Commonly used example is in market basket analysis, where an association rule Y X  means if a consumer buys the set of items X , then he/she probably also buys items Y. These items are typically referred to as itemsets [START_REF] Philippe | On selecting interestingness measures for association rules: User oriented description and multiple criteria decision aid[END_REF]. The problem of finding association rules Y X  was first introduced in [START_REF] Agrawal | Mining association rules between sets of items in large databases[END_REF][START_REF] Aggarwal | A new framework for itemset generation. Book a new framework for itemset generation[END_REF] as a data mining task of finding frequently co-occurring items in a large Boolean transaction database. Let X and Y , while the confidence of a rule Y X  is the number of transactions containing X , that also contain Y . In this research we employed an efficient breadth-first method in generating candidate set called Apriori algorithm [START_REF] Toivonen | Sampling Large Databases for Association Rules[END_REF]. The generation of all possible rules was essential in ascertaining the quality of the rules. As this established approach is based on the user specifying the constraints such as support and confidence that must be satisfied. Still, [START_REF] Bing | Mining association rules with multiple minimum supports[END_REF][START_REF] Yun | Mining association rules on significant rare data using relative support[END_REF] argue that for a real large database that is often comprised of either relatively frequent/infrequent items, using multiple and relative support should be considered.

The rules satisfying the standard support and confidence constraints are often too numerous to be utilized efficiently and effectively for the application at hand [START_REF] Nada | Rule Evaluation Measures: A Unifying View[END_REF]. Many patterns from the frequent pattern set are often redundant. Thus we discussed in detailed two useful approaches in handling this redundant problem in [START_REF] Shaharanee | Interestingness of Association Rules Using Symmetrical Tau and Logistic Regression[END_REF]. In the datasets where there is a predefined class label (i.e. classification tasks), frequent pattern mining can contribute to discovering strong associations between occurring attribute and class values. In [START_REF] Cheng | Direct discriminative pattern mining for effective classification[END_REF] the potential usage of frequent pattern mining for classification problem was investigated and successfully applied to the problem. Their approach discovered classification rules by directly discovering the frequent patterns from the datasets with predefined class labels. The results reported were promising since the discovered knowledge model had high accuracy and efficiency for the classification problem.

Proposed Method

Although there are various criteria in determining the usefulness of rules [START_REF] Han | Data mining : concepts and techniques[END_REF][START_REF] Mcgarry | A survey of interestingness measures for knowledge discovery[END_REF][START_REF] Nada | Rule Evaluation Measures: A Unifying View[END_REF] the measures usually just reflect the usefulness of rules with respect to the specific database being observed [START_REF] Webb | Discovering Significant Patterns[END_REF]. The data mining approaches consider the whole search space to find all possible pattern/rules satisfying specific criteria (i.e. association rules). While these criteria, offer some constrains in discovering strong patterns/rules, many misleading, uninteresting and insignificant rules in that domains may still be produced [START_REF] Han | Data mining : concepts and techniques[END_REF]. This problem arises because some association rules are discovered due to pure coincidence resulting from certain randomness in the particular dataset being analyzed. Statistics has previously addressed the issues of how to separate out the random effects to determine if the measured association (or difference in other areas) is significant [START_REF] Agresti | An Intro to Categorical Data Analysis[END_REF][START_REF] Hosmer | Applied logistic regression[END_REF]. Thus additional measures based on statistical independence and correlation analysis are needed to ensure that the results have a sound statistical basis and are not purely random coincidence. The statistical approach offers a firm way of identifying significant rules that are statistically valid. Therefore, the motivation behind our proposed method is to investigate how data mining and statistical measurement techniques can be combined to arrive at more reliable and interesting set of rules. Generally speaking we interpret interesting rules as those rules that have a sound statistical basis and are not redundant. Such an approach requires sampling process, hypothesis development, model building and finally a measurement using statistical analysis techniques to verify and ascertain the usefulness and quality of the rules discovered. This will filter out the redundant, misleading, random and coincidentally occurring rules, while at the same time the accuracy of the rule set will still be sustained.

Conceptual Framework

Figure 1 shows the proposed framework. The dataset is first divided into two partitions. The first partition is used for association rule generation and statistical evaluation, while the second partition acts as a sample data drawn from the database, used to verify the accuracy of discovered rules. To ensure clean and consistent data, standard preprocessing techniques are applied. These preprocessing techniques include the removal of missing values and discretization of attributes with continuous values. As the next step, we determine the relevance of attributes by classifying their importance to characterize an association. A powerful technique for this purpose is the Symmetrical Tau [START_REF] Zhou | A statistical-heuristic feature selection criterion for decision tree induction[END_REF], which is a statistical-heuristic feature selection criterion. It measures the capability of an attribute in predicting the class of another attribute. The measure is based on the probabilities of one attribute value occurring together with the value of the second attribute. [START_REF] Zhou | A statistical-heuristic feature selection criterion for decision tree induction[END_REF] define the Symmetrical Tau measure for the capability of input attribute in predicting the class attribute. Higher values of the Tau measure would indicate better discriminating criterions (features) for the class that is to be predicted in the domain. Symmetrical Tau has many more desirable properties in comparison to other feature selection techniques, as was reported in [START_REF] Zhou | A statistical-heuristic feature selection criterion for decision tree induction[END_REF].

In Section 4.2 we evaluate the capabilities of Symmetrical Tau as the determinant of relevance attributes by comparing it with an information-theoretic measure. The information-theoretic measures are principally comprehensible and useful since they can be interpreted in terms of information. For a rule interestingness measure, the relation is interesting when the antecedent provides a great deal of information about the consequent [START_REF] Julien | Using Information-Theoretic Measures to Assess Association Rule Interestingness[END_REF]. Although several information-theoretic measures exist, we only compared Symmetrical Tau with Mutual Information measurement technique. The Mutual Information is based on information theory to evaluate rules. This approach describes how much information one random variable tells about another one [START_REF] Lotfi | Mining Fuzzy Association Rules Using Mutual Information[END_REF]. The definition of Mutual Information is based on [START_REF] Tan | Selecting the right interestingness measure for association patterns[END_REF]. The features selection technique is utilized in our approach to provide the relative usefulness of attributes in predicting the value of the class attribute, and discard any of the attributes whose relevance value is low. This would prevent the generation of rules which then would need to be discarded anyway once it was found that they comprise of some irrelevant attributes.

The rules are then generated based on the minimum support and confidence framework. However, the application of minimum support assumes that all items in the data are of the same nature and/have similar frequency in the database. This will encounter problems given that the frequency distribution of the attribute values or items in the dataset can be significantly different [START_REF] Bing | Mining association rules with multiple minimum supports[END_REF]. [START_REF] Yun | Mining association rules on significant rare data using relative support[END_REF] assert that the data distribution in database may somehow occur either relatively frequently or not, uniformly or non-uniformly distributed according to the characteristics of the database. In response to this rare items problem, several researchers proposed and successfully developed a solution such as multiple minimum support [START_REF] Bing | Mining association rules with multiple minimum supports[END_REF] and relative support [START_REF] Yun | Mining association rules on significant rare data using relative support[END_REF]. For comparison purpose, we apply both relative support and classical Apriori algorithm framework for association rules mining generation. This is to ensure that we treat the dataset that contains rare items correctly, and this will be demonstrated in the experiments provided in Section 4.3. The discovered rules are then ascertained with statistical techniques. For associations between categorical and continuous variables there are several inferential methods involved. Chi-squared analysis is often used to measure the correlation between items. For a given chi-squared values it can be used to determine if the correlation is statistically significant [START_REF] Han | Data mining : concepts and techniques[END_REF]. The logistic regression methods have become an integral component of any data analysis concerned with describing the relationship between a target variable and one or more input variables [START_REF] Agresti | An Intro to Categorical Data Analysis[END_REF]. Logistics regression is used to estimate the probability that a particular outcome will occur. The coefficients are estimated using a statistical technique called maximum likelihood estimation. These coefficient values are useful in testing the statistical significance of input variables towards target variables [START_REF] Hosmer | Applied logistic regression[END_REF]. The interpretation of regression coefficient in terms of odd ratios is a familiar concept in analysis of categorical data [START_REF] Hosmer | Applied logistic regression[END_REF]. The selection of logistic regression model involves two competing goals: the model should be complex enough to fit the data well, while at the same time simpler models are preferred since they are easier to interpret and are expected to have better generalization [START_REF] Agresti | An Intro to Categorical Data Analysis[END_REF].

We also use some constraint measurement techniques in order to discard the existence of redundant rules [START_REF] Shaharanee | Interestingness of Association Rules Using Symmetrical Tau and Logistic Regression[END_REF]. The combination of these rule ascertaining strategies will facilitate the association rule mining framework to determine the right and high quality rules.

Statistical Analysis

Chi Squared Analysis Logistic Regression Model Building

Rules verification proses

Dataset

The evaluation of the unification framework is performed using the Adult, Iris and Wine dataset, which are real world datasets obtained form UCI Machine Learning Repository. Since all the datasets used are supervised which reflects a classification problem, we have chosen the target variable as the right hand side/consequence of the association rules discovered during association rule mining analysis. In this section, we first show how we handle continuous and discrete attributes. Then we compare two established features selection algorithms namely Symmetrical Tau and Mutual Information in term of their feature subset selection capabilities. Next, we discuss the effect of the frequent and infrequent item in dataset towards the framework. Finally we demonstrate the whole performance of the framework towards both datasets.

Discrete and Continuous Attributes

For many real-world problems, the forms of the input and target attributes emerge form wide range of measurement levels. In handling these types of attributes, we apply the binning approach in improving the boundary of the continuous variables. These bound are created to reflect the upper and lower values for the input variables [START_REF] Dillon | Improvements in Supervised BRAINNE: A Method for Symbolic Data Mining Using Neural Networks[END_REF]. For all continuous attributes in Adult, Iris and Wine, we apply equal depth binning approach methods. This equal depth binning approach will ensure that, we have a manageable data sizes by reducing the number of distinct values per attributes [START_REF] Han | Data mining : concepts and techniques[END_REF]. Other discrete attributes in Adult dataset were preserved in original state.

Comparing Symmetrical Tau (ST) with Mutual Information (MI)

ST and MI are capable of defining irrelevant attributes; they are different from each other in terms of their approach as aforementioned in Section 3.1. Throughout the experiment as shown in Table 1, we found that MI approach favors variables with more values. This observation is in accord with [START_REF] Julien | Using Information-Theoretic Measures to Assess Association Rule Interestingness[END_REF]. On the contrary, the procedure based on ST produces a more stable variables selection which is not in favor to any specific variables criterion. This is in agreement with the claim in [START_REF] Zhou | A statistical-heuristic feature selection criterion for decision tree induction[END_REF][START_REF] Shaharanee | Evaluation and optimization of frequent, closed and maximal association rule based classification[END_REF][START_REF] Shaharanee | Interestingness measures for association rules based on statistical validity[END_REF], of ST being fair towards handling of multi-valued variables.

Unified Target Data and Rare Target Data Problems

As mention in Section 3.1, [START_REF] Yun | Mining association rules on significant rare data using relative support[END_REF] assert that the data distribution in database may somehow occur either relatively frequently or not according to the database's characteristics. In response to this, we compared the dataset that contains both unified target data and rare target data. 2 shows four experiments done for Adult dataset. For the Adult dataset, we have limited the consequence of the rules to be either Income > 50K or Income =<50K. The initial proportion of this target data is unbalanced, making the target data for Adult dataset consist of an infrequent target value (rare target data).

Firstly, we apply the minimum support approach based on the initial proportion of training and test data. Next, we show the results when we have balanced the training dataset, so that we can have a similar proportion between the training and testing data. Then, we made some replication of records in the training dataset. This replication process has generated additional records for training data so that any value from the set of target values has a more similar frequency of occurrence in the training dataset and this will represent a similar proportion between each target item. Finally, we generated the rules based on relative support approach proposed by [START_REF] Yun | Mining association rules on significant rare data using relative support[END_REF]. Based on results obtained, we conclude that, for a rare target data, the most suitable approach in generating the rules are by applying the relative support. This agrees with [START_REF] Yun | Mining association rules on significant rare data using relative support[END_REF], which have successfully applied the relative support in identifying the strong co-relation of significant rare data items compared to classical minimum support approach. While the work presented in [START_REF] Yun | Mining association rules on significant rare data using relative support[END_REF] purposely aims for the efficiency of rare item rule generation, our proposed framework, demonstrated its capabilities in ascertaining the generated rules. Table 3 shows the rules obtained for Iris and Wine dataset as in these datasets contained balanced target values (unified target data).

Overall Framework Performance

Taking in the whole dataset as input would produce a large number of rules, many of which are caused by the presence of irrelevant attributes. Since the ST has more advantageous assets in comparison to MI, ST feature selection criterion is used earlier in the process to remove any irrelevant attributes. This would prevent the generation of rules that comprise of some irrelevant attributes. Hence in this experiment it is not necessary to use ST to further verify the rules as the rules were created from the attribute subset considered as relevant by the measure, as was done in [START_REF] Shaharanee | Interestingness of Association Rules Using Symmetrical Tau and Logistic Regression[END_REF][START_REF] Shaharanee | Ascertaining Association Rules Using Statistical Analysis[END_REF]. The attributes were ranked according to their decreasing ST and a relevance cut-off point was picked. In this experiment, the cut off value was picked based on the significant difference between the ST values in decreasing order. The significant difference was considered to occur in the ranking at the position where that attribute's ST value is less than half of the previous attribute's ST value in the ranking. At this point and below in the ranking, all attributes are considered as irrelevant, as is indicated in Table 1. For example, for Adult dataset, the relevance cutoff value is 0.0166. This is due to the ST value of attribute 'Hours Per Week' being more than double of the ST value for attribute 'Work Class'. Thus, the subset of data consists now of 10 attributes: Marital Status, Relationship, Capital Gain, Education Number, Education, Sex , Occupation, Age, Capital Loss and Hours Per Week. We proceed with the application of an association rule mining algorithm and verification of the extracted rules through statistical analysis. As discussed in 4.3, we concluded that relative support approach is capable of generating rules form a rare target data as in the Adult dataset. On the right hand side of Table 2, we show the progressive difference in the number of rules generated as statistical analysis and redundancy checks are being utilized. The combination of statistical significance analysis and redundant analysis provided proper ways in discarding non-significant rules, which is a significant reduction in the overall complexity of the rule set. From Table 2 we can also see that this great reduction of rules was not at a cost of a significant reduction in accuracy.

In Table 2, based on the statistical and redundant rules analysis performed at rules obtained from relative support approach, we managed to get 42+4 rules as our final significance rules. Input variables namely Marital Status, Relationship, Education, Sex, Occupation, Age, Capital Loss and Hours per Week are selected for the final rules. As depicted in Table 3, the result for Iris and Wine dataset also show no significant deterioration in accuracy with the reduced rule set. As to gauge the effect of rules accuracy on difference set of partitioning for each dataset, k-fold cross validation approach has being utilized, to ensure that we obtained relatively low bias and variance [START_REF] Han | Data mining : concepts and techniques[END_REF]. Based on the experimental results, we have found that the average reduction in the accuracy of the rules set is minor in comparison to the major reduction in the complexity of the rule set.

Conclusions and Future Works

This paper has presented a framework to ascertain the quality of the rules discovered from association rule mining which has a huge amount of rules and complex attributes measurement levels with an integrated statistical and heuristic measurement technique. The experimental results show that, this framework managed to reduce a large number of non-significant and redundant rules while at the same time relatively high accuracy was preserved. This indicates the potential of the framework in providing significant rules when applied to the structured or relational data. As part of our ongoing works, we intend to use the proposed framework to ascertain more complex rules which are discovered from semi-structured data.



  be a set of items. Each transaction T is a set of items, such that I T  . An association rule is a condition of the form of Y X  where I X  and I Y  are two sets of items. The support of a rule Y X  is the number of transactions that contain both

Fig. 1 .

 1 Fig. 1. Proposed framework for rule interestingness analysis.

Table 1 :

 1 Comparison between ST and MI for Adult Dataset (Initial Proportion)

	# of Values Variables	ST Values	# of Values Variable	MI Values
	7	Marital Status	0.1448	6	Relationships	0.1662
	6	Relationship	0.1206	7	Marital Status	0.1575
	6	Capital Gain	0.0706	16	Education	0.0934
	8	Education Number	0.0688	14	Occupation	0.0932
	16	Education	0.0528	8	Education Number 0.0900
	2	Sex	0.0470	10	Age	0.0894
	14	Occupation	0.0469	10	Hours Per Week	0.0545
	10	Age	0.0432	6	Capital Gain	0.0475
	5	Capital Loss	0.0361	2	Sex	0.0374
	10	Hours Per Week	0.0354	5	Capital Loss	0.0238
	7	Work Class	0.0166	7	Work Class	0.0171
	5	Race	0.0085	41	Native Country	0.0093
	41	Native Country	0.0077	5	Race	0.0083
	10	FNLWGT	0.0002	10	FNLWGT	0.0002

Table 2 :

 2 Rules accuracy for Adult data

	Experimental	Dataset	Rule	Type of	Accuracy
	Approaches	Description	#	Statistical Analysis	Training	Testing
	Initial	Training : 30162 records	164	Initial rules	86.75%	86.87%
	Proportion	Testing : 15060 records	53	Statistical Analysis	87.73%	87.92%
			42	Redundancy Check	87.99%	88.13%
	Balanced	Training :	421	Initial rules	71.55%	60.56%
	Data	30162~15016 records	51	Statistical Analysis	73.87%	58.28%
		Testing : 15060 records	30	Redundancy Check	74.00%	63.80%
	Replication	Training :	255	Initial rules	71.65%	59.86%
	Data	30162~45178 records	51	Statistical Analysis	73.64%	58.28%
		Testing : 15060 records	32	Redundancy Check	73.61%	61.70%
	*Multiple	Training: 30162 records	164	Initial rules	86.75%	86.87%
	Support	Testing : 15060 records	53	Statistical Analysis	87.73%	87.92%
			42	Redundancy Check	87.99%	88.13%
		*5% as 2 nd support	42+*4	Redundancy Check	87.34%	87.47%
	Table

Table 3 :

 3 Rules accuracy for Iris and Wine data.

	Dataset	Dataset	Rule	Type of	Accuracy
	Name	Description	#	Statistical Analysis	Training	Testing
	Iris	Train : 90 records	52	Initial Rules	92.86%	90.99%
		Test : 60 records	22	Redundancy Check	88.15%	85.29%
	Wine	Train : 107 records	195	Initial Rules	87.53%	79.44%
		Test : 71 records	17	Statistical Analysis	85.07%	81.98%
			16	Redundancy Check	85.07%	81.98%


Falak Nawaz

Ahmad Mohsin

Syda Fatima
email: fatima@aumc.edu.pk

Naeem Khalid Janjua
email: n.janjua@unsw.edu.au

Rule-based multi-criteria framework for SaaS application architecture selection

Keywords: cloud computing, software as a service, software architecture, SOA, REST, SOAP, architectural style, decision support system

Software-as-a-service (SaaS) is a very successful model for providing cloud-based services over the internet. However, due to the dynamic nature of SaaS services, it becomes very challenging to ensure provision of scalability, applying frequent maintenance and functionality updates to SaaS Services. SOAP and REST are the two mostly used software architectural styles for accessing and consuming SaaS services in cloud environment and each have its distinct advantages. Therefore, to address above mentioned challenges, it is critical to choose the suitable architectural style because the success of a SaaS is strongly coupled with its architecture style. Choosing the right software architecture for a system is a multi-criteria decision making problem and it takes into consideration the architectural style characteristics, non-functional requirements and working domain requirements. In this paper, we propose a rule-based multicriteria decision support system (DSS) for a SaaS application architecture selection. Our proposed DSS uses weighted sum model (WSM) that take into account the architectural style characteristics, non-functional and domain specific requirements.

Introduction

Cloud computing provides ubiquitous access to shared pool of configurable computing resources, software and data services hosted over the internet. The five important characteristics of cloud computing model are on-demand self-service, broad network access, resource pooling, rapid elasticity and measured service [START_REF] Mell | The NIST definition of Cloud Computing[END_REF]. The cloud computing service models are software-as-a-service (SaaS), infrastructure-as-a-service (IaaS), and platform-as-a-service (PaaS). Cloud computing is built on top of virtualization that consists of compute, storage and network components in servers. Virtualization provides a solid foundation for cloud computing to expand. However, cloud computing service models are not mature enough. These service models require proven software design and architecture in order to provide scalable cloud computing platforms [START_REF] Zhang | Research on Cloud Computing Technology Serving Space TT&C Applications[END_REF]. Therefore, selecting a suita-ble architecture for SaaS service provisioning is a critical decision and it contributes towards success of the software. Software architecture describes the complete design of any system. This includes identifying system components, their interactions, granularity of communication needed for interactions, placement of system components in different subsystems, and interface protocols used for communication. Hence, selection of the correct software architecture is critical for the successful implementation of cloud computing platform.

In literature, different architectural styles have been used for delivery of service over the Internet. Service-oriented Architecture (SOA), in broader context, is an evolution of software architecture that provides software application functionality as a service to other applications or services. SOA provides software architectural guidelines for non-functional requirements such as reusability, modularity, extensibility and flexibility. Cloud computing platforms have not fully adopted SOA to make them more reusable, flexible and extensible [START_REF] Zhang | CCOA: Cloud Computing Open Architecture[END_REF]. To build scalable cloud computing platform, we need to leverage SOA to build reusable components, standardized interfaces, and extensible solution.

A Web service provides basic building block to develop a SOA based application. Web service is a method of communications between two computers over a network. There are two very famous types of web service architectural styles that are SOAP and REST. SOAP based implementations are also termed as Service oriented Architecture (SOA) while REST based implementations are known as Resource Oriented Architecture (ROA). We will explore these two architectural styles in next section.

Due to the complexity of cloud computing environment and the software systems running on it, success of the system strongly depends on their architecture. Software architecture has been a key element in software development process for last two decades [START_REF] Moaven | A Decision Support System for Software Architecture-Style Selection[END_REF]. Moreover, software architecture selection is a multi-criteria decisionmaking problem in which software architect try to analyze different goals and objectives for the system under consideration. Some architectural styles may have good effects on a particular problem domain but may not be suitable for another problem domain. Similarly, both REST and SOAP have their own quality attributes and limitations that may have good or bad effect(s) on different problem domains. Although comprehensive list of attributes have been listed for each style in different texts, but we cannot understand the extent to what advantages and disadvantages of quality and quantity attributes of architecture are considered [START_REF] Moaven | A Decision Support System for Software Architecture-Style Selection[END_REF]. Therefore, comparing capabilities, characteristics and benefits of software architecture is somehow difficult task. In this paper, a rule based decision support system (DSS) has been developed which tries to support the decision making process by considering different related criteria for web service architecture including quality attributes, domain requirements and architectural style characteristics.

In the next section, a comparison of REST and SOAP architectural styles is presented. In Section 3 we will describe the motivation for selection of architectural style in cloud-based environment. Section 4 will present factors and criteria for architectural styles, non-functional requirements and domain requirements for multi-criteria decision making. Section 5 will present proposed DSS for Web service architectural selection and finally Section 6 will conclude this paper.

REST vs. SOAP

A Web service provides software functionality online that is accessible through network endpoints. As discussed above, Web service can be technically distinguished between SOAP and RESTful Web services.

SOAP Web services use XML for message format and description of interfaces (WSDL). A SOAP-based architectural style is suitable for application domain with following characteristics [5].

• A formal contract is required between service provider and consumer. Web Services Description Language (WSDL) takes care of all the information required about the web service operations, parameters, types and message format. This is useful in situations where service composition is required in complex business process models. • Architecture needs to handle complex non-functional requirements such as security, trust and coordination. • Architecture needs to handle asynchronous invocation and processing of service requests.

RESTful web services are usually considered suitable for lightweight and ad hoc services. RESTful web services use existing W3C standards, e.g. HTTP. A RESTbased architectural style is suitable for application domain with following characteristics.

• There is no formal description of service endpoints (e.g. no WSDL) between service producer and consumer. The service producers usually provide some additional toolkits describing how to invoke REST web services. • Architecture needs to handle non-functional requirements such as scalability, network bandwidth, and integration. • Architectures where web services are completely stateless just like HTTP requests on a website. For example, existing web applications can expose their functionality easily through CRUD operations using RESTful web services.

Many comparative studies have been carried out to identify strengths and weaknesses of SOAP and RESTful Web services with respect to architecture, technology, security, performance, and scalability. The conceptual, principal and technological level comparison of RESTful Web services and SOAP based "big" web services is performed in [START_REF] Pautasso | Big" Web Services: Making the Right Architectural Decision[END_REF]. The author says, "On the principle level, two approaches have similar quantitative characteristics. On the conceptual level, less architectural decisions must be made when deciding for SOAP-based Web services. On the technology level, the same number of decisions must be made but fewer alternatives have to be considered when building RESTful Web services". SOAP-based Web services produce considerable network traffic and high latency due to large message size [START_REF] Mumbaikar | Web Services Based On SOAP and REST Principles[END_REF]. But payload size in SOAP can be reduced using different compression techniques [START_REF] Wagh | A Comparative Study of SOAP Vs REST Web Services Provisioning Techniques for Mobile Host[END_REF]. RESTful Web services have better performance than SOAP-based Web services in wired and wire-less communication networks. The RESTful web services are lightweight, easy and self-descriptive with higher flexibility and lower overhead.

Another main difference between REST and SOAP Web services [START_REF] Muehlen | Developing Web Services Choreography Standards -The Case of REST vs. SOAP[END_REF] is that SOAP is a tightly coupled design similar to RPC (Remote Procedure Call), and REST is a loosely coupled design similar to navigating Web links. As a study of two opposing standards based on SOAP and REST, SOAP has the advantage of tight coupling of operations, while REST has the advantage of scalability and lightweight access to its operations. REST has the disadvantage of managing the name space with a large number of objects, and a disadvantage of SOAP is the need for dedicated client ports for different types of notification. REST principles also played an important role in standardizing SOAP 1.2.

Internet of Things (IoTs) and Mobile devices has taken this discussion of selecting REST or SOAP style architecture to the next level. A comparative study of SOAP vs REST for provisioning Web services on Mobile phone is performed in [START_REF] Wagh | A Comparative Study of SOAP Vs REST Web Services Provisioning Techniques for Mobile Host[END_REF]. A benchmarking evaluation of SOAP and RESTful Web services is performed in [START_REF] Hamad | Performance Evaluation of RESTful Web Services for Mobile Devices[END_REF] for mobile devices. Benchmarking includes string concatenation and float number addition web services. The performance evaluation results show the advantages of using RESTful web services over conventional web services for mobile devices. Authors in [START_REF] Belqasmi | SOAP-Based vs. RESTful Web Services: A Case Study for Multimedia Conferencing[END_REF] compared REST and SOAP Web services with respect to performance. They concluded that RESTful Web services are now emerging as an alternative to SOAPbased Web services and might be a more suitable choice in some cases.

The more recent work on SOAP and REST highlights that Web Service performance is becoming an important factor. This work "concluded that RESTful web service is a better alternative for SOAP based web services. SOAP based web services are produces considerable network traffic, high latency and the message size is also large this is not in the case of RESTful. The RESTful web services have better performance than SOAP based web services in wired and wireless communication network. The RESTful web services are lightweight, easy and self-descriptive with higher flexibility and lower overhead [START_REF] Mumbaikar | Web Services Based On SOAP and REST Principles[END_REF].

Motivation

Evolution of Cloud services, Internet of Things (IoTs) and Web mobile apps are shaping the future of internet. This will determine how smartphones and physical devices will interact with each other. In this paper, we have focused on why choosing the right architecture for cloud based software-as-a-service (SaaS) is important and what factors drive architecture selection. Each architectural style chosen has trade-offs related to Non-functional requirements (NFRs) and Architectural characteristics (ACs). Domain requirements (DRs) are equally important while choosing the right architecture. Currently software architect give less preference to the domain requirements while choosing the architecture. Within each of above mentioned area, there are number factors that need to be taken care of while choosing the right architecture. But there is no such automated tool that makes this multi-criteria decision making easy. Very rare work has been done in this domain. Existing systems focus on ACs and DRs only [START_REF] Moaven | A Decision Support System for Software Architecture-Style Selection[END_REF] and does not take NFRs into account. Our proposed DSS takes all three areas into consideration and uses a knowledge base that has the ability to update its knowledge and suggest suitable choices to the software architect.

Fig. 1. Pictorial representation of the problem

Architecture must be chosen according to the nature of the applications, its domain requirements, non-functional requirements and required characteristics of architecture. Architecture style chosen by considering these requirements will be best suited for cloud based software-as-a-services (SaaS). Therefore a need for decision support systems in this domain is appreciated.

Multi-criteria decision maker for architecture selection

A multi-criteria decision maker will give suitable option to software architect for solving above mentioned problem. A decision support system (DSS) provides support for all phases of the decision making process. It should be easy to use meanwhile providing support for users at all levels to make decision [START_REF] Moaven | A Decision Support System for Software Architecture-Style Selection[END_REF]. We have used CLIPS as a Rule-Based DSS in which the knowledge component includes procedural and inferential rules. We found number of ACs and NFRs for various domains and tried to analyze cloud based SaaS web services. We then prepared a comparative analysis of these ACs and NFRs for both SOAP and REST implementations.

Domain Requirements (DRs)

We have selected some domain characteristics (or high level functional requirements) according to type of application. The selected cloud-based SaaS web services fall in the following domains: E-commerce, Logistics, Telecommunications and Health-care.

Architectural Characteristics (ACs)

We explored the architectural characteristics for SOAP and REST that important for selection of architecture. The characteristics that are considered in our DSS are given below in Table 1.

Table 1. Architectural Characteristics

Non-functional Requirements (NFRs)

There are a number of non-functional requirements and their sub-factors but we only selected security, reliability and performance as key NFRs for our DSS for architecture selection process. The reason for selecting these NFRs is that these are the most common NFRs required for applications domains mentioned above.

NFR Sub-factors SOAP REST

Security Encryption   Integrity   Authentication   Authorization   Non-repudiation   Confidentiality   Reliability Point-to-Point   Ordered delivery of message   Delivery status   Elimination of duplicate message   Architectural Style Characteris- SOA REST Heterogeneity   Protocol layering   Loose coupling   Integration style   Resource identification   URI design   Resource interaction semantic   Resource relationship   Contract design   Data representation   Message exchange pattern   Traffic monitoring   Traffic determination   Traffic transformation   Service description   Service identification   Service composition   Resending message   Reliable delivery of message   Performance Caching   Load balancing   Throughput   Response time   Latency   Execution time   Table 2.
Common NFRs required for applications domains

Working of proposed rule-based DSS

In order to select architecture style correctly and precisely, all existing information related to the application are considered. The proposed DSS uses characteristics of Web service architectural styles, characteristics of domain of application being developed and non-functional requirements. We assigned weightage to these characteristics and requirements against both REST and SOAP architectures and used this weighted criteria for inference in DSS. The complete design of proposed DSS is depicted in figure 2 in detail. The proposed DSS has five essential components that help in decision making process.

• Repository • Tools • Rule base • Decision maker • User interface
Responsibilities of each component and what they contribute to decision making process is given below.

Repository

We have three types of repositories which are DRs (Domain Requirements), NFRs (Non-Functional Requirements) and ACs (Architecture Characteristics). In DRs we have characteristics regarding requirements for different domains mentioned in the previous section. NFRs contain requirements provided by different quality attributes and also information regarding the number of sub-attributes of quality attributes provided by specific web service architectural style. ACs has information of all the characteristics of web service architectural styles SOAP and REST so that architect can select according to the nature of application being developed.

Tools

Domain requirements and architectural styles characteristics would be prioritized on the basis of number of characteristics selected.

When the necessary information is gathered and importance of levels of quality attributes are obtained, the DSS applies Weighted Sum Model (WSM) for NFRs and number of characteristics required for specific Web service architectural style and domain requirements would already be counted while gathering information according to the need of app being developed as shown in figure 2. WSM is the simplest MCDA (multi-criteria decision analysis/making) method for calculating a single result from a number of alternatives [START_REF] Triantaphyllou | Multi-Criteria Decision Making: A Comparative Study[END_REF].

We can generalize an MCDA problem by assuming M alternatives and N decision criteria [START_REF] Nawaz | SEMREG-Pro: A Semantic based Registry for Proactive Web Service Discovery using Publish Subscribe Model[END_REF]. We further assume that all criteria/factors mentioned in Table 1 and Table 2 are our benefit criteria which means that higher the values, the better it is. A tick mark in the table is assumed to have a value of 1 and cross mark is assigned 0. Next suppose that w j denotes the relative weight of importance (priority) of the criterion c j and a ij is the performance value of alternative A i when it is evaluated in terms of criterion c j . Then, the total (i.e., when all the criteria are considered simultaneously) importance of alternative A i , denoted as A i WSM-score , is defined as follows:

A i WSM-Score = ∑ j=1 w j a ij , for i = 1, 2, . . , m

In this case, SOAP and REST are alternatives and quality attributes (such as security, reliability, performance) and architectural style characteristics are criteria. The rules that are extracted from architecture characteristics, Non-functional requirements, domain requirements and the priorities as defined by the user of the DSS will become inputs to the system.

5.3

Rule Base

In order to make decisions, we need some rules which indicate the interaction among DRs, NFRs and ACs with respect to the nature of Web services being developed. In other words, the rules determine which domain characteristics are required, which characteristics of architecture are required, what is the importance level of each quality attribute. These rules are kept in the rule base and are extracted from repository.

Decision maker

Decision maker provides a rule-based engine to allow a wide range of knowledge to be represented as heuristics or rule of thumb. These rules specify a set of actions to be performed for a given situation. This component is responsible of receiving information about the priorities ci of every domain requirements, architecture characteristics and non-functional requirements from software architect via user interface. These priorities ci are considered as inputs for the weighted sum model for NFRs and ACs, the decision maker determines a particular architectural style suitable for the cloudbased services being developed. This result is then displayed to the user through user interface. If there is more than one architectural style suitable for a particular domain, the system recommends all of them, leaving the final choice to the software architect.

The system also has the learning capability of storing the final selected architecture by the architect in knowledge base and uses it in future for similar situations to recommend right architecture.

User interface

The user interface is responsible for receiving the information from user regarding domain requirements, architectural style characteristics, non-functional requirements and also the priorities for NFRs and ACs. The suggested architecture style(s) is/are represented by user interface as a suggestion to the architect by their priority.

Conclusion

REST and SOAP are the most used architectural styles for implementing cloud-based software-as-a-services (SaaS). Due to complexity, frequent maintenance and updates of the implemented services, it is important to choose the right architectural style based on NFRs, architectural style characteristics and domain requirements. However, choosing the right architectural style is a multi-criteria decision making problem that requires all the factors to be considered along with their priorities by the software architect. Our proposed system solves this problem by using the weighted sum model (WSM) and rule-based DSS. By defining ACs and NFRs criteria in the form of rules and using relative weight of importance (priority) to WSM, we get a suggested architectural style that is suitable for the given domain.

Fig. 2 .

 2 Fig. 2. Rule-based DSS for Web service architectural style selection


Nicolas Maille
email: nicolas.maille@onera.fr

Modeling Airline Crew Activity to Improve Flight Safety Analysis

This paper reviews processes used by airlines to analyze inflight recorded parameters in their safety management system. Then it investigates how the expected crew activity can be modeled, based on airlines' standard operating procedures, and used to get a better understanding of operational risks. The language allowing the hierarchical description of the crew activity is described as well as the process dedicated to the identification of these elements from the recorded flight parameters. As an application case study, a go-around procedure is modeled and used to analyze a set of real aviation data. Results emphasize how this new flight data analysis process can complement existing tools.

INTRODUCTION

Even if the global accident rate 1 for 2015, is the lowest recorded over the last few years (see Figure 1), safety remains a major challenge for regulators and air operators while the traffic is still increasing in a complex environment. A coordinated, structured, risk-based approach is essential to improve global aviation safety. At the airline level, one of the key elements is the development of Safety Management Systems (SMS) that identify a standardized, data driven system of hazards identifiers, risk assessment, and risk mitigation that enable highly complex systems to run [START_REF] Wolf | The Emerging Role of Safety Management Systems in Aerospace[END_REF].

The cornerstone of a successful SMS program is a proper risk assessment which is based on the estimate of severity and likelihood of life-threatening situations encountered in flights. The digital flight data available on all modern commercial aircraft represent a major source of information 1 As calculated by the International Civil Aviation Organization (ICAO), cf. [1] and [2]. to acquire such an understanding of what happens during routine operations. For that matter, ICAO includes in their definition of SMS [START_REF] Icao | Annex 6: Operation of Aircraft[END_REF]Section 3.3.6] the need for flight data analysis and accumulation of these data. Also major airlines set up highly sophisticated data collection programs (called FOQA 2 or FDM 3 program). Nevertheless, there is no instruction defining how the analysis has to be performed and existing tools cannot afford an efficient proactive safety management based on the identification of new threats. Thus, huge databases with highly valuable information about the operational conditions are available in airlines, but are under-exploited for predictive risk assessment. The development of new analysis methodologies that will complement existing FOQA tools is a step required for a real safety enhancement. The main contribution of this paper is to investigate how to improve on the analysis process with a better use of airline's Standard Operating Procedures (SOPs). The standard methodology used in commercial FOQA tools and other approaches are described in section 2 of this paper. Section 3 presents how flight operating procedures can be modeled and identified from the temporal evolution of recorded flight parameters. In section 4 this framework is applied to real aviation data in order to describe and analyze go-around situations. Then, section 5 discusses how this new approach, based on the identification of crew activity can help to improve aviation safety management. Section 6 provides conclusions and perspectives.

FLIGHT DATA MONITORING

Major commercial airlines routinely download digital flight data recorded during flights, store and analyze them to manage safety. They handle a large number of flights 4 , and the process has to be highly automated. Worldwide, the most widely used FDM tools [START_REF] Gain | Role of Analytical Tools in Airline Flight Safety Management Systems[END_REF] (Teledyne AirFASE, Spirent GRAF, and Sagem AGS) primarily rely on the identification of prescribed exceedances that characterize events that fall outside operator-determined standards. Section 2.1 describes this exceedance based analysis process while the following one illustrates other underdevelopment approaches.

Prescribed exceedances

Recorded flight data are decoded and analyzed on the ground FDM workstation generating reports according to the safety limits imposed by the airline. The analysis process includes: (1) filtering and data cleaning, (2) triggering of safety events (when flight parameters values fall outside of the safety limits) and, (3) statistics on the identified events. The crucial point in this process is the definition of prescribed exceedances that characterizes safety events. Indeed, the accurate definition of these deviations from normal operations is a key element for a comprehensive flight data analysis [START_REF] Speyer | The Flight Operations Monitoring System: A Bundled Approach for Synergistic Safety Management[END_REF]. Their characterization is generally realized in two stages. First, the flight is decomposed in flight phases, for example from taxiout to taxi-in. The number of possible flight phases and their starting conditions vary with the software, and starting conditions are usually based on combination of thresholds for some parameters.

As an example, in one tool, the Approach phase starts when:

1. The aircraft has been, at least for 10 seconds, below 4000Ft above ground level with a rate of descent over 420Ft/mn and one engine below 90% N2 5 . 2. When this first condition has been fulfilled, the Approach phase is validated if the aircraft is between 3000Ft and 1000Ft AGL (Above Ground Level) for at least 10s.

The set of rules defining all the possible flight phases is set by experts and is embedded in the tool. After this first stage, the flight is split up in successive flight phases (Figure 2).

The second stage is the triggering of the events. It relies on the definition of prescribed exceedances which capture deviations from normal operations that the safety department wants to identify and use as safety performance indicators. These unwanted situations are characterized by small sets of parameters that exceed some predefined 4 Typically from 500 000 up to 2 000 000 flights a year. 5 N1 and N2 are the rotational speeds of the engine sections (N1 for low pressure compressor and N2 for the high pressure compressor) expressed as a percentage of the nominal value.

thresholds. Both the choice of parameters and thresholds rely on human expertise. For example the triggering of "Heading deviation at takeoff" events could be defined as follow:

(1) the flight phase is Take-OFF (2) the Ground Speed of the aircraft is over 100Kt and the Pitch is under 1°

(3) the reference heading (HeadingRef) is the heading of the aircraft when it just goes over 100Kt during the Take-Off flight phase (4) the current heading deviation (Dev) is calculated as the absolute value of the difference between the current heading of the aircraft and the heading Reference Dev(t) = ABS(Heading(t) -HeadingRef)

(5) an event of high severity is triggered if the current heading deviation is over 4° during 3s else an event of medium severity is triggered if the current heading deviation is over 3° during 3s else an event of low severity is triggered if the current heading deviation is over 2.5° during 2s

Rules (1) and (2) define the temporal window in which the exceedance will be searched for. Rules (3) and (4) set how the parameter, which has to be compared to prescribed thresholds, is calculated. Finally rule (5) describes the conditions required to trigger the events with an associated severity level.

The definition of all these rules is a complex task that requires lot of expertise. These sets of rules are usually built in the airline and completed when new issues are discovered [START_REF] Jesse | FOQA/FDM in Times of Change -Case Studies[END_REF]. A set of prescribed exceedances (often between 50 and 150) is defined for each fleet (as the thresholds will depend on the aircraft characteristics and performances). These exceedances concentrate on the final approach and landing phases which are known to be the most critical on a safety point of view (cf. [START_REF] Boeing | Statistical Summary of Commercial Jet Airplane Accidents[END_REF] page 21: 49% of fatal accidents occurred during the final approach or the landing for worldwide commercial jet fleet from 2006 through 2015). This approach is well fitted to manage known safety issues that can be clearly specified thanks to operational deviations. Significant safety improvements have been done by airlines that routinely use these tools on all their flight data. The strength of this approach consists in the reliability of the search of these prescribed events on all the available flights. So an objective view of how flights comply with these specific deviations can be set up. Such tools are a major piece in a Safety Management System because it allows an objective valuation of the severity and the likelihood of safety events (such as un-stabilized approaches). Nevertheless these tools are limited to the search of known safety events and there is a flagrant need of complementing methodologies that help to discover new emerging risks, even if more complex definitions of events can be included in these tools [START_REF] Soares | Flight Data Monitoring and its Application on Algorithms for Precursor Detection[END_REF].

Data-mining

The development of new research fields, such has "Knowledge Discovery in Database" or "data-mining", combined with the rapid increase of computing power offers new perspectives for the analysis of these flight data. Since the late 90's, guided by the will to find "the unknown" some data-mining methods have been specialized in the analysis of these flight data. The idea was to free of prescribed exceedances but to use large collections of data to identify abnormal situations. Clustering and anomaly detections techniques have been investigated [START_REF] Amidan | Atypical event and typical pattern detection within complex systems[END_REF][START_REF] Li | Analysis of Flight Data Using Clustering Techniques for Detecting Abnormal Operations[END_REF] and brought valuable results [START_REF] Maille | On the use of data-mining algorithms to improve FOQA tools for Airlines[END_REF]. They allow the identification of flights that differ from the other ones from a statistical point of view. The review of such flights by experts could highlight new safety issues. Nevertheless these techniques have not been widely included in commercial tools. Advances in data-mining techniques are also used to search for precursors of adverse events [START_REF] Janakiraman | Discovery of Precursors to Adverse Events using Time Series Data[END_REF]. The methodology supposes to have identified a set of nominal flights (without the adverse event) and a set of adverse flights (with the adverse event) for the learning process.

These new algorithms offer valuable insights in the operational reality but require a subject matter expert to evaluate the impact on safety and links with standard operating procedures.

Crew activity identification

A third way can be investigated to manage safety. Instead of describing what is outside safety margins (as prescribed exceedances) or considering "average" flight as references and search for atypicalities (roughly as data-mining tools), the idea is firstly to model the expected parameters evolutions, in terms of crew activity and aircraft attitudes and in relation with airlines procedures. Then the model can be used to better understand how crews apply these procedures, to find atypical achievement of specific procedures, or to set up the operational context of some exceedances (for example to make differences between unstabilized approaches according to the landing procedure used -Visual or ILS 6 or Radar approach…). Also this third way is not opposed to the two other ones but has to be combined for a better understanding of operational hazards. 6 Instrument Landing System Section 3 is dedicated to the modeling of airlines' procedures and section 4 presents an application on Go-Around 7 procedures using real aviation data.

MODELING FLIGHT OPERATING PROCEDURES

Airline and airport procedures

Operating an aircraft for an airline is a complex task that requires many skills. While modern airplanes are highly automated and their systems can normally manage the flight, the flight crew has always a monitoring role which requires interacting with these highly automated systems.

As commercial flights are highly standardized, normal, abnormal and emergency operations are described through procedures. Depending on the operational context, the crew is supposed to select and apply the appropriate procedure. Many of these procedures are defined by the aircraft manufacturer and adapted if necessary by the air carrier. Figure 3 gives an example of such a procedure in which we can find physical actions (e.g. push the Thrust Levers to the Take-Off and Go-Around (TOGA) position), crew coordination (announce the Go-Around manoeuver) and monitoring actions (Check the Flight Mode Annunciator (FMA) which gives the state of the autopilot). Such a procedure specifies the sequence of actions that should be made to conduct a go-around.

But implementing these actions will also require further information to take into account contextual elements: air traffic controller clearances, weather, airport configuration, aircraft constraints (i.e. "retract flaps on schedule" refers to airspeed limitations for each flaps position) and state of its systems. While the airline SOP holds true for all go-around manoeuvers, specific elements related to the airport and the runway used are defined in dedicated airport procedures.

Figure 4 illustrates what can be found in the airport arrival procedure, including trajectory constraints (climb, turn left…). The number of available procedures is large as for example some airline can have 4 different go-around procedures to take into account the flight level at which the maneuver is started. Moreover each procedure can be adapted by the crew if required while SOPs cannot take into account all the specificities of the operational situation. Also, the idea is not to precisely model all the available procedures but to extract from them typical activity patterns which can be useful to set up a flight context and better identify and understand possible safety events occurring during flights.

Crew activity profiles

The leading idea is to use SOPs and other procedure elements to give a more precise operational context to flight sections. As an example, instead of knowing only that the aircraft is in final approach (as in current FDM tools), we could know which type of final approach is used (visual approach, ILS approach…) and in which part of the associated procedure the aircraft is (capturing or tracking the ILS slopes, doing the flare…). Having a more precise operational context, even if it will never be an exhaustive one, will help to better define new exceedances, to evaluate their safety impact but also to assess how the adherence to SOPs. Of course some deviations from SOPs can be justified by specific contextual factors (ATC clearances, weather…) but identifying them in a more systematic way will help safety officers to conduct analyses and decide which safety actions have to be implemented: modify, adapt or complete some procedures, improve the training process, add new exceedances to follow specific situations…).

Activity model-The use of elements described in SOPs or in training manuals to identify coherent flight portions requires first to represent them in an adequate model. Also, our work relies on a language dedicated to model the crew activity. The approach adopted to represent these activities is inspired by the script definition given by Schank and Abelson [START_REF] Schank | Scripts, Plans, Goals, and Understanding: An Inquiry into Human Knowledge Structures[END_REF]: "A script, as we use it, is a structure that describes an appropriate sequence of events in a particular context… For our purpose, a script is a predetermined, stereotyped sequence of actions that define a well-known situation". The resulting language, called LDA (a French acronym standing for Activity Description Language), has been shortly presented in [START_REF] Bayssié | Human Activity Modeling for Flight Safety[END_REF] and more completely specified in [START_REF] Bonnet | Un langage de description de l'activité de pilotage, pour l'analyse systématique des vols et l'étude de la variabilité de l'activité[END_REF].

Roughly, the LDA is based on a hierarchical representation of the activity. The higher level is called a "Phase" and is used to model a significant part of the flight, as for example the descent or the final approach or the whole go-around phase. It can normally be associated with a crew high level objective. Often, there are several possibilities allowing completing this task. In other words, there are several stereotyped sequences of events usable to fulfil the goal. Each of them will be captured by a crew activity "Profile". So a profile is a specific sequence of actions that can be used to reach the objectives of the phase.

Each profile contains a sequence of events (or actions) representing this particular "way of doing" that can be used. These events that compose the profile are called 'Schemas'. For example, one profile representing the Final Approach phase may contain the three following schemas: "Align the runway", "Final descent", and "Flare" that represent the three main goals for the crew during this flight phase. Then, each "Schema" can contain more elementary actions, called "Modules" that represent how the "Schema" is implemented by the crew. Modules are the lowest level of actions taken into account by the LDA language and should represent elementary activities which make sense for the task captured by the Schema. For example, inside an ILS final approach, the schema "Align the runway" could contain at least three modules: "Follow a converging segment", "Capture the Localizer" and, "Capture the Glide Slope". Building a representation of the target activity from the SOPs and other materials is a complex task which requires a good understanding of the crew activity.

Temporal constraints can also be associated with modules and schemas, allowing the specification of sequences of events. Figure 5 shows a hypothetical flight phase X for which two ways of doing have been described. Profile 1 displays the first one for which three sequential schemas are expected. While the first schema (1.1) contains two Modules that can be made in parallel, the second schema is directly considered as an elementary action and so has no decomposition in modules. In the second profile, which describes the other way of doing, the second schema (2.2) can start before the end of the first one. From recorded data to activity-The next step is to be able to identify which profile has been used in a given flight through the analysis of the time evolution of the flight parameters. As the flight profile is mainly characterized by its composing schemas and modules, the first stage is to be able to identify these sub-elements. The approach adopted relies firstly on rules defining specific points (key points) in the flights (such as the touch down) from the evolution of the flight parameters.

Roughly, each key point is defined by a temporal window which specifies in which part of the flight to search for this key point, an identification rule (with dedicated thresholds) based on the evolution of flight parameters and the search direction. The description of the key point is made thanks to a xml file. As an example, a key point corresponding to the touch down can be defined with: This description indicates that the key point named TD has to be searched on the whole flight (no constraint (-1) for the beginning and no constraint for the end; names of already found key points can be used). The rule "deviate_from" has to be applied on the "IN_FLIGHT_STATE" parameter (which is one of the flight recorded parameters whose value is 0 when the aircraft is on the ground and 1 when the aircraft is in the air) with a reference value of 0 and a threshold of 0.5. The search has to be done backward (because direction is set to -1), from the end of the search window to the beginning. The key point is validated only if the value of the parameter is over 0.5 for more than 30s as the "TIME" is set to 30 (see figure 6).

Figure 6: Determination of the TD key point with the "deviate_from" rule

The backward search is possible because the flight safety analysis is made when the flight is over. It has to be noticed that the take-off could also be easily identified using the same rule with a forward search. In the tool used for this study, 8 generic rules (as "deviate_from", "close-to"…) were defined.

Once all the defined key-points have been searched (some of them have perhaps not been found), these key points are used to identify modules, schemas, profiles and phases in the flight. This activity identification process uses an xml description of the hierarchical structure of modules and schemas that compose the profile. Each element is defined by two key-points identifying the beginning and the end of the element and a status flag that indicates if the element is required, optional or forbidden. Here is an example of a profile description where the profile starts at the FA keypoint and finishes at the TD key-point. The profile is composed of the schema "ILS Capture" that contains the two modules "Localizer Capture" and "Glide Capture". All these elements are required, so all their defining key-points have to exist in order to identify the activity. Figure 7 shows a sequence of key-points that allow the identification of the profile.

APPLICATION TO GO-AROUND PROCEDURES

Let us now apply this methodology to model and identify go-around situations using aviation data provided by an air carrier. All go-around flights were selected from the database of airbus A319 flights landed in five major European airports over two years. It resulted in a set of more than 700 go-around flights. Each data record contains the time evolution of over 300 parameters (latitude, longitude, altitude, speed, autopilot modes, auto-thrust mode, flap and slat positions, landing gear position…) recorded at a frequency of 1Hz.

Go-around profiles

The standard operating procedures (cf. Figure 3) used by the airline were used to build the go-around profile. It was selected to emphasize in this study the automation level used by the crew to achieve the go-around maneuver as it is a relevant cue to safety management. So a set of four profiles was defined, with one dedicated to capture goaround maneuvers that do not start with the setting of the thrust levers to the take-off and go-around (TOGA) position:

L1: TOGA full automation. The go-around manoeuver relies on the autopilot and auto-thrust automation. The crew never directly flies the aircraft but monitors the automation and manages the aircraft configuration.

L2: Automated TOGA. The go-around maneuver is initiated with automated autopilot or auto thrust modes but the crew takes over control of the plane during the procedure.

L3: Manual TOGA. The crew manually controls the aircraft (with the side sticks) when the go-around maneuver is initiated. Automated modes can be engaged afterwards.

L4: Without TOGA detent. The landing attempt is aborted but without pushing throttles to TOGA detent.

Hierarchical structures of modules and schemas that compose each profile are very similar and in this paper we present only the one of profile L1 (see figure 8).

Identified profiles in the flights

Based on the definition of these four profiles, the XML files defining all the relevant key points (from the evolution of the recorded parameters) and all the activity elements were built. Then all the flights were processed by a dedicated tool (called ASVM, see figure 9) which automatically identifies for each flight the go-around procedure used and all the related schemas and modules. The identified activity is reported in an xml file which can be used for statistics, for adding information on a flight parameter display (as on Figure 10) or to extract focused parts of the flight for further analysis. The profile used to manage the go-around maneuver and all its constituting elements were identified for all the flights. Then, an unbiased view of how crews rely on automation to manage go-around maneuvers can be extracted. Figure 11 shows that in many cases the crew uses a full automation strategy (profile L1). Nevertheless, profile L3 is also often used. In this case the crew took manually the control of the airplane before deciding to abort the landing. Profile 2 is nearly never used (2% of flights) indicating that crews decide before initiating the go-around if they want to manually fly the aircraft but rarely disconnect the automation after the beginning of the maneuver. Profile 4 represents 12% of the analyzed flights so in practice goarounds are quite regularly made without pushing the throttle lever to the dedicated TOGA position.

NEW POSSIBILITIES FOR AVIATION SAFETY ANALYSIS

In this section we discuss how such crew activity identification can contribute to enhance aviation safety analysis. The main idea is not to replace the actual exceedances based approach but rather to complement it by (1) the access to additional information on the adherence to SOPs, (2) the ability to use data exploration tools on more specific parts of flights and, (3) the possible definition of more specific exceedances. We will now review these three points.

Adherence to SOPs

The Airbus' Flight Operations Briefing Notes on Standard Operating Procedures [START_REF] Airbus | Flight Operations Briefing Notes -Standard Operating Procedure[END_REF] recalls that a strict adherence to suitable SOPs and normal checklists is an effective method to:

• Prevent or mitigate crew errors;

• Anticipate or manage operational threats; and thus, • Enhance ground and flight operations safety.

Nowadays airline safety officers have only a limited view of how their flights adhere to SOPs. Of courses the defined exceedances used in their FOQA tools are closely related to SOPs but this exceedance based approach does allow neither the identification of which SOP was chosen by the crew, nor the real evaluation how the sequence of actions adhere to the selected SOP or to airline recommendations, nor the assessment of the suitability of the selected procedure.

As shown in the previous section, the analysis methodology described in this paper relies on the modeling and the identification of the procedure applied by the crew and also produces for each flight an understanding of which procedures were used. This is a first important step towards an analysis of the validity of the selected procedure. For example a flight safety officer could wonder if the profile L4, which is used in 12% of the studied flights, is always the most relevant one as the throttle lever is not pushed in the TOGA position. A first cue is given by the analysis of the altitude above the runway at the beginning of the goaround maneuver. As shown on figure 12, profile L4's goarounds are usually made at high altitude while profile L3's go-around are low level ones, often around the decision altitude defined by the procedure. Thus the selection of the procedure seems coherent with the airline standard while the specific TOGA throttle position is not used only in case of high altitude interrupted approaches.

Exploration tools

The identification of the accomplishment of the procedure allows the extraction of the track of specific parameters during that period of time. Then several achievements of the procedure can be compared, either with graphical tools or with data-mining tools. Let us for example have a look at the evolution of the bank angle during the go-around procedure. Figure 14 displays the bank angle evolution from the initialization of the go-around to the end of the procedure (aircraft stabilized in altitude) for the whole set of studied flights. The color indicates the aircraft flaps and slats configuration while the figure has been generated with the FromDady tool [START_REF] Hurter | FromDaDy: Spreading Aircraft Trajectories Across Views to Support Iterative Queries[END_REF].

On the upper-right figure a flight with a bank angle over 20° and a flaps "full extended" configuration can be identified. This is a questionable maneuver as explained in the Gulf Air Flight GF-072 accident report [START_REF] Bea | Accident Investigation Report[END_REF]: "Flaps 'full' is a flapsetting intended only for the final phases of flight: approach and landing. It is generally selected when a landing can be accomplished. Due to the associated drag, flaps 'full' is not a setting for manoeuvring. A recommended setting for manoeuvring is flaps 'three', especially if the landing gear is 'down'. The effect of the high drag induced by the setting of flaps 'full' is to degrade the manoeuvrability of the aircraft. This typically results in exaggerated control inputs, or over-controlling, by the pilot." Thus such a visual examination of flight parameter tracks during a specific procedure can help to highlight flights that should be more specifically investigated.

Atypical achievement of the procedure can also be searched with more advanced data-mining tools [START_REF] Amidan | Atypical event and typical pattern detection within complex systems[END_REF][START_REF] Li | Analysis of Flight Data Using Clustering Techniques for Detecting Abnormal Operations[END_REF]. The extraction of the precise part of the flight, such as a procedure or an even more specific piloting activity, brings a more focused use of these tools. The MKAD tool [START_REF] Das | Multiple Kernel Learning for Heterogeneous Anomaly Detection: Algorithm and Aviation Safety Case Study[END_REF] was applied on this set of data to rank the atypicality of the goaround procedure. Two kernels were used, one based on discrete data (autopilot status, lateral and longitudinal modes; auto throttle status and mode; flight director status; flaps configuration; landing gear position; engine throttle position) and the other one with continuous data (altitude; air speed; vertical speed; engines' N1 and N2 values; pitch; and bank angle). Figure 15 displays the evolution of some parameters of the more atypical achievement of the go-around procedure identified by this data mining tool (with these settings). The parameters that most contribute to the atypical score of this flight are: engine parameter N2, airspeed, throttle position and autopilot management parameters. It can be noted that the "initial maneuver" schema is particularly long (28s) with changes of the throttle position (between TOGA and FLX/MCT 8 detent) and disconnection of both autopilots. Moreover, after the throttle lever was pushed back to the climb level and one autopilot reengaged the captain pushed again the throttle to the TOGA detent and disconnected the autopilot, as if he/she considered that the initial maneuver was not completely achieved. It can also be noted the late landing gear retraction (35 seconds after the initial TOGA action). The highlight of nonstandard flight management like this one has a high value for safety officers who want to investigate other safety risks than the one already captured by the prescribed exceedances of their FOQA tool.

Also this study contributes to demonstrate the value of using visualization and data mining tools on more specific parts of the flight.

Specific exceedances

The identification of the procedure used could also allow the definition of new exceedances, related to specific sequences of actions (flaps retraction before landing gear retraction in a go-around maneuver for example) or precise search 8 MCT: Maximum Continuous Thrust. windows (turn during the initial go-around maneuver for example). The study of flights with unusual application of procedures is often inspiring for setting efficient new safety markers. Nevertheless this step has not been done yet with the airline.

CONCLUSIONS

This paper questions the process used to analyze inflight recorded data for safety purpose and suggest a new paradigm based on the modelling of airline procedures. An xml based language and an identification process are both described and applied to a go-around procedure. Based on the analysis of a set of 700 commercial flights with an interrupted landing, the study demonstrates the feasibility of the proposed methodology. Then the paper demonstrates that this new approach increases the understanding of how crew activity adheres to SOPs. This is an important step towards an adaption of the training and eventually the SOPs in order to better mitigate risks. Moreover this fine identification of crew behavior can be used to lead the use of other tools and we demonstrate that data mining tools can be efficiently applied on small portions of flight.

Figure 1 :

 1 Figure 1: Global accident rate trends, based on scheduled commercial operations involving aircraft having a maximum take-off weight above 5700kg.

Figure 2 :

 2 Figure 2: Example of flight phases.

Figure 3 :

 3 Figure 3: Generic go-around procedure for airbus aircraft. The figure represents neither the actual Airbus procedure nor a real airline procedure but is close from both of them. .

Figure 4 :

 4 Figure 4: Representative elements of a typical ILS procedure on a large airport. The figure is not the official chart but was adapted from an expired one.

Figure 5 :

 5 Figure 5: Example of crew activity representation with the LDA

Figure 7 :

 7 Figure 7: Example of crew activity identification (Pink: modules, Green: Schema, Blue: Profile)

Figure 8 :

 8 Figure 8: Hierarchical structure of profile L1 (Pink: modules, Green: Schema, Blue: Profile)

Figure 10 :Figure 9 :

 109 Figure 10: Activity identified and flight parameters evolution for one specific flight. (Red (_ALTITUDE): altitude above runway level in feet, Blue (_CAS): controlled air speed in knots, Green (_CONF): flaps and slats configuration, Khaki (_ENG_THR_CMD1): position of the engine 1 throttle, and Pink (_LDG_SEL): landing gear position (0 for down))

Figure 10

 10 Figure 10 displays the result for a specific flight where the go-around was initiated at 455 Ft when the pilot pushed the throttle from the Climb position (parameter _ENG_THR_CMD_1=3) to the TOGA detent. Nevertheless the pilot moved back the throttle to the Climb position 7 seconds later, just after the first flaps retraction. The pilot kept engaged the autopilot and the auto-thrust during the entire maneuver and so the L1 profile was recognized (TOGA full automation). It can be noticed that the pilot retracted only two steps of flaps (_CONF parameter).

Figure 11 :

 11 Figure 11: Breakdown of the flights with go-around by profiles

Figure 12 :

 12 Figure 12: Mean altitude and confidence interval (p=0.95) at the beginning of the go-around maneuver for each profile

Figure 13 :

 13 Figure 13: Sequences of flaps/Slats and landing gear changes. Blue numbers on lines indicate the number of flights that contain that specific sub-sequence. Red numbers are the numbers of flight ending the procedure in that configuration. Also the identification of the procedure used with the extraction of the sequence of action made paves the way to a more circumstantial assessment of the adherence to SOPs. It can also highlight other "ways of doing" used in flight. All these elements should help to manage the training and perhaps the update of SOPs.

Figure 14 :

 14 Figure 14: Evolution of the bank angle during go-around procedures for 700 flights. Colors indicate flaps/slats configurations from full Retracted (blue) to full extended (red).

Figure 15 :

 15 Figure 15: The most atypical go-around procedure achievement identified by the MKAD tool. The upper part shows the evolution of 4 parameters (Red: altitude in Feet, Blue: Airspeed in Knots, Pink: position of the engine 1 throttle, and Green: value of the N2 parameter for engine 1) while the lower part displays the autopilot status (AP_Status), the mode of the auto-thrust (ATS_Mode), and which member of the crew is manually flying the aircraft (Flying)

Flight Operations Quality Assurance.

Flight Data Monitoring.

BIOGRAPHY