Muhammad Amirul Abdullah

Salmiah Ahmad
email: salmiah@iium.edu.my

Siti Fauziah Toha

Design & Material Analysis of Spherical Mobile Robot for Bouncing Mechanism

Keywords: Spherical mobile robot, bouncing, design technique 1

A spherical mobile robot with a spherical steel spokes shell is introduced in this paper. To have the optimum bouncing capability, the shell configurations should be optimized. The design technique of the optimization is by varying and testing the three main parameters which are the material of the spokes, the number of spokes and diameter for each spoke. The material used in the study are five common commercial material available -1060 alloy, ASTM A36 steel, cast alloy steel, e-glass fiber and plain carbon steel. The test simulation is run by model in Simulink and SimMechanics. The highest maximum height of bounce indicates the best parameter. It is shown that the ASTM A36 steel with 14 spokes and 4.00 mm in diameter is the optimum configuration of the spherical mobile robot.

Introduction

The mobile robot is widely used in various applications, such as security surveillance, search and rescue, children's education, entertainment and testbed for research. To be safe and adaptive to that kind of environment, the mobile robot area has been evolved in term of design and mechanism. There are many types of design for mobile robot introduced from wheeled, legged and winged. Inspired by Halme in 1996 [START_REF] Halme | Motion control of a spherical mobile robot[END_REF], a spherical mobile robot has come to the mobile robot area which having many advantages compared to the other designs. It can have both rolling and bouncing mechanism in one single body. With one single contact point with the ground, the spherical structure can freely rotate about any axis of rotation and all positions are stable. The shape of the sphere provides complete symmetry and a soft, safe and friendly looking without any sharp corners or protrusions. There are researches on the mechanism of the spherical mobile robot and yet there is lack of research on the material and configuration of the spherical mobile robot. Current research on the spherical mobile robot mechanism can be referred to Roball [START_REF] Michaud | Roball, the rolling robot[END_REF], Omnibola [START_REF] Jaimez | Design and Modelling of Omnibola ©, A Spherical Mobile Robot[END_REF], BYQ-III [START_REF] Liu | Motion control of a spherical mobile robot by feedback linearization[END_REF], Sphericle [START_REF] Bicchi | Introducing the 'SPHERICLE': an experimental testbed for research and teaching in nonholonomic[END_REF], SpheRobot [START_REF] Pokhrel | Design and Development of a Spherical Robot (SpheRobot)[END_REF], and OmniQiu [START_REF] Chen | Design and implementation of an omnidirectional spherical robot Omnicron[END_REF]. Each of them is having the different material used and size for the spherical mobile robot. These material and size are not take part as an important aspect for the spherical mobile robot as the mechanism is far being concerned. So in this study, five common materials available for industrial and commercial are being simulated to find the best performance for the proposed design of spherical mobile robot with then later studied with the different number and size of spokes of the robot.

Spherical Mobile Robot

The bouncing spherical mobile robot considered in this study is a mobile robot with a sphere shape closed by metal spokes. The mechanism of the spherical mobile robot is an internal driving unit (IDU) which inside of the metal spokes. These metal spokes are chosen due to it can absorb impacts either during rolling or landing. Moreover, the used of the metal spokes is to replicate the spring to create a jumping motion for the spherical mobile robot. This jumping ability add-up the advantages of the spherical mobile robot to relocate its position easily from point to point.

Mathematical Model

As the metal spokes replicate as a spring, the spherical mobile robot shell design can be modeled as a mass-spring-damper model [START_REF] Nagurka | A mass-spring-damper model of a bouncing ball[END_REF] as in Figure 2. It indicated that Metal spokes the spherical mobile robot is subjected to the equation of motion in Equation 1 when the ball is in contact with the ground where m is mass, x is spherical mobile robot position measured from its center of mass, c is viscous damping, k is linear stiffness or spring constant, and g is gravity acceleration.

Fig. 2. A mass-spring-damper model of the spherical mobile robot

݉ݔሷ + ܿݔሶ + ݔ݇ = -݉݃ (1)

Design Scope

Spokes Material

Different materials give different properties of the design. The type of material will influence the design shape, stress, strain, fatigue, impact, wear, creep, and corrosion of the spherical mobile robot. As the study is not to introduce a new type or new composition of material, five common type of materials available are chosen for the study.

1060 Alloy -is a relatively low strength, high purity alloy with a 99.6% minimum aluminum content. It is noted for its excellent welding characteristics and formability along with good corrosion resistance. It has excellent forming capability by cold or hot working with commercial techniques and may be welded by standard commercial methods.

ASTM A36 Steel -is a typical constructional grade ferritic-pearlitic steel widely used for land vehicles and structures and is easily welded without special heat treatments.

Cast Alloy Steel -is a common type of alloy steel available where it can be customized through casting process to improve its material properties.

E-Glass Fiber -is known in the industry as a general-purpose fiber for its strength and electrical resistance. It is the most commonly used fiber in the fiber reinforced polymer composite industry.

Plain Carbon Steel -is a basic carbon steel. Its strength depends on its carbon content. Low carbon content makes a soft and easily form steel. Higher the carbon content, harder and stronger the steel.

Number of Spokes

A different number of spokes will be represented by different number of spring by the mass-spring-damper model in Figure 2. It should affect the value of stiffness. Even though the spokes can be varied in numbers and diameter, the parameter is limited to the overall outer diameter of the spherical robot. In the proposed design of the spherical mobile robot, its outer diameter is limited to 240 mm which is a standard size of a basketball ball. This is to ensure the spherical mobile robot can be further study and validated with respect to the basketball ball.

Diameter of Spokes

In contrast with the different number of spokes, the different size of the diameter of the spokes indicates the diameter size of the spring in the mass-spring-damper model. Each number of spokes and diameter of spokes should have a different weight of effect in the spherical mobile robot. This different weight of effect can be further shown by the results of the simulation.

Simulation Setup

The experiment is modeled using MATLAB software which involved Simulink and SimMechanics features. The spherical mobile robot designed using Solidworks is exported to the experiment. Each of the parameters is designed and exported separately. The experiment is run by varying the initial drop height of the spherical mobile robot from 2.4 m (which is ten times the overall diameter of the robot) to 0.0 m. The first run of the simulation by varying the material of the spokes. From the first run result, the best material is selected and used for the second run. The best material from the first run is re-run with different number of spokes (10 to 14 spokes). Again, the best number spokes obtained from the second run is simulated with different size of spokes diameter (1.0 mm to 4.0 mm) in the third run. The best parameter is selected by taking the highest maximum bouncing height. The data are collected and stored for each run. The graph for each comparison is plotted.

Results and Discussion

The collected experimental simulation data are plotted based on the subjected parameter as in Figure 6 to

In Figure 6, the five different type of spokes materials have given two group of material which are ASTM A36 steel, plain carbon steel maximum height and the others E maximum height. For these three materials in the higher maximum height group, the ASTM A36 steel shows the best performance and proceeded to the

Flowchart of Optimization for Bouncing Spherical Mobile Robot

Discussion

The collected experimental simulation data are plotted based on the subjected to 8.

, the five different type of spokes materials have given two group of material which are ASTM A36 steel, plain carbon steel and cast alloy steel in higher maximum height and the others E-glass fiber and 1060 alloy in the group of lower maximum height. For these three materials in the higher maximum height group, the ASTM A36 steel shows the best performance and proceeded to the next step.

The collected experimental simulation data are plotted based on the subjected the five different type of spokes materials have given two group of and cast alloy steel in higher glass fiber and 1060 alloy in the group of lower maximum height. For these three materials in the higher maximum height group, the In Figure 7, it is shown that a different number of spokes are not significant to the maximum bouncing height within the same size of spokes diameter. The maximum different of maximum height between the spokes is 0.0387 m which is relatively small to take an impact to the spherical mobile robot. Though it gives relatively small affect to the maximum height, the result is considered as the objective of the study is to find the best configuration for the spherical mobile robot shell. The number of spokes of 14 is chosen.

As mentioned in 2.2.3, the diameter of the spokes shows a high weightage of effect on the spherical mobile robot maximum height. The different results of each diameter sizes prove that the diameter of spring in the mass-spring-damper model really make a role in the part of design any mass-spring-damper related robot. By result in Figure 8, spokes diameter of 4.0 mm size is selected as the final size for the spherical mobile robot.

Conclusion

An optimized design for a spherical mobile robot has successfully achieved. The final result of the optimized design is shown in Figure 8 with the red plot of 4.0 mm. The optimized design will be fabricated into prototyped and will be validated. The study also showed that the number of spokes has less impact towards the maximum height compared to the other two parameters which are material and diameter of the spokes. The new prototype will be designed based on the optimized value as in Table 2.

Fig. 1 .

 1 Fig. 1. The proposed design of spherical mobile robot

Fig. 3 .Fig. 4 .Fig. 5 .

 345 Fig. 3. The SimMechanic Model of Bouncing Spherical Mobile Robot

Fig. 6 .

 6 Fig. 6. Maximum bouncing height for the different material of spokes versus initial height.

Fig. 7 .

 7 Fig. 7. Maximum bouncing height for a different number of spokes versus initial height.

Fig. 8 .

 8 Fig. 8. Maximum bouncing height for the different diameter size of spokes versus initial height.

Table 2 .

 2 Optimized Parameter

	Parameter	Value
	Material	ASTM A36 Steel
	Number of Spokes	14
	Diameter of Spokes 4.0 mm

Acknowledgments. Authors would like to express their gratitude to International Islamic University Malaysia for providing facilities to conduct this research. This research was funded by Ministry of Education under Fundamental Research Grant Scheme (FRGS).


Ladislav Lenc
email: llenc@kiv.zcu.cz

Pavel Král

SAPKOS: Experimental Czech Multi-label Document Classification and Analysis System

Keywords: Document Classification, Czech News Agency, Maximum Entropy Classifier, Confidence Measure, System

This paper presents an experimental multi-label document classification and analysis system called SAPKOS. The system which integrates the state-of-the-art machine learning and natural language processing approaches is intended to be used by the Czech news Agency ( ČTK). Its main purpose is to save human resources in the task of annotation of newspaper articles with topics. Another important functionality is automatic comparison of the ČTK production with popular Czech media. The results of this analysis will be used to adapt the ČTK production to better correspond to the today's market requirements. An interesting contribution is that, to the best of our knowledge, no other automatic Czech document classification system exists. It is also worth mentioning that the system accuracy is very high. This score is obtained due to the unique system architecture which integrates a maximum entropy based classification engine with the novel confidence measure method.

Introduction

The amount of textual data available online has been growing rapidly since the origin of the World wide web. In the last decade, the online press substitutes to a considerable extent the traditional printed media. This trend affects in particular press agencies that are forced to organize much larger amounts of data. Unfortunately, it brings considerable demands on the people to process the data and is highly desirable to automatize this task as much as possible.

Document classification (DC) is one of the tasks that are very important for the press agencies and that can be delegated to a computer. A broad list of publications in this domain shows that a number of machine learning algorithms can be employed. In many applications, a single-label classifier is applied. It means that each document is assigned exactly to one class (category). However, it is not sufficient in many real systems where one document usually belongs to multiple categories. The multi-label document classification became thus the today's center of interest [START_REF] Hrala | Multi-label document classification in Czech[END_REF].

The main goal of this paper is to present a novel experimental multi-label document classification system which integrates the state-of-the art machine learning and natural language processing methods. This system is used by the Czech News Agency ( ČTK3 ), the major press agency in the Czech Republic, to automatically assign topics to newspaper documents. Another functionality of this system consists in automatic analysis of the ČTK media production and a subsequent comparison with popular Czech media (Lidové noviny, Mladá Fronta Dnes, etc.). The results of this analysis will enable ČTK to obtain an overview about what information is interesting and which not. Moreover, based on these results it is possible to adapt the direction of its production to better respond to the current market needs. It is worth mentioning that ČTK produces daily several thousands of documents that must be annotated and analysed. The processed documents are mainly in Czech language, however a significant amount is in English.

The following section summarizes important approaches in the document classification and confidence measure domains which are necessary to create the proposed system. Section 3 describes the methods and algorithms that are used in order to create the system and the following section presents its architecture. Section 5 shows experimental evaluation of the system on the real Czech ČTK document corpus. In the last section, we discuss the results and we propose some future system improvement.

Related Work

This section describes first the main approaches in the document classification and confidence measure fields which are beneficial for our document classification system. Then, we present some other existing document classification systems.

The methods used for document classification usually employ supervised machine learning. An annotated corpus must be available for training of a classifier which is subsequently used to annotate unlabelled documents. Vector Space Models (VSMs) are often used as document representation. It usually represents each document as a vector of word occurrences weighted by their Term Frequency-Inverse Document Frequency (TF-IDF).

There are several machine learning approaches successfully used for the document classification [START_REF] Bratko | Exploiting structural information for semi-structured document categorization[END_REF][START_REF] Della Pietra | Inducing features of random fields[END_REF]: Bayesian classifiers, Maximum Entropy (ME), Support Vector Machines (SVMs), etc. However, the main issue of this task is that the feature space in the VSM is highly dimensional which decreases the accuracy of the classifier. Numerous feature selection/reduction approaches have thus been introduced [START_REF] Forman | An extensive empirical study of feature selection metrics for text classification[END_REF][START_REF] Lamirel | Optimizing text classification through efficient feature selection based on quality metric[END_REF] to solve this problem. Furthermore, a better document representation should help to decrease the feature vector dimension, e.g. using lexical and syntactic features [START_REF] Lim | Multiple sets of features for automatic genre classification of web documents[END_REF]. It has been demonstrated in [START_REF] Chandrasekar | Using syntactic information in document filtering: A comparative study of part-of-speech tagging and supertagging[END_REF] that it is beneficial to use POS-tag filtration in order to represent a document more accurately.

More recently, some interesting methods based on Labelled Latent Dirichlet Allocation (L-LDA) [START_REF] Ramage | Labeled lda: A supervised topic model for credit attribution in multi-labeled corpora[END_REF] have been proposed. Another recent work exploits partial labels to discover latent topics [START_REF] Ramage | Partially labeled topic models for interpretable text mining[END_REF]. Principal Component Analysis (PCA) [START_REF] Gomez | Pca document reconstruction for email classification[END_REF] incorporating semantic concepts [START_REF] Yun | A multi-layer text classification framework based on two-level representation model[END_REF] has been also introduced for DC. Nigam et al. proposed in [START_REF] Nigam | Text Classification from Labeled and Unlabeled Documents Using EM[END_REF] an interesting semi-supervised approach, which progressively augments labelled corpus with unlabelled documents. Another approach using enhanced grid based clustering algorithm is presented in [START_REF] Rashad | Document classification using enhanced grid based clustering algorithm[END_REF].

The most of the proposed approaches is focused on English and only few works deal with Czech language. Hrala et al. use in [START_REF] Hrala | Evaluation of the Document Classification Approaches[END_REF] lemmatization and Part-Of-Speech (POS) filtering for a precise representation of Czech documents. In [START_REF] Hrala | Multi-label document classification in Czech[END_REF], three different multi-label classification approaches are compared and evaluated. Other recent works propose novel features based on named entities [START_REF] Král | Named entities as new features for Czech document classification[END_REF] or on unsupervised machine learning [START_REF] Brychcín | Novel unsupervised features for Czech multi-label document classification[END_REF].

Confidence measure is used as a post-processing of the recognition/classification to determine whether a result is correct or not. The incorrectly recognized samples should be removed from the resulting set or another processing (e.g. manual correction) can be further realized.

This technique is mainly used in the automatic speech processing field [START_REF] Senay | Confidence measure for speech indexing based on latent dirichlet allocation[END_REF][START_REF] Jiang | Confidence measures for speech recognition: A survey[END_REF] and is mostly based on the posterior class probability. However, it can be successfully used in another research areas as shown in [START_REF] Servin | Statistical confidence measures for genome maps: application to the validation of genome assemblies[END_REF] for genome maps construction, in [START_REF] Hu | A quantitative evaluation of confidence measures for stereo vision[END_REF] for stereo vision, in [START_REF] Marukatat | Rejection measures for handwriting sentence recognition[END_REF] for handwriting sentence recognition or in [START_REF] Li | Open world face recognition with credibility and confidence measures[END_REF] for automatic face recognition.

The confidence measures are mostly used in the single-label classification. But the nature of many real-world classification problems is multi-label. One approach using confidence measures in the multi-label setting is proposed in [START_REF] Rodrigues | Using confidence values in multilabel classification problems with semi-supervised learning[END_REF]. The authors use semi-supervised learning algorithms and include a confidence parameter when assigning the labels. Two methods for the confidence value computation are proposed.

Another possibility how to deal with the confidence measures is to use a so called Conformal Predictor (CP) [START_REF] Nouretdinov | Machine learning classification with confidence: application of transductive conformal predictors to mri-based diagnostic and prognostic markers in depression[END_REF]. CP assigns a reliable measure of confidence and is used as a complement of machine learning algorithms. Author of [START_REF] Papadopoulos | A cross-conformal predictor for multi-label classification[END_REF] proposes to use a modification called Cross-Conformal Predictor (CCP) to handle the multi-label classification task. He states that this modification is more suitable for this task because of its lower computational costs.

Unfortunately, to the best of our knowledge, no other automatic system for classification of the Czech text documents exists. However, there are some systems dedicated to classify the documents in other languages, particularly in English. The most important ones (often protected by the US patent) are shortly described next.

The first system [START_REF] Wnek | Multi-strategy document classification system and method[END_REF] uses Latent Semantic Indexing (LSI) model to compute the word based feature vectors. Then these vectors are clustered to create a centroid vector representing a category. An inductive learning from examples is further used to associate the document labels to not annotated English text documents.

Glas et al. proposed in [START_REF] Glass | Document similarity detection and classification system[END_REF] another document similarity detection and classification system to process e-mail messages. It can be thus used for example for spam detection and removal. The system is based on comparison of message fingerprints stored as their hash representations. The message is classified only if a significant resemblance level exceeds a predetermined threshold.

In [START_REF] Nagatsuka | Document classification system and method for classifying a document according to contents of the document[END_REF], another document classification system is proposed. The authors have done a morphological analysis in order to keep the significant words from the document. The feature vector creation is based on the classical vector space models. The classification utilizes three different techniques, namely a Chi-square test, discriminant analysis and cluster analysis.

Another interesting system for document classification is introduced by Rocha et al. in [START_REF] Rocha | Semantic annotation and classification in practice[END_REF]. It uses semantic information contained in the linked data to improve the document classification score. The classification itself uses both word and linked data based features. The system is used by a telecommunication operator for English and Italian document classification.

Technical Background

The following sections details our feature set, multi-label document classification approach and confidence measure methods. These methods are integrated into the proposed system.

Feature Set & Classification

The feature set is created according to Brychcín et al. [START_REF] Brychcín | Novel unsupervised features for Czech multi-label document classification[END_REF]. They are used because the authors experimentally proved that the additional unsupervised features significantly improve the classification.

-Words -Occurrence of a word in a document. Tf-idf weighting is used.

-Stems -Occurrence of a stem in a document. Tf-idf weighting is used.

-LDA -Latent Dirichlet Allocation topic probabilities for a document.

-S-LDA -Stem-based Latent Dirichlet Allocation topic probabilities for a document. -HAL -Occurrence of a Hyperspace Analogue to Language cluster in a document. Tf-idf weighting is used. -COALS -Occurrence of a Correlated Occurrence Analogue to Lexical Semantic cluster in a document. Tf-idf weighting is used.

For multi-label classification, we use an efficient approach presented by Tsoumakas et al. in [START_REF] Tsoumakas | Multi-label classification: An overview[END_REF]. This method employs n binary classifiers C n i=1 : d → l, ¬l (i.e. each binary classifier assigns the document d to the label l if the label is included in the document, ¬l otherwise). The classification result is given by the following equation:

C(d) = ∪ n i=1 : C i (d) (1)
The Maximum Entropy (ME) [START_REF] Berger | A maximum entropy approach to natural language processing[END_REF] model is used for classification. For implementation of the multi-label classifier we used Brainy [START_REF] Konkol | Brainy: A machine learning library[END_REF] implementation of Maximum entropy classifier. It has been chosen mainly because of our experience with this tool.

Confidence Measures

As already stated, the confidence measure is used as a post-processing of the classification to identify whether the result is correct or not. In this work, we combine two measures based on the posterior class probability by a multi-layer perceptron classifier as already presented in [START_REF] Kral | Confidence measure for Czech document classification[END_REF].

Posterior class probability approaches The output of an individual binary classifier C i is the posterior probability P (L|F ), where L ∈ {l, ¬l} represents a binary class and F represents the feature vector created from the text document d.

We use two different approaches. The first approach, called absolute confidence value, assumes that higher recognition score confirms the classification result. For the correct classification L the following two equations must be satisfied:

L = arg max L (P (L|F )) (2)
P ( L|F ) > T 1 (3)
The second approach, called relative confidence value, computes the difference between the l score and the ¬l score by the following equation:

∆P = abs(P (l|F ) -P (¬l|F )) (4)
Only the classification results with ∆P > T 2 are accepted. This approach assumes that the significant difference between l and ¬l classification scores confirms the classification result.

T 1 and T 2 are the acceptance thresholds and their optimal values are set experimentally.

Composed supervised approach Let R abs and R rel be the scores obtained by the absolute confidence value and relative confidence value methods, respectively. Let variable H determine whether the document is classified correctly or not. The Multi-Layer Perceptron (MLP) classifier which models the posterior probability P (H|R abs , R rel ) is used to combine the two partial measures in a supervised way.

The following MLP topology was set experimentally using a small development corpus (containing 100 randomly chosen examples) as the best one: two input nodes (R abs and R rel ), ten nodes in the hidden layer and two output nodes (classes correct / not correct).

System Architecture

The system is designed as a web application and Java Vaadin 7 4 framework is used to facilitate the implementation. The open source search engine SOLR5 is used for text documents storage, indexing and retrieval. This platform is used because it provides many advanced features in text processing field and is freely available even for commercial usage. The PostgreSQL6 database management system is used for storing the classification models, results and statistics. The machine learning algorithms are implemented using the Brainy [START_REF] Konkol | Brainy: A machine learning library[END_REF] library.

The system has a modular architecture and is composed from three main modules. Its architecture is depicted in Figure 1.

M1 -Data Acquisition/Preparation Module

This module is used to manage all main data flows in the SAPKOS system. It is connected with the SOLR engine which serves as the source of text documents.

Its main goal is to load/store text (annotated/unannotated) documents to/from the system. The annotated documents are then used for training of the model in the classification module while the unannotated documents are used for classification. The documents from the SOLR engine are loaded to the system as xml files. The exact form of the input documents is a priori not known and therefore, the highly configurable interface is available.

There are several scenarios to load the data into the system. The basic one uses directly the annotated documents loaded from the search engine. Another possibility is loading documents from the engine and annotate them manually in the system. This module also allows exporting the loaded documents to xml format, setting/adjusting the assigned categories and importing the documents back into the system.

Figure 2 shows the configurable interface for loading the documents using the SOLR server. This figure shows that a user can set several filters to chose only the appropriate documents (e.g. date, priority, source media, keywords, text, etc.).

M2 -Classification Module

This module is the main part of the system. It provides three separate tasks: 1) feature construction; 2) model training and 3) classification itself. These tasks are described in details in Section 3.

The feature construction is a shared task which is used both for training and classification. During training, a classification model is built using annotated text documents provided by the module M1. This model is stored to the PostgreSQL database. During classification (testing) the learned model is used to assign topics to unannotated documents. The classification results are then stored into the database.

Note that the confidence measure method is also integrated to this module after the classification task.

M3 -Data Analysis Module

This module is used to prepare statistical data from the processed documents and to analyse, compare and visualize the statistics. There is also a possibility to export the data to a spreadsheet. As already stated, this analysis is very important to automatically compare the ČTK production with popular Czech media. The results of this analysis will be used to adapt the ČTK production to better correspond to the today's market requirements. Figure 3 shows one example of comparison of the ČTK production with one Czech media. This figure shows that the document topics frequencies are almost the same 7 . Therefore, the ČTK production in this example corresponds to the market needs. This graph is generated by JFreeChart 8 . Fig. 2. Configurable dialogue window for data selection and loading from the SOLR search engine. The main field captions (from above) with English translations: "Datum vydání od/do" (publication date from/to), "Priorita od/do" (priority from/to), "Zdroj" (source media), "Servis" (service), "Klíčová slova" (keywords), "Hledaný text" (searched text) and "Délka zprávy od/do" (length of the document from/to); the button captions: "Hledat" (search), "Zobrazit data" (show the data) and "Zrušit (cancel). The large text area contains a query example in the SOLR syntax.

System Evaluation

We use two different real ČTK datasets to evaluate the performance of the proposed system. The first one is relatively small and is chosen because it Fig. 3. Comparison example of the ČTK production with one Czech media. The main captions with English translations: " Četnost kategorií" (category frequencies), "Kategorie" (categories) and "Počet" (number). The horizontal axis depicts the categories while the vertical one shows the number of the documents containing the category.

is freely available for research purposes (download at http://home.zcu.cz/ ~pkral/sw/). Therefore, it is possible to compare our results with other stateof-the-art methods.

The second corpus is significantly larger. Unfortunately, this dataset is not publicly available yet, due to the commercial politics of the ČTK . However, we would like to show on this corpus that our system performance is independent to the size of the dataset.

Evaluation on the Free Czech ČTK Document Corpus

This corpus contains 11,955 documents composed of 2,974,040 individual words. The documents are manually annotated with at least one of 37 categories. Figure 4 shows the distribution of labels depending to the document numbers.

Table 1 shows the document classification scores in two cases: without and with the confidence measure. The five-folds cross validation procedure is used (20% of the corpus is reserved for testing and the remaining part for training of our system) with the standard Precision (Prec), Recall (Rec) and F-measure (F-mes) [START_REF] Powers | Evaluation: From precision, recall and f-measure to roc., informedness, markedness & correlation[END_REF] metrics.

The first line of the table shows the classification scores without the confidence measure module. The two following lines depict the results of the absolute and relative confidence value methods. The optimal values of the thresholds T 1 and T 2 are set to 0.99. These values are found experimentally using small development corpus. The last line shows the results of the composed supervised approach based on the MLP classifier.

Number of documents

Number of labels

It is clearly visible that every individual confidence measure method improves the classification results. The improvement is then further significantly increased when the MLP is used to combine the two measures. Therefore, this approach is integrated into our system.

Evaluation on the Private Czech ČTK Document Corpus

This corpus contains 120,000 documents: 20,000 documents is used for training and the testing part is composed of the remaining 100,000 documents. The num-ber of classes is similar as in the previous case, i.e. 37 ones. The distribution of the document labels is also similar. The documents are collected from one Czech newspaper for a period of 6 months. Table 2 details the results of this experiment. This table clearly shows that the document number does not influence the recognition accuracy. When the best supervised confidence measure is used, the resulting F-measure is reduced only slightly. This experiment confirms high robustness of our system. In this paper, we presented a novel experimental multi-label document classification and analysis system called SAPKOS. The system will be used by the Czech news Agency to save human resources in the task of annotation of newspaper articles with the topics. Another important functionality is an automatic comparison of the ČTK production with the newspaper articles from popular Czech media. This analysis will be used to adapt ČTK production to better correspond to the today's market requirements. An interesting contribution is that, to the best of our knowledge, no other automatic Czech document classification system exists. The system accuracy is very high (F-measure 98.3% on the free and 97.8% on the private ČTK document corpus). These results are obtained due to the unique system architecture which integrates a maximum entropy based classifier with the novel confidence measure.

The current version of the system is not optimized to the speed and memory requirements. Therefore, our first perspective consists in the optimization of the system in this way. The system is adapted to the Czech language. However, the majority of the methods is language independent. Therefore another perspective consists in the adaptation of the system to the other languages.

Fig. 1 .

 1 Fig. 1. SAPKOS system architecture

Fig. 4 .

 4 Fig. 4. Distribution of label numbers for documents in the ČTK corpus

Table 1 .

 1 Classification results without / with the proposed confidence measures [in %] for the free Czech ČTK document corpus

	Confidence Measure Approach	Prec Rec F-mes
	-	89.0 75.6	81.7
	Absolute confidence value	93.8 78.3	85.3
	Relative confidence value	94.3 79.4	86.2
	Composed supervised approach (MLP)	97.4 99.3	98.3

Table 2 .

 2 Classification results without / with the proposed measures [in %] for the private Czech ČTK document corpus

	Confidence Measure Approach	Prec Rec F-mes
	-	85.9 68.0	77.1
	Absolute confidence value	92.8 74.4	82.6
	Relative confidence value	93.4 75.6	83.6
	Composed supervised approach (MLP)	97.1 98.5	97.8
	6 Conclusions and Future Work

http://www.ctk.eu

https://vaadin.com/home

http://lucene.apache.org/solr/

http://www.postgresql.org/

red colour= ČTK production, blue colour=Czech newspaper Lidové noviny

http://www.jfree.org/jfreechart/

Acknowledgements

This work has been partly supported by the project LO1506 of the Czech Ministry of Education, Youth and Sports. We would like also to thank Czech New Agency ( ČTK) for support and for providing the data.


Abdulrazak Yahya Saleh
email: abdulrazakalhababi@gmail.com

Siti Mariyam

Haza Nuzly

Abdull Hamed

Multi-Objective Differential Evolution Of Evolving Spiking Neural Networks For Classification Problems

Keywords: Differential Evolution, Evolutionary algorithms, Evolving spiking neural networks, Multi objective Optimization. 1

Spiking neural network (SNN) plays an essential role in classification problems. Although there are many models of SNN, Evolving Spiking Neural Network (ESNN) is widely used in many recent research works. Evolutionary algorithms, mainly differential evolution (DE) have been used for enhancing ESNN algorithm. However, many real-world optimization problems include several contradictory objectives. Rather than single optimization, Multi-Objective Optimization (MOO) can be utilized as a set of optimal solutions to solve these problems. In this paper, MOO is used in a hybrid learning of ESNN to determine the optimal pre-synaptic neurons (network structure) and accuracy performance for classification problems simultaneously. Standard data sets from the UCI machine learning are used for evaluating the performance of this multi objective hybrid model. The experimental results have proved that the multiobjective hybrid of Differential Evolution with Evolving Spiking Neural Network (MODE-ESNN) gives better results in terms of accuracy and network structure.

Introduction

Classification of patterns is vital to several data mining processes. Classification is one of the most commonly obverse processing tasks for a decision support system [START_REF] Ahmed | Improved SpikeProp for Using Particle Swarm Optimization[END_REF]. There are many areas in life which need classification such as medical diagnoses, medicine, science, industry, speech recognition and handwritten character recognition. Among feasible classifiers, Spiking neural network (SNN) plays an essential role in biological information processing [START_REF] Gerstner | Spiking neuron models: Single neurons, populations, plasticity[END_REF].

Although there are many models of SNN, Evolving Spiking Neural Network (ESNN) is widely used in many recent research works. ESNN has several attractive advantages [START_REF] Schliebs | Integrated feature and parameter optimization for an evolving spiking neural network: Exploring heterogeneous probabilistic models[END_REF] including: simplicity, efficiency, trained by a fast one-pass learning algorithm. The evolving nature of model can be updated whenever new data becomes accessible with no requirement to retrain earlier existing samples. However, ESNN model is affected by its choice of parameters. The right selection of parameters allows the network to evolve towards reaching the best structure thus guaranteeing the best output. Of all the issues that require exploration in ESNN determining the optimal number of pre-synaptic neurons for a given data set is the most important one [START_REF] Hamed | Novel Integrated Methods of Evolving Spiking Neural Network and Particle Swarm Optimisation[END_REF]. For these reasons, achieving an optimized trade-off between accuracy and the network structure is needed to find the best combination of parameters and pre-synaptic neurons.

Optimization has been used for enhancing ESNN algorithm. Multi-Objective Optimization (MOO) can be utilized as a set of optimal solutions to solve these problems. Every MOO solution appears to be a new trade-off between the objectives. The key objective of MOO is to improve ESNN optimal solutions of both structure and classification accuracy. MOO approach is preferred to traditional learning algorithms for the following reasons. First, as a result of using MOO a good performance of these learning algorithms can be achieved [START_REF] Abbass | Speeding up backpropagation using multiobjective evolutionary algorithms[END_REF]. Second, various objectives are taken into consideration in the generation of multiple learning models. For example: accuracy, complexity [START_REF] Igel | Multi-objective Model Selection for Support Vector Machines[END_REF][START_REF] Jin | Multi-objective machine learning[END_REF][START_REF] Jin | Neural network regularization and ensembling using multi-objective evolutionary algorithms[END_REF], interpretability and accuracy [START_REF] Jin | Evolutionary Multi-objective Optimization for Simultaneous Generation of Signal-Type and Symbol-Type Representations[END_REF], multiple error measures [START_REF] Fieldsend | Pareto evolutionary neural networks[END_REF]. Third, it is superior to building learning ensembles to use models [START_REF] Jin | Neural network regularization and ensembling using multi-objective evolutionary algorithms[END_REF][START_REF] Abbass | Pareto neuro-evolution: Constructing ensemble of neural networks using multi-objective optimization[END_REF][START_REF] Chandra | DIVACE: Diverse and accurate ensemble learning algorithm[END_REF]. The important goal of MOO algorithm is to find a set of solutions from which the best one is chosen. Based on Tan et al. [START_REF] Tan | Evolutionary algorithms with dynamic population size and local exploration for multiobjective optimization[END_REF], the ability of evolutionary algorithms (EAs) to search for optimal solutions gives it the priority to be selected in MOO problems. EAs have the ability to explore different parts of the related algorithm in the optimal set because of the population-based algorithms.

In spite of the fact that one Multi-Objective Evolutionary Algorithms (MOEAs) was used and only for SpikeProp learning, most of the related work which use Multi-Objective Genetic Algorithms (MOGAs) is by Yaochu Jin et al. [START_REF] Jin | Evolutionary multi-objective optimization of spiking neural networks[END_REF]. Both classification performance and connectivity of SNN with latency coding are optimized by MOGA. During optimization, both delay and weight between the two neurons are evolved. Furthermore, they minimize the classification error in percentage or the root mean square error for optimizing performance, and minimize the number of connections or the sum of delays for connectivity to explore the objectives influence on the connectivity and performance of SNNs. This is a very motivating finding. However, more experiments must be performed to verify this observation. No conclusion can be made on classification error should be used for classification. They have also revealed that complexities of the SNNs are equivalent, when the number of connections or the sum of delays is utilized to optimize connectivity. This study needs improvement to enhance its promising results.

Unlike previous study mentioned earlier in [START_REF] Jin | Evolutionary multi-objective optimization of spiking neural networks[END_REF] and other single objective studies [START_REF] Saleh | A Novel hybrid algorithm of Differential evolution with Evolving Spiking Neural Network for pre-synaptic neurons Optimization[END_REF][START_REF] Saleh | Parameter Tuning of Evolving Spiking Neural Network with Differen-tial Evolution Algorithm[END_REF], this paper deals with an improved method to obtain simple and accurate ESNN. The proposed method evolves toward optimal values defined by several objectives with model accuracy and ESNN's structure to improve performance for classification problems. The remaining parts of this paper are organized as follows: Methods including: Evolving Spiking Neural Network, Multi Objective Differential Evolution are presented in section 2. In addition, section 3 clarifies the proposed method multi objective hybrid of Differential Evolution with Evolving Spiking Neural Network (MODE-ESNN) used in this paper, Experimental design is discussed in section 4, section 5 explains in detail the results and discussion, and finally, section 6 concludes the paper with future works.

Methods

This section reviews the vital foundation of evolving spiking neural network(ESNN) and discusses the evolutionary algorithms that have been utilized for enhancement. In the first part, an introduction of ESNN, neuron coding, learning method and ESNN design, and its algorithm is presented. The second part focuses on the algorithms which are used for improvement of ESNN. The concepts and methods of multi objective optimization (MOO) are highlighted. After that, the literature focuses on the working of Multi-Objective Differential Evolution(MODE) which is used to improve and enhance the performance of classification .

1 Evolving Spiking Neural Network ( ESNN)

Currently, several enhancements of SNN have been proposed. Wysoski improved one of these new models known as Evolving Spiking Neural Network (ESNN) [START_REF] Wysoski | On-line learning with structural adaptation in a network of spiking neurons for visual pattern recognition[END_REF]. Generally, ESNN used the principles of evolving connectionist systems (ECOS) where neurons are created incrementally [START_REF] Kasabov | Dynamic evolving spiking neural networks for on-line spatioand spectro-temporal pattern recognition[END_REF][START_REF] Schliebs | Evolving spiking neural network-a survey[END_REF]. ESNN can learn data gradually by one-pass propagation of the data through creating and merging spiking neurons [START_REF] Kasabov | NeuCube: A spiking neural network architecture for mapping, learning and understanding of spatio-temporal brain data[END_REF] making it possible to attain very fast learning in an ESNN [START_REF] Kasabov | Evolving spiking neural networks and neurogenetic systems for spatio-and spectro-temporal data modelling and pattern recognition[END_REF]. The learning of ESNN has many good advantages as it can be applied incrementally, is adaptive and theoretically 'lifelong'. Therefore, the system can learn any new pattern via creating new output neurons, connecting them to the input neurons and merging with the similar ones [START_REF] Kasabov | Evolving spiking neural networks for personalised modelling, classification and prediction of spatio-temporal patterns with a case study on stroke[END_REF]. This model stands on two principles: possibility of establishment of new classes and the merging of the similarities. The encoding method which is used for ESNN is the population as explained in [START_REF] Bohte | Error-backpropagation in temporally encoded networks of spiking neurons[END_REF]. The population distributes a single input value to multiple input neurons denotes as M. Each input neuron holds a firing time as input spikes. Firing times can be calculated which represent the input neuron e using the intersection of Gaussian function. Thorpe model [START_REF] Thorpe | How can the human visual system process a natural scene in under 150ms? experiments and neural network models[END_REF] is similar to the Fast Integrate and Fire Model used in our paper. Thorpe's model shows that the earliest spikes received by a neuron will get a better weight depending on the later spikes. The Post-Synaptic Potential (PSP) will fire and become disabled only if it beats the threshold value. The computation of PSP of neuron e can be shown in Equation 3,

( ) 0 * { order f e if fired otherwise fe Mod e w    (3)
where W fe is considered as the weight of pre-synaptic neuron f; Mod e is the parameter factor of modulation with an interval of [0,1] and order (f) denotes the rank of spike emitted by the neuron. The order (f) starts with 0 if it spikes first among all pre-synaptic neurons and increases according to the firing time. Moreover, each training sample creates a new output neuron in the One-pass Learning algorithm. ESNN training steps can be understood from algorithm 1 as illustrated above in section 2. Fig. 1 depicts a simplified architecture of ESNN model which was explained in more detail in [START_REF] Schliebs | Integrated feature and parameter optimization for an evolving spiking neural network: Exploring heterogeneous probabilistic models[END_REF].

Fig. 1 A simplified architecture of ESNN [START_REF] Hamed | String Pattern Recognition Using Evolving Spiking Neural Networks and Quantum Inspired Particle Swarm Optimization[END_REF] The training starts with initialization of three ESNN parameters -modulation factor (Mod), proportion factor (C) and similarity value (Sim) in the interval [0, 1]. Mod is the modulation factor of the Thorpe neural model. The firing threshold  is calculated as based on propor- tion factor (C) with a value between [0, 1]. As the training process continues, every sample produces an output neuron. The similarity of output neurons is calculated according to the Euclidean distance between the weight vector of the neurons. The parameter Sim controls the similarity distance. In the training stage, the output neuron stores the computed weight of all pre-synaptic neurons, a threshold value to determine when the output neuron will spike and the class label to which the input sample belongs. In the testing stage, the multiple pre-synaptic neurons encode each testing sample to spikes. After that, the PSP of the output class neurons is calculated. Once the neuron attains definite amount of spikes and the PSP exceeds the threshold value, it fires an output spike and becomes disabled. The testing sample belongs to the output class defined by the output neuron that fires first among all output neurons.

2 Multi-Objective Differential Evolution (MODE)

This section discusses MOO and Evolutionary algorithms. First, the concept of MOO is introduced followed by a discussion of the essential points of MOO methods. Then, the concept of the Evolutionary Algorithm (EA) mainly Differential Evolution(DE) is clarified. After that, a brief discussion of the MOEA for DE enhancement is explained. Finally, the related works of this study are discussed in-depth.

2.1 Multi-Objective Optimization (MOO)

MOO adds a new concept to the previous concepts of optimization. The process of systematically optimizing a set of objective functions at the same time is known as multi objective optimization (MOO) or vector optimization [START_REF] Marler | Survey of multi-objective optimization methods for engineering[END_REF]. According to Lu [START_REF] Yen | Dynamic multiobjective evolutionary algorithm: adaptive cell-based rank and density estimation[END_REF], the optimal solutions obtained by individual optimization of the objectives are not a feasible solution to the multi-objective problem. Most practical optimization problems need the synchronized optimization of more than one objective function.

Methods of MOO Algorithms

Typical methods aggregate the objectives into one objective function by using decision making before search. Conversely, the parameters of this function are assorted by the optimizer systematically. A number of optimization runs with various parameter settings are conducted with the intention of reaching a set of solutions. Essentially, this process is independent of the primary optimization algorithm. A number of examples of this class of techniques are the weighting method [START_REF] Cohon | Multiobjective programming and planning[END_REF], the constraint method [START_REF] Cohon | Multiobjective programming and planning[END_REF], goal programming [START_REF] Steuer | Multiple criteria optimization: theory, computation, and application[END_REF]. In exchange for the different methods, these three common methods are briefly discussed here.

a. Weighting Method

All objectives are multiplied by weighting coefficient (w i ) After that, all new functions are added together to get a single cost function. Finally, Single Objective (SO) method can be used to solve this new single cost function . Mathematically, the latest function is written as

    1 I ii i f x w f x    (4)
where 01 i w 

    h y f x f x  (5) Subject to   () i i i e x f x   ,   1, i I i h    , f xX 
The lower bounds, i  , are the parameters that are diverse by the optimizer to find multiple optimal solutions.

c. Goal Programming Method

According to Philipson and Ravindran [START_REF] Philipson | Application of goal programming to machinability data optimization[END_REF], Goal Programming Method is considered as the most well-known method of solving MOPs. It was initially enhanced by [START_REF] Charnes | Management models and industrial applications of linear programming[END_REF] and [START_REF] Ijiri | Management goals and accounting for control[END_REF]. This method depends on ranking the goals as created by the designer. Finally, the SO function is considered as the minimization of the deviations from these goals.

EA Algorithms

EAs have been utilized to optimize ESNNs, which have been used to solve the problems of optimization learning. The right classification measures are defined by the number of true positives (TPs) and the number of true negatives (TNs). Additionally, the misclassifications made by the classifier are determined by the number of false positives (FPs) and the number of false negatives (FNs). Many EAs types have been used for optimization, but DE algorithm is used in this study for many reasons, which will be explained in sections below:

a. Differential Evolution (DE)

In global optimization, DE is considered as one of the mainly powerful tools of global optimization in EAs [START_REF] Abbass | PDE: a Pareto-frontier differential evolution approach for multi-objective optimization problems[END_REF]. Compared to some other competitive optimizers, DE has several strong advantages: it is much simpler to implement, has much better performance, has a small number of control parameters and has low space complexity [START_REF] Das | Differential evolution: A survey of the state-of-theart[END_REF]. DE, which belongs to the EAs, uses three steps: mutation, crossover and selection. In the stage of initialization, a population of individual candidates, each of dimension Nvariables (number of decision variable in the optimization problem), is randomly generated over the feasible region. A typical value of an individual is about 5-10 times Nvariables in order to guarantee that DE has enough to work with. The fitness of every individual candidate is evaluated. Out of these individual candidates, one is randomly selected as the target candidate. The DE algorithm contains three parameters: size of population Nindividuals, mutation constant F and crossover constant CR.

The MODE Algorithm

According to Schaffer in [START_REF] Schaffer | Multiple Objective Optimization with Vector Evaluated Genetic Algorithms[END_REF], Vector evaluated genetic algorithm (VEGA) is considered as the first multi-objective proposed in the literature. VEGA is an adapted single-objective genetic algorithm with a modified selection method. Many literature surveys found provide more detail about the history and enhancement of multi objective with GA like Zitzler et al. [START_REF] Zitzler | A tutorial on evolutionary multiobjective optimization[END_REF] and Deb [START_REF] Deb | Scalable test problems for evolutionary multiobjective optimization[END_REF]. Unlike GAs, where binary encoding can be utilized, DE solutions are coded with real values. More recently, evolutionary algorithms have also been adapted to solve optimization problems with multiple conflicting objectives by approximating the Pareto set in such problems such as in [START_REF] Qasem | Memetic elitist pareto differential evolution algorithm based radial basis function networks for classification problems[END_REF]. A complete tutorial on evolutionary multiobjective optimization with DE can be found in [START_REF] Mezura-Montes | Multi-objective optimization using differential evolution: a survey of the state-of-the-art[END_REF].

The Proposed Hybridization of Multi-Objective DE based ESNN (MODE-ESNN)

This section discusses the proposed algorithm called MODE-ESNN which is a multiobjective optimization approach for ESNN training. This algorithm will simultaneously determine the ESNN structure (pre-synaptic neurons) and its corresponding model parameters by treating this problem as a multi-objective optimization problem. In MODE_ESNN, the pre-synaptic neuron of ESNN is represented as a candidate. DE and MOO are combined to carry out fitness evaluation and mating selection schemes.

MODE_ESNN starts by collecting, normalizing and reading the dataset. The maximum number of iterations are then set and the candidate size is computed. In addition, the presynaptic neurons and parameters of ESNN are determined randomly. A population of the hybrid method is then generated and initialized. Every candidate is evaluated for each iteration based on Differential Evolution algorithm. The proposed method stops after the maximum iterations is reached. Fig. 2 describes the schematic representation of MODE_ESNN.

Fig. 2 Schematic representation of the proposed MODE_ESNN

The main procedure of proposed MODE-ESNN is presented below: .( )

t t t t i r r r V X F X X     (6)
the r 1 ,r 2 and r 3 are randomly selected indexes and r 1 ,r 2 ,r 3 [ 1,2,….., Nindividuals ]. F is a real number to control the amplification of the difference vector

() tt rr XX  Range of F is in [0, 1] 0<F<=1 II.
Perform crossover and selection for each candidate. The target individual candidate is mixed with the mutated vector, using the following scheme, to yield the trial vector u by updating the presynaptic neuron and ESNN parameters vector simultaneously using Equation 71

{

t ij t ij v rand j CR or j randn i t ij x rand j CR or j randn i u              (7)
where j=1,2,….., Nvariables, rand ( j )  [0,1] is the j th evolu- tion of a uniform random generator number  r. CR  [ 0,1 ] is the crossover probability constant which has to be determined previously by the user rand ( i ) . If not, no new parent candidate would be produced and the population would not alter. f) End while g) Sort population P(t+1) according to their fitness value. h) t=t+1 5. End while

1 t i v  ( 1,

Experimental Design

This section presents the experiments of study on hybrid learning of ESNN network based on multi-objective method. Many techniques can be used for validation, but kfold cross-validation is used in this paper. The advantage of k-fold cross-validation over hold-out validation is that all observations are used for both training and testing, and each observation is used for testing exactly once. 10-fold cross-validation is commonly used [START_REF] Do | Analyzing microarray gene expression data[END_REF]. In order to evaluate the effectiveness of the proposed method, a detailed empirical study is carried out on seven different data sets which is explained in detail below.

Data Preparation

Several real-world data sets from UCI repository are used in this study to represent some of the most challenging problems in machine learning. Many researchers have used these data sets as a benchmark in validating the performance of their algorithms. The key characteristics of these problems and their associated learning tasks are summarized in Table 1. The performance of the algorithms is evaluated by conducting analysis on ten evaluations. In this experiment, the evolutionary process of the proposed algorithm is analyzed and the performance is evaluated accordingly. The objective of this evaluation is to establish the effectiveness of the proposed method in designing ESNN network. This involves both the training of the ESNN network and the evolved structure (pre-synaptic neurons) of the ESNN network. The capabilities of the proposed method have been investigated through a comparison with each other that has been applied for classification problems. In order to evaluate the classification performance of the proposed method, the comparisons are conducted and compared with the standard ESNN, DE-ESNN, DEPT-ESNN and other data mining methods. The results of all proposed methods in terms of all measures are presented in Table 3, Table 4 and Table 6. In these tables, the best results are highlighted in bold font. The results for all data sets involved are analyzed based on structure of ESNN (number of pre-synaptic neurons), parameters values (Mod, Sim and Threshold), sensitivity (SEN), specificity (SPEC), geometric means (GM), negative predictive value (NPV), positive predictive value (PPV), average site performance (ASP) and classification accuracy (ACC) for all data sets. The results of the proposed methods for each data set are analyzed and presented in the following section.

Results and Discussion

This section displays the results of the proposed method MODE-ESNN. The results of MODE-ESNN are measured in terms of number of pre-synaptic neurons, Mod, Sim and Threshold. The experiments are run 10 times in the training and testing for all data sets, respectively as shown in Tables 3. The results of MODE-ESNN have revealed the values of pre-synaptic neurons and parameters (Mod, Sim and Threshold) for all data sets. From Table 3, the values of different parameters: Mod, Sim and Threshold are illustrated for all data sets. In addition, the results of pre-synaptic neurons also shown simultaneously because the multiobjective technique is used here. Table 3 shows the average results of the proposed algorithm in terms of parameter Mod. The Mod parameter importance comes from its representation for the connection weight in ESNN model. If a high value was selected, it means most weights will have a connection value which reflects a well presented weight patterns according to their output class. On the contrary, selecting too low values leads to ending up with most connections assigned with the zero weight value due to the nature of weight computation. The findings show the ability of the proposed method to produce the optimum values of Mod parameter which is different from data set to another. The nature of data set can affect the value of parameter. This clear effect is due to the correlation between parameter optimization and classification accuracy as integrated using the known Wrapper approach. Knowledge discovery from these results of Mod parameter shows that there is an important impact of high values of decision of the output classes of instances. 3 and Fig. 3. It is noticed that no specific value can be applied to all problems because each problem has its own combination of parameters. Generally, each data set requires specific analysis to understand the problem it represents. Fast decision making demands a comprehensive understanding. Knowledge discovery from these results shows that the high values of Sim parameter leads to establishing better network architecture. This is due to the effect of the parameter Sim in controlling the merge rate of output neurons. Fig. 3 shows that the average results of the proposed algorithm in terms of parameter Threshold. The importance of the Threshold parameter is in its function of controlling the PSP threshold. The different values of Threshold parameter from data set to another for the proposed method reflects the influence of nature of data set and the hybrid method which control the process of the algorithm. Knowledge discovery from these results of Threshold parameter shows that there is a main influence of parameter values of making the decision. Higher values means more spikes and time for this decision. However, if there are smaller values, few spikes are enough to fire an output spike and determine the class of instance.

Moreover, the performance of MODE_ESNN is evaluated in terms of sensitivity, specificity, geometric mean and accuracy. Table 4 illustrates the results of MODE_ESNN in terms of SEN, SPE, GM, NPV, PPV, ASP and ACC for all data sets, respectively.

From Table 4, sensitivity gives high results for Appendicitis, Haberman, Heart, Hepatitis and Iris data sets as well as the specificity values of Ionosphere and Iris data sets. Similarly, the geometric mean gives high results in Iris data set. The observation from the results in Tables 4 has indicated that MODE_ESNN produced high results on accuracy for four out of seven data sets. Additionally as in Table 4, MODE_ESNN performs high NPV on all data sets except Haberman. For positive predictive value (PPV), MODE_ESNN generates high values for all data sets except Hepatitis and Liver. However, ASP gives the highest results in all data sets except Liver. For negative predictive value (NPV), MODE_ESNN generates high values for Heart, Ionosphere and Iris data sets.

As shown below, the proposed method has been evaluated with several data mining algorithms by using KEEL tool. KEEL software is an open source Data Mining tool widely used in research and real life applications. More details are presented in [START_REF] García | A Data Mining Software Package Including Data Preparation and Reduction: KEEL[END_REF][START_REF] Derrac | Using KEEL software as a educational tool: A case of study teaching data mining[END_REF]. The methods are steady-state genetic algorithm for extracting fuzzy classification rules from data(SGERD),CO-evolutionary rele Extractor (CORE), hierarchical decision rules (HIDER) and tree analysis with randomly generated and evolved trees (TARGET) [START_REF] Alcalá | Keel data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework[END_REF]. Additional details concerning these algorithms are provided in Table 5.

Analysis are accomplished on five benchmark data sets from UCI repository (Appendicitis, Haberman, Ionosphere, Iris and Liver). Results of Accuracy performance are shown in Table 6.

Table 5: Description of KEEL data mining algorithm used [START_REF] Alcalá | Keel data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework[END_REF] Algorithm Description CORE A CO-evolutionary algorithm which employs as a fitness measure a combination of the true positive rate and the false positive rate.

SGERD

A steady state GA which generates a prespecified number of rules per class following a GCCL approach HIDER A method which iteratively creates rules that cover randomly selected examples of the training set.

TARGET

A GA where each chromosome represents a complete decision tree.

As in Table 6, the proposed method MODE-ESNN outperforms other data mining methods for all data sets in terms of accuracy performance.

Conclusion and Future Works

A hybrid MODE-ESNN method was proposed in this paper to determine the optimal number of pre-synaptic neurons as well as the parameters for a given dataset simultaneously. A comparative study has been conducted between MODE-ESNN and ESNN, DE-ESNN and DEPT-ESNN with other data mining methods to show the performance improvement of ESNN. Both MODE-ESNN and other methods have been used to perform the classification on standard data sets. The results show that MODE-ESNN is able to classify data set with mostly better accuracy than the other algorithms. Moreover, MODE-ESNN uncovered the optimum parameters of ESNN which is considered important for good accuracy. Addi-tionally, MODE-ESNN mostly shows better results in sensitivity, PPV, ASP and other measurements. For future work, enhancement of the hybridization of a multi differential evolution MODE-ESNN with an ESNN algorithm may be a good trend to be explored to obtain more results. In another direction, these hybrid multi objective evolutionary algorithms with ESNN motivate researchers to investigate the usefulness of integration of other new meta heuristic algorithms to enhance ESNN.

  Constraint Method depends on transforming k-1 of the k objectives into constraints. The remaining objective, which can be selected subjectively, is the objective function of the resulting SO: Maximize

23

 23

Fig. 3 A

 3 Fig. 3 A comparison of the MODE-ESNN for 10-fold cross-validation in terms of parameter analysis

  1. Generate an initial population P(t) at t=0 of size N, where each candidate represents ESNN and where t is the number of the actual iteration.

	2. Use the random selection technique for obtaining the initial values for both
	pre-synaptic neurons, which are considered as the structure of ESNN and the
	parameters of the model.
	3. Determine candidate dimension and assign DE parameters values.
	4. While stopping criterion is not met do:
	a) Use ESNN algorithm to find the candidate fitness.
	b) Achieve results of pre-synaptic neurons and ESNN parameters.
	c) evaluate the candidates of population Q(t) according to its accuracy value.
	d) Determine the best candidate according to the best parents' values.
	e) While size of population P(t+1) is < N do:
	I. Calculate mutated candidates according to Equation 6
	1
	1	2	3

Table 1 :

 1 Description of data setsInitially, the data set has been divided randomly into ten subsets of equal size. One subset is used as the testing data set, and the other nine subsets are used as the training data sets. The training and testing processes are repeated so that all the subsets are used as a testing data set. The training process of the hybrid MODE-ESNN is explained in section 3. There are many important parameters which can control the result of training in MODE-ESNN. The various parameter settings of the proposed method are tabulated in Table2.

		Dataset	Attributes	Classes	Samples
		Appendicitis	7	2	106
		Haberman	3	2	306
		Heart	13	2	297
		Hepatitis	19	2	155
		Ionosphere	34	2	351
		Iris	4	3	150
		Liver	6	2	345
	4.2	The Learning Phase of the Proposed MODE-ESNN

Table 2 :

 2 Parameter settings for MODE-ESNN algorithms

	Parameter	MODE-ESNN
	Population size	30
	Maximal number of iterations	1000
	(Mutation) constant F	0.9
	Crossover constant CR	0.8
	Dimensionality of problem	2

Table 3 :

 3 Results of number of pre-synaptic neurons and parameters (Mod, Sim and Threshold) for MODE-ESNN

	Data set		Pre-synaptic neurons	MOD	SIM	Threshold
	Appendicitis	Mean SD	12.00 0.00	0.9535 2.3406	0.4196 5.8500	0.5518 1.0000
	Haberman	Mean SD	10.00 9.61	0.7991 0.0682	0.8234 0.0897	0.0899 0.0207
	Heart	Mean SD	45.00 11.38	0.1877 0.1848	0.5456 0.1550	0.7761 0.0263
	Hepatitis	Mean SD	23.00 8.23	0.6988 0.2128	0.5088 0.0694	0.5728 0.0492
	Ionosphere	Mean SD	46.00 9.49	0.5623 0.0792	0.5645 0.0121	0.7877 0.0815
	Iris	Mean SD	18.00 12.44	0.6005 0.1734	0.4941 0.1863	0.4824 0.2662
	Liver	Mean SD	17.00 10.40	0.6839 0.2614	0.4824 0.1531	0.7800 0.2447

Table 4 :

 4 Results of sensitivity, specificity, geometric mean, NPV, PPV, ASP and accuracy

				for MODE-ESNN
	Data set		SEN	SPE	GM	NPV PPV ASP ACC
	Appendicitis	Mean 83.50 28.30 48.61 57.90 97.80 83.68 73.00 SD 0.12 0.42 0.23 0.28 0.05 0.15 10.59
	Haberman	Mean 92.90 11.70 32.97 45.50 77.60 79.40 72.00 SD 0.15 0.22 0.18 0.15 0.03 0.07 10.09
	Heart	Mean 100 22.16 47.08 81.20 76.40 81.65 58.20 SD 0.00 6.22 0.00 0.16 0.08 0.07 4.53
	Hepatitis	Mean 99.00 36.00 59.70 52.60 55.70 70.05 54.00 SD 0.03 0.13 0.06 0.16 0.04 0.05 19.99
	Ionosphere	Mean 15.30 85.40 36.15 82.90 80.10 73.00 69.55 SD 0.13 0.11 0.12 0.07 0.17 0.16 6.30
	Iris	Mean 89.00	95.0 0	91.95	99.4 0	92.90 95.85 89.71
		SD	0.25	0.07	0.13	0.01 0.075 0.04	3.32
	Liver	Mean 44.80 55.1 49.68 66.5 49.00 59.55 50.57 SD 0.18 0.12 0.147 0.05 0.05 0.06 7.26

Table 6 :

 6 Accuracy Performance of the proposed methods with data mining methods of the results to sketch a significant ending: the classification performance of the proposed methods approximately gives best results compared to the classification performance of other methods on different datasets. Accordingly, it can be believed that the inadequate performance in some methods are affected by factors such as: noise, samples, highdimensionality, missing values, multiple classes and imbalanced classes.

	Dataset	Appendicitis	Haberman	Ionosphere	Iris	Liver
	ESNN	100	66.92	91.96	99.33	75.13
	DE-ESNN	100	76.26	96.01	99.41	100
	DEPT-ESNN	100	84.78	96.2	99.33	100
	MODE-ESNN	100	92.3	99.4	99.78	100
	CORE	89.2	76.14	59.22	96.51	63.28
	TARGET	88.47	74.07	83.19	87.03	65.08
	HIDER	92.56	76.83	97.97	97.48	73.58
	SGERD	87.52	74.07	82.36	87.03	65.08
	A careful examination

Acknowledgements

This work is supported by The Ministry of Higher Education (MOHE) under FRGS (4F347). The authors would like to thank Research Management Centre (RMC), Universiti Teknologi Malaysia (UTM) for the support in R & D, UTM Big Data Centre, Soft Computing Research Group (SCRG) for the inspiration in making this study a success.


Cesar Alberto Penz

Alberto Carlos

João Paulo Flesch

Rossetto

Carlos Alberto Flesch
email: carlos.flesch@ufsc.br

João Paulo Rossetto
email: joao.p.rossetto@embraco.com

Steady-State Inference in Performance Tests of Refrigeration Compressors Using ANN Ensemble

Keywords: neural networks ensembles, reliability improvement, steady-state inference, refrigeration compressor

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

The use of neural networks ensembles is typical in many industrial and scientific applications. The combination of a set of networks regards as a way to manage the limitations of individual networks. It is known that neural networks are susceptible to errors, but the neural networks can be combined in a way to reduce the effects of these individual errors [START_REF] Krogh | Neural networks ensembles, cross validation and active learning[END_REF][START_REF] Efron | An Introduction to the Bootstrap[END_REF][START_REF] Sharkey | Combining Artificial Neural Networks: Ensemble and Modular Multi-Net Systems[END_REF][START_REF] Winkler | Multiple experts vs. multiple methods: combining correlation assessments[END_REF][START_REF] Ahmad | Bayesian selective combination of multiple neural networks for improving long-range predictions in nonlinear process modeling[END_REF][START_REF] Zio | A study of the bootstrap method for estimating the accuracy of ANN in predicting nuclear transient processes[END_REF][START_REF] Peretti | A bootstrap neural network based heterogeneous panel unit root test: application to exchange rates[END_REF].

Reference [START_REF] Geman | Neural networks and the bias/variance dilemma[END_REF], entitled "Neural networks and the bias/variance dilemma", deals with the performance of nonparametric inference tools. An example of this type of inference tool is the multilayer perceptron neural network. The reference [START_REF] Geman | Neural networks and the bias/variance dilemma[END_REF] presents these errors as being comprised of two parts -bias and variance -and compares the features of responses obtained with nonparametric tools (model-free) to the responses obtained with model-based approaches. According to [START_REF] Geman | Neural networks and the bias/variance dilemma[END_REF] neural networks may present significant portion of variance since they may have great amount of parameters to be obtained in the training phase. In contrast, model-based solutions are difficult to implement for complex problems and can cause high bias. Additionally, it is highlighted that a high bias level is achieved when ANNs are trained with low amount of samples or the training process is driven to overfitting (poor generalization). The concepts and developments presented in the reference [START_REF] Geman | Neural networks and the bias/variance dilemma[END_REF] are supported by a series of studies using neural networks ensembles for inference in complex scientific and industrial applications [START_REF] Ahmad | Bayesian selective combination of multiple neural networks for improving long-range predictions in nonlinear process modeling[END_REF][START_REF] Zio | A study of the bootstrap method for estimating the accuracy of ANN in predicting nuclear transient processes[END_REF], [START_REF] Edwards | Minimizing risk using prediction uncertainty in neural network estimation fusion and its application to papermaking[END_REF][START_REF] Hu | Handbook of Neural Network Signal Processing[END_REF][START_REF] Granitto | Neural net-works ensembles: evaluation of aggregation algorithms[END_REF][START_REF] Fortuna | Virtual instruments based on stacked neural networks to improve product quality monitoring in a refinery[END_REF][START_REF] Yu | A neural network ensemble-based model for on-line monitoring and diagnosis of out-of-control signals in multivariate manufacturing processes[END_REF][START_REF] Wu | A neural network ensemble model for on-line monitoring of process mean and variance shifts in correlated processes[END_REF][START_REF] Trichakis | Comparison of bootstrap confidence intervals for an ANN model of a karstic aquifer response[END_REF]. These studies emphasize that the use of neural networks ensembles result on better balance between bias and variance.

An efficient ANN ensemble is composed by networks that are as accurate and diversified as possible [12][14,15][17,18]. The accuracy of the neural networks can be achieved by using appropriate training sets and administrating the training process to avoid, as far as possible, the overfitting [START_REF] Granitto | Neural net-works ensembles: evaluation of aggregation algorithms[END_REF] [START_REF] Yu | A neural network ensemble-based model for on-line monitoring and diagnosis of out-of-control signals in multivariate manufacturing processes[END_REF][START_REF] Wu | A neural network ensemble model for on-line monitoring of process mean and variance shifts in correlated processes[END_REF].

The warranty of diversity of a set of neural networks is directly related to how the ensemble will manage the variance issue [6][14,15][17]. An important feature for ensuring a diverse set of ANNs is the concurrency of errors. A set of networks which share low amount of concurrent errors tends to meet the diversity criterion [START_REF] Sharkey | Neural nets and diversity[END_REF][START_REF] Brown | Diversity creation methods: a survey and categorization[END_REF][START_REF] Rokach | Taxonomy for characterizing ensemble methods in classification tasks: a review and annotated bibliography[END_REF]. Some ways to obtain potentially different networks are: different training algorithms; different architectures (number of layers and neurons, connectivity between layers and neurons, different activation functions); different training sets; different sets of initial weights for training. Among these ways, the simultaneous use of different training sets and different sets of initial weights has shown great potential for generating diversity [START_REF] Sharkey | How to improve the reliability of artificial neural networks[END_REF] [START_REF] Brown | Diversity creation methods: a survey and categorization[END_REF][START_REF] Rokach | Taxonomy for characterizing ensemble methods in classification tasks: a review and annotated bibliography[END_REF]. This paper presents the development of a neural network ensemble for the inference of steady-state condition in performance tests of refrigeration compressors. It is shown the evolution from complete ignorance when considering neural networks individually to a high reliable state when neural networks are considered to work together. Analysis of correlation, true positive and false positive observations are made to ensure accuracy and diversity for the ANNs ensemble.

Performance test of refrigeration compressors

The performance tests are run on special rigs, which operate as a refrigeration circuit with many controlled variables. In addition, it is possible to measure quantities that are not generally monitored in refrigeration systems.

Refrigerating capacity is measured in watts and is calculated as the product of the mass flow rate of refrigerant fluid through the compressor and the difference between the specific enthalpy of the refrigerant fluid at two points in the refrigeration circuit [START_REF]ISO Testing of refrigerant compressors[END_REF]. According to the enthalpy determination points that are chosen, it is possible to obtain different interpretations for the refrigerating capacity [START_REF]ISO Testing of refrigerant compressors[END_REF]. The most common is the heat removal from the cooled environment. The simplest method for determining the mass flow rate in a refrigeration circuit is to measure it directly using mass flow meters that are available on the market. Equipments that satisfy the measurement uncertainties required by international standards can be easily found. However, the measurement of refrigerating capacity needs to be carried out using two different and independent methods [START_REF]ISO Testing of refrigerant compressors[END_REF]. Typically, in order to ensure the methods independence, one is based on the direct measurement of the mass flow rate and the other is based on the heat balance inside a calorimeter.

Fig. 1 shows a simplified schematic of the refrigeration circuit and instrumentation. In this case the calorimeter is comprised in the area defined by the dashed line, where there is an evaporator, a heating resistor, and four temperature measurements transducers.

A typical characteristic of these tests is the long time required to achieve stable conditions, under which the measurements -used for the calculation of the final results -can be performed. In general, performance tests last 2½ hours. The criteria for stability are defined for the most important quantities that are measured during the test. For each quantity there is a percentage or an absolute variation limit which must be calculated based at the average of the measured values of the last period of 1 hour. So, if a specific quantity stays within the respective variation limit during 1 hour it is said to be stable. For example, the compressor shell temperature has an absolute variation limit of ±3°C then if the final shell temperature is to be +55°C this quantity is said to be stable when it overtakes +52°C. The issue is that the final values for many quantities are just known at the end of the test, indeed it is not possible to know when the quantity enters the stability region. The refrigerating capacity obtained using the mass flow meter method is analyzed in this article. Fig. 2 presents a set of four typical examples of refrigerating capacity data obtained during performance tests. These tests were run on compressors of different models produced by the same manufacturer. Since the refrigeration capacities have different values -from 50 W to 200 W -the normalized data is presented for better visualization of the dynamic behavior.

The steady-state region for the refrigerating capacity is characterized when it remains within ±2% of the final value. This final value is obtained through the mean of the measurements taken during the last hour of the test.

Figs. 3 and4 present other two important quantities that are measured during a performance test: compressor suction pressure and the compressor shell temperature. These quantities will be used as inputs of the neural networks that compose the ensemble to infer the steady-state.

An example of refrigerating capacity data and the respective desired ANN output is presented in Fig. 5. As the refrigerating capacity enters the ±2% range the ANN output should rise from 0 to 1. It is important to highlight that, even being interested in the refrigerating capacity, its final value can just be known at the end of the test. As a consequence, the ±2% stability range can only be known at the end of the test. Accordingly, other quantities must be used to support de steady-state inference. Supposing a true positive inference one could start the computation of the test result and early stop the test. These types of data are used in Section 3 for describing the neural network used to steady-state inference of the refrigeration capacity. The next section briefly presents the concepts used to develop the neural network.

Neural network ensemble

The multilayer perceptron neural network trained with backpropagation algorithm is used for the steady-state inference. Despite this type of neural network is not an intrinsically dynamic network, it is used for this application. The dynamical modeling is done by the inputs that were chosen to compose the network. The ANN inputs are average values of refrigerating capacity, suction pressure and shell temperature (three inputs); lagged average values of these quantities (four inputs -two for the refrigerating capacity and one for each one of the other quantities); difference between actual and lagged average values (eight inputs); and standard deviation of refrigerating capacity (one input). The sample size for averages and standard deviation is 15 with sampling frequency of 0.1 Hz. The number of inputs is 16 for a 3hidden layer network (15-10-10 neurons) with hyperbolic tangent activation functions. The output layer is comprised of one neuron with logistic activation function. This architecture resulted from an extensive evaluation heuristic and literature insights. The performances of individual neural networks are presented in this section.

The Fig. 7 presents the probabilities of true positive and false positive indications for the test data set. One can note that most of the probabilities are around 0.5, which indicates a high degree of ignorance when the neural networks are observed individually. This fact could make the specialist regret on using this neural networks for the steady-state identification. Another point that should be noticed is that ANNs with high probability of not indication have true positive probability higher than false positive probability. This means that if these neural networks infer the steady-state they act mostly on a true positive way. The histograms show that there are no equal networks or that behave like so in the ensemble. Most of the ANNs presented correlation from weak to moderate, that is, they share equal levels of accurate and inaccurate indications. This information indicates that the ensemble has an acceptable level of diversity. The ANNs-outputs integration was based on the analysis of the relationship between the number of ANNs that should indicate the steady-state condition and the time that should be waited for the confirmation.

As can be seen in Fig. 10, a minimum of 27 ANNs positive answers should be waited to achieve inferences into the ±2% steady-state region. Nonetheless, intermediate rules can be created to achieve a more effective reduction on test times. The rules combine number of ANN positive answers with confirmation waiting times. So, even small numbers of ANN positive answers may produce proper steady-state inferences. Again, the 50-example training data set was evaluated using all the 30 neural networks to create graphs as presented in Figs. 11 and12. Each one represents the time that should be waited after a specified number of ANNs indicates the steady-state and the related average absolute relative difference for all the training data set (solid line). This difference is between the refrigerating capacity at the instant which a certain number of ANNs indicated steady-state and final value of the performance test. The dotted lines represent two standard deviations added and subtracted from the average. This analysis was run considering numbers of ANNs varying from 1 to 30, but for the sake of space, only the results for 10 and 25 networks are presented in this article.

The whole set of graphs was used to define the rules to integrate the ANNs outputs, as presented in Table 1, where numbers of ANNs and confirmation times were related by the analysis of Fig. 10 and all the 30 graphs (as shown in Figs. 11 and12). Then, all the 210 test data set examples were evaluated using the 30-ANN ensemble and the four rules. Fig. 13 presents a histogram of the relative differences between the refrigerating capacities at instant the steady-state was inferred by ensemble and the expected values for all the 210 performance tests. It can be noted that most of the tests had the steadystate inference within the range of ±2% of the final value, actually, more than 91%, what is a great improvement if compared to the true positive probabilities presented by the neural networks individually.

Conclusions

This paper presented the application of an ANN ensemble for steady-state inference that is part of a more complete tool to predict the refrigerating capacity in performance tests of refrigeration compressors. It was shown that ANN individual errors can be minimized when the networks are combined. Correlation analysis can be a good indication of diversity for an ANN ensemble. The inference allows a significant improvement in quality control and R&D activities. The average test time for the 210 performance tests was 2½ hours; when these tests are evaluated using the ensemble it is possible to reduce to half of that time. The application of steady-state inference could be extended to other performance variables like the power consumption, isentropic efficiency and coefficient performance (COP).

Fig. 1 .

 1 Fig. 1. Simplified schematic of the test rig

Fig. 2 .Fig. 3 .Fig. 4 .Fig. 5 .

 2345 Fig. 2. Normalized refrigerating capacity data

  training set was composed by 50 examples of performance tests obtained from different compressors. The bootstrap method was used to compose the individual ANNs training sets (40 examples randomly selected) and the remaining examples were used for on-line validation (early stop to avoid overfitting). As the bootstrap was run with replacement, the on-line validation set could have more than 10 examples. The neural network toolbox of Matlab® was used to train a set of 30 ANNs. After that, the neural networks were tested with a set of 210 examples which does not include the 50 examples used for training. The number of ANNs that was trained is based on the achieved performance. It is shown ahead that the ensemble comprised of 30 ANNs is suitable for this application when used with a confirmation time. The literature presents several applications that indicate a number between 20 and 200 for the ensemble members to provide proper results [2][6][12][14,15]. Fig. 6 presents the probability of not indication the steady-state for each one of the 30 ANNs when the test data set is used. It can be seen that the probability varies from almost zero to 0.40, which means that some ANNs indicate steady-state condition to almost all the 210 examples and some that do not indicate on almost half of the examples.

Fig. 6 .Fig. 7 .

 67 Fig. 6. Probability of no indication of steady-state

Fig. 8 .Fig. 9 .

 89 Fig. 8. Correlation of true positive indications

Fig. 10 .

 10 Fig. 10. Difference of refrigerating capacity versus number of ANNs

Fig. 11 .Fig. 12 .

 1112 Fig. 11. Confirmation waiting time graph for 10 ANNs

Fig. 13 .

 13 Fig. 13. ANN ensemble results

Table 1 .

 1 ANN Integration Rules

	Rule	Number of ANNs	Confirmation waiting time
	I	< 15	40 minutes
	II	≥ 15 and < 20	30 minutes
	III	≥ 20 and ≤ 27	10 minutes
	IV	> 27	0

Acknowledgements

The authors would like to thanks the financial support from Whirlpool S.A./Embraco and the Brazilian funding agencies CNPq (143035/2006-1), CAPES/PNPD (23038.007820/2011-61).