Mateusz Kalisch

Piotr Przystaªka

Anna Timoejczuk

Actuator Fault Diagnosis Using Single and Meta-Classication Strategies

The paper presents the application of various classication schemes for actuator fault diagnosis in industrial systems. The main objective of this study is to compare either single or meta-classication strategies that can be successfully used as reasoning means in the diagnostic expert system that is realized within the frame of the DIS-ESOR project. The applied research was conducted on the assumption that classic as well as soft computing classication methods would be adopted. The comparison study was carried out within the DAMADICS benchmark problem which provides a popular framework for confronting dierent approaches in the development of fault diagnosis systems.

Introduction

The increasing complexity of recent industrial objects makes the issue of fault diagnosis one of the most important directions of research in modern automatic control and robotics [START_REF] Caccavale | Fault Diagnosis and Fault Tolerance for Mechatronic Systems: Recent Advances[END_REF][START_REF] Ko±cielny | Diagnostyka zautomatyzowanych procesów przemysªowych[END_REF][START_REF] Patton | Issues of Fault Diagnosis for Dynamic Systems[END_REF]. Technical systems and processes are required to be safely and reliably operated due to the protection of human life and health, the quality of the environment, as well as the economic interests. It is possible to specify numerous areas of interdependence of human and technical means, where safety plays a key role, for instance in aircraft, spaceship, automotive, power or mining industry. The above mentioned factors cause that new developments in control theory such as passive and active fault-tolerant control approaches are more often applied in these areas of the industry [START_REF]Fault diagnosis. Models, articial intelligence, applications[END_REF][START_REF] Isermann | Model-based fault detection and diagnosis -status and applications[END_REF][START_REF] Blanke | Diagnosis and Fault-Tolerant Control[END_REF]. A special attention is currently paid on the second type of the advanced control methodologies, where fault diagnosis methods hold a critical importance. The present state of the art in the eld of fault diagnosis shows the really need for development of fault diagnosis expert systems. The goal is to elaborate general-purposes systems with multi-domain knowledge representations and multi-inference engines [START_REF] Moczulski | Inductive acquisition of diagnostic knowledge for states tree with complex structure[END_REF][START_REF] Moczulski | Diagnostyka techniczna. Metody pozyskiwania wiedzy[END_REF][START_REF] Cholewa | Real-time diagnostic expert systems[END_REF].

Generally, the fault diagnosis can be divided into three steps [START_REF] Isermann | Fault-Diagnosis Systems[END_REF]: fault detection, fault isolation and fault identication. Moreover, each of them can be developed by means of model-free (based on data), model-based and knowledgebased approaches [START_REF]Fault diagnosis. Models, articial intelligence, applications[END_REF]. In this paper the rst approach, where experimental data are exploited was discussed. In this kind of methods data that represents normal and faulty situations can be obtained from historical databases or from simulators as well as laboratory stands. This data is then used to create state classiers and meta-classiers.

The main goal of this paper is to compare dierent classication strategies that can be successfully used as reasoning means in the diagnostic expert system.

The development of the diagnostic expert system shell with multi-domain knowledge representations and multi-inference engines is realized within the frame of the DISESOR project. The DISESOR is an acronym of the decision support system designed for fault diagnosis of machinery and other equipment operating in underground mines as well as for monitoring potential threats that can occur in such kind of industry. The DISESOR system can be used for dierent purposes, e.g. to assess seismic hazard probabilities in the area of the coal mine, to forecast dangerous increase in the methane concentration in the mine shafts, to detect and localize endogenous res, and also to conduct fault diagnostics of machines working in such environment. This study shows the comparison research of the classication schemes for creating fault diagnosis system of the benchmark actuator [START_REF] Barty± | Introduction to the DAMADICS actuator FDI benchmark study[END_REF] which was elaborated on the basis of the activity of the DAMADICS

Single and meta-classication strategies

There are many types of classiers available in the literature, as well as dierent concepts of using them are introduced [START_REF] Kuncheva | Combining Pattern Classier: Methods and Algorithms[END_REF]. Some examples are methods based on the similarity between objects in the feature space, probabilistic methods or methods which are based on black box models. Generally, the classication problems can be divided into two groups including approaches of supervised and unsupervised machine learning techniques. In the paper, the authors concentrated the attention only on methods belonging to the rst group. Currently, the information fusion and meta-classication problems are recognized as the most important directions of the research in the domain of supervised learning.

The main idea in this approach is the application of simple classiers working together to solve a problem with better results than it can be done by means of single one or more complicated classiers. There are a lot of dierent kinds of information fusion methods, but the most popular are majority voting, weighted voting, boosting, and AdaBoost [START_REF] Kuncheva | Combining Pattern Classier: Methods and Algorithms[END_REF]. On the other hand, meta-classiers are very often used for the same reason that means its eciency is often higher, than the eciency of the best single classier [START_REF] Lam | Classier combinations: Implementations and theoretical issue[END_REF].

The current research trends in developing machine learning methods are focused on ideas of improving the general eciency of dierent classication and meta-classication methods. The most important investigations can be found for instance in [START_REF] Namdari | Process fault diagnosis using support vector machines with a genetic algorithm based parameter tuning[END_REF][START_REF] Yang | Random forests classier for machine fault diagnosis[END_REF][START_REF] Pöyhönen | Support vector machines in fault diagnostics of electrical motors[END_REF][START_REF] Wu | Car assembly line fault diagnosis based on triangular fuzzy support vector classier machine and particle swarm optimization[END_REF][START_REF] Bhadra | Dierential evolution based optimization of svm parameters for meta classier design[END_REF][START_REF] Cretulescu | Weights space exploration using genetic algorithms for meta-classier in text document classication[END_REF]. The main directions presented in these studies are concentrated on optimization techniques which are used to tune relevant parameters of the classical methods, e.g. with the use of evolutionary and particle swarm algorithms. A number of results included in the related works show the benets of using these methods. In case of a task of fault detection and isolation the key features of the signals in time or frequency domains are most commonly used. Industrial actuators may be characterized by a very high complexity which aects the large number of measuring signals and their features. Therefore, another approach aimed at improving the eciency of the classier, and often shortening the time of its learning, is to remove irrelevant variables [START_REF] Guyon | An introduction to variable and feature selection[END_REF]. There are various methods that can be used in this procedure, e.g. forward or backward selection methods, as well as elimination methods based on statistical measures. Another group of methods stands fusion methods such as bagging, boosting, and the development of these concepts that is AdaBoost method [START_REF] Jingyuan | Analog circuit fault diagnosis using adaboost and svm[END_REF][START_REF] Yao | Fault signal classication using adaptive boosting algorithm[END_REF]. These methods are often more eective than simple classiers but also show some drawbacks.

The advanced concepts were developed to take merits and positive aspects of classic methods and to eliminate their limitations [START_REF] Wo¹niak | Metody fuzji informacji dla komputerowych systemów rozpoznawania[END_REF]. There are also attempts to connect together several dierent methods such as selection of relevant features and usage of boosting into one algorithm [START_REF] Kerdprasop | Feature selection and boosting techniques to improve fault detection accuracy in the semiconductor manufacturing process[END_REF]. Such approach may lead to the nal result that should be better than the results of the methods applied separately.

Model-free fault diagnosis using dierent classication schemes

The idea of the well practised model-free fault detection and isolation method is presented in Fig. 1. It can be seen, that faults are detected and distinguished using primary and redundant process variables. In this method two separated classiers must be created. The rst classier uses the subset of process variables (U ∪ Y ) as its input and it is dedicated for generating diagnostic signals (S), whereas the second one has the same set of input variables but its task is to calculate a fault signature (F ). This classier is triggered in case when the diagnostic signal indicates a fault scenario. The proposed method can be viewed as the extension of the most often used model-free fault diagnosis approach, cf. Korbicz et al. (2004), Fig. 1.7 at p. 22 [START_REF]Fault diagnosis. Models, articial intelligence, applications[END_REF]. The novelty in this study depends on that single and also meta-classiers are automatically tuned in order to obtain the maximum accuracy of fault diagnosis.

Fault detection and isolation algorithms corresponding to the diagram presented in Fig. 1 can be designed using dierent classication methods [START_REF] Isermann | Fault-Diagnosis Systems[END_REF][START_REF]Fault diagnosis. Models, articial intelligence, applications[END_REF][START_REF] Patton | Soft computing approaches to fault diagnosis for dynamic systems: A survey[END_REF].

Generally, it is possible to apply so-called classical (e.g. decision trees, k-nearest neighbour, naive Bayes, etc.) or soft computing approaches (e.g. neural networks, bayesian networks, fuzzy systems, neuro-fuzzy systems, etc.). The paper deals with either classic or soft computing methods. In the next part of the article, model-free fault detection and isolation approaches with the use of dierent Fig. 1: A diagram of model-free fault detection and isolation classication schemes are described. As it was mentioned above, these kinds of methods require data (process variables) corresponding to regular (faultless) and faulty states of the system. In this section, dierent variants of three basic concepts with a single classier, meta-classier and a bank of classiers are applied in order to provide the fault detection and isolation system that is directly based on the process variables.

Fault detection schemas

The rst concept of fault detection is presented in Fig. 2 and this is elaborated basing on a single classier which returns a diagnostic signal corresponding to fault or faultless states of the device. In this method, the process variables are converted by a moving window in order to compute scalar features of the measuring signals. These values are used as input of a single classier which generates directly the diagnostic signal. The second fault detection scheme is presented in Fig. 3. In this approach a series of two-state classiers is applied and their task is to determine the degree of the belief for fault detection. The level of belief about faults occurring is a numerical value from 0 to 1. The signal values returned from each classier are connected to the meta-classier as its input. The features of the process variables are also connected to the meta-classier as the additional input.

Fault isolation schemes

The rst method of fault isolation is comparable to the method that was proposed for the fault detection. It is presented in The engines of fault detection and isolation schemes presented above can be elaborated with the use of well practised classication methods. The classication problem is possible to be solved using many known approaches, however, in this research the following methods are applied: k -nearest neighbour [START_REF] Akthar | RapidMiner 5, Operator Reference[END_REF], naive Bayes [START_REF] Cichosz | Systemy ucz¡ce si¦[END_REF], decision tree [START_REF] Lile | Analyzing e-learning systems using educational data mining techniques[END_REF][START_REF] Breiman | Classication and Regression Trees[END_REF], rules induction [START_REF] Cohen | Fast eective rule induction[END_REF], neural networks [START_REF]Metody i techniki sztucznej inteligencji[END_REF][START_REF] Haykin | Neural Networks: A Comprehensive Foundation[END_REF] and support vector machine [START_REF] Hsu | A practical guide to support vector classication[END_REF]. Each of these classiers returns a label of a chosen The other two groups were used for a meta-classier. In this approach, the size of the dataset for each group was equal to 25200 samples, where 12600 samples were prepared from data without faults and rest of them contained data with all considered faults.

The data prepared for the rst two fault isolation methods consists of characteristic process values for all chosen faults. The number of elements for each fault was the same for all sets. For learning and verication process four independent groups of data were prepared (like in fault detection methods, two for base classiers and two for meta-classier). The dataset for a single fault for one group contains 900 samples, while the full dataset size is equal to 12600 samples. The third method of fault isolation requires a dierent type of data. The initial classier needs data, where a half of the elements describes an actuator device working with one specic fault and the rest of the elements describe the device working with the other faults. In this case, a classier can generate a two-state signal where the rst state denes one specic fault and the other ones are correlated with unknown faults. The size of the dataset in this approach is similar as in case for the method of fault detection. The size of the dataset for the considered fault is equal to 3900 samples and it is equal to the rest of a dataset which contains samples corresponding to other faults.

Statistical analysis and features computation

Linear correlation and mutual information analysis were used for choosing relevant process variables and a proper value of the width of a moving window function. In the analysis, all of available process variables were compared for dierent device status (e.g. device without faults and with a chosen fault). The results of these tests showed very strong correlations between states F8, F14 and F0. A group of useful process signals was prepared on the basis of results of these tests. Most of the process signals had very dicult character for modelfree fault detection and isolation methods. Therefore, the authors decided to apply scalar features of the process variables. The authors employed a few well known features often used in fault detection and isolation processes such as average, maximum and minimum values, standard deviation, root mean square, shape factor (a factor determines the shape of the periodic signal), kurtosis (the measure for evaluating the shape of the probability distribution), energy, skewness (the measure of the asymmetry of the probability distribution) and entropy.

The scalar features were computed using a moving window of 100 samples width.

Such the width value was assumed on the basis of frequency of the harmonic control signal of the valve which was equal to 0, 01 Hz. The authors also studied other values of the window width, however, expected eects in increasing of the eciencies of the models were not observed.

Results of verication studies

The learning process for the whole set of applied classiers was conducted using the X-validation method. To show the eciency of a classication process, the authors used three measures corresponding to accuracy, sensitivity and precision.

The accuracy presents the proportion of correct guesses which can be used directly as eciency of classication because all datasets were very good balanced (the number of samples for each class was equal). The sensitivity corresponds to the proportion of samples in actual class which are correctly identied. The precision denotes the probability that an example assigned to specic class should be really connected with this class. The sensitivity and precision were calculated for each class, whereas the accuracy were calculated as one value for the whole confusion matrix. In order to have study results more clearly in each table within this section the following notation was assumed: kNN -k-nearest neighbours, NB -Naive Bayes, DT -Decision Tree, RI -Rule Induction, NN -Neural Net, SVM -Support Vector Machine. The prex letter M placed before each label of the classier means the meta-version of this classier, for example, the label MNB denotes the meta-classier which is based on a naive Bayes classier.

Moreover, the authors decided to apply the parameter optimization operator included in Rapidminer software in order to tune behavioural parameters of the investigated classiers. The described schemes of fault detection and isolation were examined taking into consideration all types of classiers. The optimization operator for each variant of a classier was based on the evolutionary computation. The standard evolutionary algorithm was used [START_REF]Metody i techniki sztucznej inteligencji[END_REF]. The maximal number of generations was set to 30 and the population size was equal to 5.

The tournament method was used to select parents for creating new generations kNN 0,933 0,978 0,887 0,93 1,00 1,00 0,67 0,84 0,84 0,67 1,00 0,67 1,00 0,78 1,00 1,00 0,99 NB 0,878 0,992 0,764 1,00 1,00 1,00 0,00 0,65 1,00 0,71 1,00 0,00 1,00 0,33 1,00 1,00 1,00 DT 0,960 0,976 0,946 1,00 1,00 1,00 0,65 0,98 1,00 1,00 1,00 0,67 1,00 0,94 1,00 1,00 1,00 RI 0,957 0,973 0,939 1,00 1,00 1,00 0,67 0,95 1,00 0,98 1,00 0,72 1,00 0,81 1,00 1,00 1,00 NN 0,829 0,977 0,681 0,75 1,00 1,00 0,24 0,49 0,17 0,25 1,00 0,24 1,00 0,50 1,00 1,00 0,89 SVM 0,938 0,982 0,894 0,96 1,00 1,00 0,67 0,86 0,95 0,67 1,00 0,67 1,00 0,78 1,00 1,00 0,96 MkNN 0,862 0,931 0,793 1,00 1,00 1,00 0,01 0,84 1,00 0,92 1,00 0,01 1,00 0,34 1,00 1,00 1,00 MNB 0,889 0,980 0,799 1,00 1,00 1,00 0,00 0,87 1,00 0,99 1,00 0,00 1,00 0,33 1,00 1,00 1,00 MDT 0,883 0,971 0,794 1,00 1,00 1,00 0,00 0,83 1,00 0,95 1,00 0,00 1,00 0,33 1,00 1,00 1,00 MRI 0,888 0,974 0,803 1,00 1,00 1,00 0,00 0,85 1,00 1,00 1,00 0,00 1,00 0,39 1,00 1,00 1,00 MNN 0,875 0,926 0,824 1,00 1,00 1,00 0,15 0,86 1,00 0,93 1,00 0,15 1,00 0,45 1,00 1,00 1,00 MSVM 0,773 0.959 0.587 0,69 0,89 0,75 0,00 0,57 0,87 0,63 0,39 0,00 0,88 0,23 0,87 0,69 0,76 because the sensitivity in this case is equal to 1, 00. Faults highly-correlated with faultless state have very low sensitivity value, sometimes even equal to zero. The precision values presented in the Tab. 2 shows proportion between the number of samples which belong to this class and the number of all samples assigned by classier to the specic class. In the second scheme of fault detection metaclassiers were used. All six classiers trained in the previous process based on the rst scheme (Fig. 3) were used to calculate degrees of belief of fault detection.

The obtained data were merged with base classier's input data. The nal results of classication for the second scheme of fault detection is presented in Tab. 1 and 2. In most cases (except classier based on neural net and naive Bayes) the general accuracy, sensitivity and precision of classication in the second scheme are lower or similar to analogous measures in the rst scheme. The meta-classier based on neural net provided the higher accuracy, sensitivity and precision than neural net used as a simple classier. It means that additional variables (degrees 0,90 0,81 0,95 0,94 0,75 0,90 0,82 0,83 0,83 0,83 0,84 0,70 F1-F19 0,98 0,99 0,97 0,97 0,97 0,98 0,92 0,98 0,97 0,97 0,92 0,93 of beliefs of base classiers) can be successfully applied to improve the eciency of classication.

Results of fault isolation

All F1 F2 F7 F8 F10 F11 F12 F13 F14 F15 F16 F17 F18 F19
kNN 0,73 0,84 1,00 1,00 0,35 0,76 0,72 0,51 0,81 0,05 1,00 0,48 1,00 1,00 0,67 NB 0,77 0.67 1,00 1,00 0,58 0,63 1,00 1,00 0,58 0,40 1,00 0,28 1,00 0,91 0,67 DT 0,07 1,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 RI 0,78 1,00 1,00 1,00 0,59 0,67 1,00 0,98 0,61 0,00 1,00 0,73 1,00 0,63 0,67 NN 0,77 1,00 1,00 1,00 0,00 0,62 0,96 1,00 0,81 0,00 1,00 0,65 1,00 1,00 0,67 SVM 0,67 0,48 0,96 1,00 0,09 0,78 0,95 0,79 0,64 0,00 0,98 0,50 0,91 0,63 0,67

Tab. 4 presents the precision of classication for each class and classier. The results of tested classiers (Tab. 3) were varied, e.g. the accuracy of decision tree was 0.07 and the second one (sorted by their accuracy values) was 0.67. The sensitivity of decision tree for fault 1 is equal to 1, 00 but the precision is equal to 0, 07. The second method presented in Fig. 5 uses six classiers as in the previous method but the outputs of these classiers are connected to a meta-classier. kNN 0,99 1,00 1,00 0,27 0,61 0,50 0,47 0,99 0,15 1,00 0,61 0,75 0,84 1,00 NB 1,00 1,00 1,00 0,34 0,69 0,86 0,97 0,87 0,35 1,00 0,73 0,75 0,68 1,00 DT 0,07 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 RI 0,75 1,00 1,00 0,47 1,00 0,88 0,95 0,58 0,00 1,00 0,37 1,00 0,74 1,00 NN 1,00 1,00 1,00 0,05 0,94 1,00 0,57 0,71 0,00 1,00 0,26 1,00 0,84 0,99 SVM 0,99 1,00 0,34 0,39 0,72 0,92 0,72 0,72 0,04 1,00 0,26 1,00 0,99 0,95

The results obtained for the meta-classier are compared in Tab. 5 and Tab. 6.

The results obtained for the second scheme of fault isolation (Fig. 5), which was based on the meta-classier were more similar to each other than in the rst case (Fig. 4). The accuracy of classication (Tab 5) in the second scheme is much better than in the rst approach of fault isolation. Meta-classiers used only degrees of beliefs from base classiers and they were able to signicantly increase the general eciency. The most of precision values (Tab 5) for each class are very similar. The similarity between sensitivity and precision tables (Tab 6) is very high but there is one exception. The sensitivity of fault 12 is less then 1, 00 but the precision for almost all classiers is equal to 1, 00. It means that not all samples connected with fault 12 are recognized but one can be sure that all recognized samples are really connected with this fault. MkNN 0,86 1,00 1,00 1,00 0,63 0,98 0,99 0,88 0,98 0,11 1,00 0,53 1,00 1,00 1,00 MNB 0,84 0,97 1,00 1,00 0,14 0,91 0,99 0,78 0,97 0,65 1,00 0,49 1,00 0,91 1,00 MDT 0,84 0,99 1,00 1,00 0,00 0,84 0,99 0,74 0,97 0,84 1,00 0,43 1,00 1,00 1,00 MRI 0,85 0,99 1,00 1,00 0,70 0,99 1,00 0,74 0,98 0,14 1,00 0,40 1,00 1,00 1,00 MNN 0,87 1,00 1,00 1,00 1,00 0,95 1,00 0,88 0,97 0,00 1,00 0,41 1,00 1,00 1,00 MSVM 0,78 0,94 1,00 1,00 0,00 0,89 0,99 0,81 0,93 0,00 1,00 0,99 1,00 0,38 1,00 MkNN 1,00 1,00 1,00 0,40 0,94 0,93 0,74 1,00 0,33 1,00 0,66 1,00 0,98 1,00 MNB 0,90 1,00 1,00 0,36 0,98 1,00 1,00 0,91 0,38 1,00 0,44 1,00 0,97 1,00 MDT 0,99 1,00 1,00 0,00 1,00 0,86 1,00 1,00 0,37 1,00 0,46 0,99 0,97 0,99 MRI 1,00 1,00 1,00 0,39 0,68 0,99 1,00 1,00 0,28 1,00 0,79 0,99 0,98 1,00 MNN 1,00 0,99 1,00 0,38 0,99 0,90 1,00 1,00 0,00 1,00 0,99 1,00 0,97 1,00 MSVM 1,00 1,00 1,00 0,00 0,99 1,00 1,00 1,00 0,00 1,00 0,25 1,00 0,95 1,00 a single fault. After the analysis of the results presented in Tab. 7 the authors nominated the classiers for single fault detection. These classiers were chosen on the basis of general results. In case more than one classier had the same eciency value (more classiers with the eciency equal to 1, 000 for the fault at the same time) the authors pointed out a classier with more stable results in the time domain. The accuracy and sensitivity of meta-classiers are presented in Tab. 8. The accuracy of this method is a little bit higher than in the rst scheme of fault isolation (Fig. 4) but the sensitivity (Tab. 8) and precision (Tab. 9) for the third scheme are more stable and similar for each class.

Table 9: The precision of fault isolation for meta-classiers with a bank of classiers for isolating single faults

F1 F2 F7 F8 F10 F11 F12 F13 F14 F15 F16 F17 F18 F19
MkNN 1,00 1,00 1,00 0,35 0,47 0,95 0,72 0,96 0,36 1,00 0,57 1,00 0,84 0,91 MNB 1,00 1,00 1,00 0,32 0,44 0,69 0,70 0,99 0,30 1,00 0,84 1,00 0,84 0,75 MDT 0,99 1,00 1,00 0,00 0,82 0,70 0,72 0,95 0,40 1,00 0,34 1,00 0,84 0,90 MRI 1,00 1,00 1,00 0,38 0,55 0,70 0,94 0,96 0,36 1,00 0,41 1,00 0,81 0,89 MNN 1,00 1,00 0,75 0,31 0,84 0,96 0,72 1,00 0,00 1,00 0,99 1,00 0,84 0,86 MSVM 1,00 1,00 1,00 0,34 0,34 0,93 0,72 0,98 0,40 1,00 0,56 1,00 0,84 0,91

Five of the six single classiers used in the rst fault isolation process reached results between 0, 67 and 0, 78. One classier is useless (decision tree) and its accuracy is equal to 0, 07. Meta-classiers were able to signicantly increase the accuracy of fault isolation. In this case the accuracy values of all classiers are between 0, 78 and 0, 87. The sensitivity and precision values are also more stable and similar in each class. The last approach based on the third scheme of fault isolation (Fig. 6) shows also small improvement compared to the rst scheme (see Fig. 4). Moreover, in this scheme the general eciency of fault isolation is close to the result achieved by means of the single classier. Generally, meta-classiers in fault-isolation schemes are more stable and their results are similar to each other. In case of the second and third schema it is possible to use classier based on decision tree so data prepared by base classiers are simpler and more varied.

In this study, the authors used a confusion matrix in order to evaluate fault diagnosis systems that were created applying dierent classication schemes.

Nevertheless, the accuracy, sensitivity and precision obtained from a confusion matrix can be directly compared with false and true detection/isolation rates proposed by the authors of the DAMADICS simulator [START_REF] Barty± | Introduction to the DAMADICS actuator FDI benchmark study[END_REF]. The results of fault detection and isolation using single or meta-classication strategies that were achieved in this study are comparable to even more advanced methods described in the literature [8] [34]. Furthermore, in this study the whole set of potential faults were investigated, whereas in the related papers only selected states were taken into consideration.

Conclusion

In the paper the application of selected classication schemes for fault diagnosis of the actuator systems was presented. The main purpose of the paper was to compare single and meta-classication strategies that could be successfully used as knowledge representation in the diagnostic expert system that is realized within the frame of the DISESOR project. The research was carried out basing on the well-practised hard and soft computing classication methods. The current paper can be viewed as the extension of the research work presented in [START_REF] Kalisch | Application of selected classication schemes for fault diagnosis of actuator systems[END_REF]. In this study, the authors proposed a new approach for searching proper values of relevant parameters of classiers used to fault diagnosis.

The examined methods were tested in the context to be used for designing the engine of the DISESOR system. The comparison study was carried out within the DAMADICS benchmark problem. The classication schemes were implemented in RapidMiner software which is a well-known open source system for data mining and knowledge discovery. The particular results of the fault detection procedures showed that for simple industrial actuators it is possible to apply simple classication schemes without the necessity of using more advanced methods which are based on meta-classiers. The nal results reached in this paper are much better than results showed in [START_REF] Kalisch | Application of selected classication schemes for fault diagnosis of actuator systems[END_REF]. The features and parameters of classiers can be automatically tuned to increase their accuracy and sensitivity.

Overall, the application of single or meta-classication strategies with optimizing of relevant parameters allows to create eective as well as relatively less-complicated computational fault detection and isolation systems that can be successfully employed for on-line and o-line fault diagnosis of industrial actuators.

(

  Development and Application of Methods for Actuator Diagnosis in Industrial Control Systems) Research Training Network funded by the European Commission. The current paper is a continuation of the research work presented in [20]. The authors taken into account the majority of reviewers' comments and also proposed a new approach for searching proper values of relevant parameters of classiers used to fault diagnosis. The examined methods are planed to be used for designing the engine of the DISESOR system.

Fig. 2 :Fig. 3 :

 23 Fig. 2: A scheme of fault detection using the global classier

Fig. 4 .Fig. 4 :Fig. 5 :

 445 Fig. 4: A scheme of fault isolation using the global classier

Fig. 6 :

 6 Fig. 6: A scheme of fault isolation using the set of local classiers (fault detectors) and meta-classier

Fig. 7 :

 7 schemes. This simulator was elaborated in collaboration of scientists and engineers to simplify the process of evaluating and comparing dierent methods of fault detection and isolation for industrial systems. In the literature there are available several papers where case study results deal with this problem are presented, see e.g.[START_REF] Puig | A GMDH neural network-based approach to passive robust fault detection using a constraint satisfaction backward test[END_REF][START_REF] Mrugalski | Condence estimation of the multi-layer perceptron and its application in fault detection systems[END_REF][START_REF] Korbicz | Neuro-fuzzy networks and their application to fault detection of dynamical systems[END_REF]. The numeric model is used to simulate an electropneumatic valve (Fig.7) which is a part of the production line in Lublin sugar factory in Poland.

( 2 and 3 .

 23 the tournament fraction was equal to 0.25). Reproduction was realized based on Gaussian mutation function and crossover operator. The crossover probability was equal to 0.9. The optimization operator maximized accuracy value of chosen classier. In the case of Naive Bayes classier the optimize selection operator was only applied to choose the most relevant attributes. The features of this algorithm were similar to the previous one. The k-nearest neighbours classier was optimized by searching a proper value of the parameter k. In case of decision tree following values of parameters were chosen: the minimal leaf size, the maximal depth of tree, the minimal gain which aects the ability of the splitting of leaf as well as the condence used for the pessimistic error calculation of pruning. The rule induction was optimized in the context of the sample ratio that species the sample ratio of training data used for growing and pruning, the pureness corresponds to the desired pureness and the minimal prune benet. In the case of a neural net classier the number of layers and the number of neurons in each layer were tested and chosen by the authors based on heuristics known from the literature. The other parameters of neural classiers were optimized by the operator: the number of training cycles, the learning rate determines how much the weights are adjusted at each step, the momentum coecient smooths optimization directions and the error epsilon as training error threshold. The SVM based classier has many behavioural parameters and therefore the authors decided to use the methodology described in[START_REF] Hsu | A practical guide to support vector classication[END_REF] and focused their attention only on γ and C parameters of the radial base function. The more detailed description of relevant parameters of classiers and the optimization operator can be found in[START_REF] Akthar | RapidMiner 5, Operator Reference[END_REF]. The minimum and maximum values of the optimized parameters were dened on the basis of the information contained in the literature.Results of fault detectionIn the rst concept of fault detection (Fig. 2) six single classiers were compared. Tab. 1 shows the accuracy and sensitivity of two concepts of fault detection realized relating to the schemes presented in Fig. The sensitivity values of classiers are given in columns. The column indicated as All includes the general eciency calculated on the basis of the confusion matrix which was generated after the classier verication process. The next column (F0) includes the sensitivity obtained for faultless states. The rest of the columns (F1-F19) show the sensitivity of fault detection for all considered faults, separately. Above these columns the general result of the sensitivity of fault detection is presented. Rows from 1 to 6 show results for single classiers, whereas the next six rows show results for considered meta-classiers. The second

  The last three methods concerned fault isolation without taking into account a faultless scenario. The results of comparison of the rst fault isolation method (Fig. 4) are included in Tab. 3 and 4. The column indicated as All in Tab. 3 includes values of the accuracy for single classiers of dierent types. The rest of the columns show information deals with the sensitivity of fault isolation for each scenario.

table (

 ( Tab. 2) shows the precision of fault detection for each classier and class (faultless state and failure state). All precision values are less than 1, 00 which means that there always exist some samples which are assigned to another state than it should.

	In rst fault detection scheme the accuracy of most used classiers is high
	(above 0, 93). The accuracy of neural net and naive bayes based classiers is
	a little bit lower than other ones but it is still in the acceptable range. The
	sensitivity of faultless state detection is very close to 1, 00, which means that
	almost all samples corresponding to faultless state were classied correctly to this
	class. It is easy to distinguish faults which are low-correlated with faultless state

Table 1 :

 1 The accuracy and sensitivity of fault detection for global classiers and

	meta-classiers
	All	F0	F1	F2	F7	F8 F10 F11 F12 F13 F14 F15 F16 F17 F18 F19

Table 2 :

 2 The precision of fault detection for global classiers and meta-classiers

	kNN NB DT RI	NN SVM MkNN MNB MDT MRI MNN MSVM
	F0

Table 3 :

 3 Accuracy and sensitivity of fault isolation for global classiers

  It means that a classier is able to classify correctly samples related to fault 1 but unfortunately the rest of samples connected with other faults are also classied as fault 1. This kind of classier is not useful. The accuracy of the next group of classiers are better and rule induction based ones reach the best result equal to 0, 78. In the sensitivity table (Tab. 2) all classiers (except decision tree) reach very similar results for most of the classes. The sensitivity value is more varied for classes which are not easily distinguishable e.g. classication sensitivity for fault 16 is the range from 0, 00 to 0, 73 and each classier reach a dierent value. The precision values (Tab 4) are correlated with the sensitivity value. For faults which are easy to classify the precision is equal or close to 1, 00, for more dicult faults to recognize the precision values are varied.

Table 4 :

 4 The precision of fault isolation for global classiers

	F1	F2	F7	F8 F10 F11 F12 F13 F14 F15 F16 F17 F18 F19

Table 5 :

 5 The accuracy and sensitivity of fault isolation for meta-classiers

	All F1	F2	F7	F8	F10 F11 F12 F13 F14 F15 F16 F17 F18 F19

Table 6 :

 6 The precision of fault isolation for meta-classiers

	F1	F2	F7	F8	F10 F11 F12 F13 F14 F15 F16 F17 F18 F19

Acknowledgement

The research presented in the paper was partially nanced by the National Centre of Research and Development (Poland) within the frame of the project titled "Zintegrowany, szkieletowy system wspomagania decyzji dla systemów monitorowania procesów, urz¡dze« i zagro»e«" (in Polish) carried out in the path B of Applied Research Programme -grant No. PBS2/B9/20/2013. The part of the research was also nanced from the statutory funds of the Institute of Fundamentals of Machinery Design.

The last method of fault isolation (Fig. 6) is based on series of single classiers, where each classier is used for detecting a single fault. The rst task of the verication process was to choose a single classier (from six available) for the fault detection purpose. To solve this problem the authors tested all classiers for all available faults. The results are presented in Tab. 7. The values included in the table present the general accuracy of each classier. The bold values are related to the classiers which were chosen as the basic classiers for the meta-classier.

kNN 0,92 1,00 1,00 0,72 0,80 0,84 0,80 0,98 0,72 1,00 0,74 1,00 0,98 0,98 NB 0,64 1,00 1,00 0,92 0,71 0,99 0,89 0,94 0,90 1,00 0,72 1,00 0,96 1,00 DT 1,00 1,00 1,00 0,77 0,59 1,00 0,95 0,92 0,79 1,00 0,63 0,99 0,91 0,81 RI 0,93 1,00 1,00 0,87 0,62 1,00 0,92 0,94 0,85 1,00 0,56 0,99 0,94 0,89 NN 0,94 0,99 1,00 0,81 0,85 0,99 0,89 0,97 0,81 1,00 0,80 1,00 0,98 0,71 SVM 0,97 0,99 0,98 0,79 0,88 0,92 0,86 0,92 0,99 0,80 0,80 0,99 0,96 0,97

In the next step of the method the meta-classier is used. Its inputs are connected to the outputs of basic classiers (the degrees of the belief for single fault detection). The main task of this meta-classier is to compute the nal result. Tab. 8 presents the sensitivity of dierent types of classiers which are presented in the same form as in the second method of fault isolation (Tab. 5).

In the rst column indicated as All there are included values of the accuracy of the meta-classiers. In the next columns the sensitivity values of single fault isolation obtained by means of meta-classiers are included.

Table 8: The accuracy and sensitivity of fault isolation for meta-classiers with a bank of classiers for isolating single faults

MkNN 0,80 1,00 1,00 1,00 0,28 0,74 0,86 0,94 0,81 0,21 1,00 0,42 1,00 0,99 0,98 MNB 0,76 1,00 1,00 1,00 0,46 0,55 0,93 0,86 0,81 0,18 1,00 0,33 1,00 1,00 0,55 MDT 0,78 1,00 1,00 1,00 0,00 0,60 0,79 0,96 0,81 0,20 1,00 0,86 1,00 1,00 0,65 MRI 0,78 1,00 1,00 1,00 0,32 0,94 0,83 0,79 0,74 0,14 1,00 0,53 1,00 0,96 0,67 MNN 0,79 1,00 1,00 1,00 0,85 0,56 0,85 0,96 0,81 0,00 1,00 0,33 0,98 1,00 0,66 MSVM 0,77 0,99 1,00 1,00 0,04 0,88 0,85 0,96 0,81 0,21 1,00 0,48 0,98 1,00 0,62

Tab. 9 includes the precision of fault isolation based on the third scheme (Fig. 6). This scheme of fault isolation (Fig. 6) was divided into two parts. The rst part was dealt with the selection of the basic applied to isolate


Gulgun Kayakutlu
email: kayakutlu@itu.edu.tr

Irem Duzdar
email: iremduzdar@arel.edu.tr

Eunika Mercier-Laurent
email: e.mercier-laurent@univ-lyon3.fr

Intelligent Association Rules for Innovative SME Collaboration

Keywords: Collaborative Innovation, Association Rules, SVM, SOM

SMEs are encouraged to collaborate for research and innovation in order to survive in tough global competition. Even the technology SMEs with high knowledge capital have the fear to collaborate with other SMEs or bigger companies. This study aims to illuminate the preferences in customer, supplier and competitor collaboration within industry or inter industry. A survey is run on more than 110 companies and Machine Learning methods are used to define the association rules that will lead for success.

Introduction

Knowledge based SMEs need to construct successful alliances in order to have sustainable business in a competitive environment. Global experiences with randomly chosen collaborators have shown failures that caused the fear of new collaborative work. Causes of failure based on the culture and the type of collaboration are studied [START_REF] Kakabadse | The ASP phenomenon: an example of solution innovation that liberates organization from technology or captures it?[END_REF]. The collaboration and creativity must be in the required skills to develop an effective knowledge management [START_REF] Horak | Dealing with human factors and managing change in knowledge management: a phased approach[END_REF] [START_REF] Yahya | Managing human resources toward achieving knowledge management[END_REF]. Abereijo et al. states that the innovation capability of any company depends on its capacity to combine the information assets and the experiences which can be found in the literature about the national innovation studies [START_REF] Abereijo | Technological innovation sources and institutional supports for manufacturing small and medium enterprises in Nigeria[END_REF]. The approaches of owners to innovation, their tendencies and enthusiasm, and the social activities in the company, and dependence among workers are very important factors for effective knowledge management in SMEs [START_REF] Wee | The peculiarities of knowledge management processes in SMEs: the case of Singapore[END_REF]. Alliance in new product development has been the focus of industrial researchers [START_REF] Bouncken | Innovation in knowledge-intensive industries: The double-edged sword of coopetition[END_REF] [7] [START_REF] Narula | R&D collaboration by SMEs: New opportunities and limitations in the face of globalisation[END_REF].

One of the effective tools in knowledge management is becoming the association rules. There are many practices in technology firms and development projects for new products, but it is not used widely for cooperation of companies yet. Basically, to allocate the resources efficiently, the rules must be stated to regulate the information transfer. The well designed rules will be very effective to get benefits to SMEs which have limited capital and sources to continue the innovative operations. It can be seen in literature that the achievement measures and performance studies are useful for innovative collaboration. In business world, these criteria is trying to state a strategy for an individual company without employing the information coming from the knowledge assets of worldwide applications by decision makers.

This study aims to provide a pre-analysis of the path for successful alliances that will lead improvements in innovative power. Both qualitative and quantitative analysis of the SME alliances is realized to find the conditions causing failures and supporting the success in innovation. Support Vector Machine and Self Organized Maps are used to define the most frequent patterns that will give the support and confidence to identify the relationships. Association rules achieved will determine the optimal use of resources. This paper is so organized that the literature review will be given in the second section and the methodology definition will follow. The fourth section will be reserved for presenting the survey and the results. The conclusion will be given in the fifth and last section.

The implication of the study is generic enough to help any SME or research organization or large business to reduce risks in future alliances.

Background

The surveys on the interactions of the industrial stockholders had begun at the early 21st century. It can be seen easily that the expansive applications of outsourcing increases the importance of this matter.

More than 70 publications have been examined to have an understanding about the applications of the rules for cooperation between SMEs aimed at innovation. These studies implied that the main attention was on the various natures of interactions with SMEs and with Large Companies, either in an identified industry or different industries; either only for customers or both of the customers and suppliers. At the end of the detailed examination of this topic, the number of analyzed researches was 112, which were useful to classify on the base of methods.

The first research on Association Rule Mining and Methods is found in 1996 [START_REF]Parallel mining of association rules[END_REF] trying to find the most frequent occurrences of events to support the linked processes. The research in the field followed the timeline shown in Figure 1.

The analysis in engineering branch shows that beginning from 2005, it takes its place increasingly reached 8 researches on annual average. The 25% of these researches were done in 2009, and the maximum number is reached in this year.

Fig. 1. Research timeline on Association Rules

The researches on association rule started in 1997 and the number of publications increased until 2007, most of the studies were done about general subjects in this period. Then the general purpose studies reduced because the specification starts (Figure 2). The annual distribution of the published studies on 'Association Rule Methodology in Data Mining' is researched. The number of published applications about this subject increased until 2006. Between 2009-2011 years there is slight decrease, and then, last 3 years an increasing trend is observed.

Fig. 2. Publication timeline on Association Rules

The usage of association rule in engineering is analyzed and the numbers of publications, the starting time seems 1999, but the reel beginning time is 2005. This matches the starting time of specialization on association rule studies.

Kakabadse et Al. showed that using improved communication and information technologies will improve the SME collaboration [START_REF] Kakabadse | The ASP phenomenon: an example of solution innovation that liberates organization from technology or captures it?[END_REF]. Also Post et al. showed the fact that SMEs would like to collaborate only for developing new products [START_REF] Post | Indicators for establishing SME product development networks[END_REF]. New product based collaboration has evolved fast [START_REF] Narula | R&D collaboration by SMEs: New opportunities and limitations in the face of globalisation[END_REF]. Corporation and competition are found as flaming collaboration types that feed the SME improvements in innovation [START_REF] Gnyawali | Co-opetition and technological innovation in small and medium sized enterprizes A Multilevel Conceptual Model[END_REF]. Association rules defined for the failure types have opened a new dimension for the research on failure of collaboration [START_REF] Zheng | System log pre-processing to improve failure prediction[END_REF]. The first study on mining the SME innovation by Wang et Al has found some patterns for allocating the R&D resources [START_REF] Wang | Mining the R&D innovation performance processes for hightech firms based on rough set theory[END_REF]. Suh & Kim have detailed the R&D collaboration in service industries detected the positive relations of technology and the product or process innovation [START_REF] Suh | Effects of SME collaboration on R&D in the service sector in open innovation[END_REF]. Swarnkar et Al. analyzed when and how the collaboration strategies will be used in virtual organizations [START_REF] Swarnkar | A framework for collaboration moderator services to support knowledge based collaboration[END_REF]. Wiltsey et al. claimed that extent, nature or impact of R&D programs are studied rarely. The interactions among the influences must be given in multiple levels and fidelity and changes must be observed in time [START_REF] Wiltsey Stirman | The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research[END_REF]. Woodland & Hutton introduced the social dimension on the collaborative success [START_REF] Woodland | Evaluating Organizational Collaborations: Suggested Entry Points and Strategies[END_REF]. Both the fear issues and the success causes studied by Bouncken et al. defined technology influencers, sharing the knowledge and learning from the partner as the main influencers [START_REF] Bouncken | Innovation in knowledge-intensive industries: The double-edged sword of coopetition[END_REF]. Bayo-Moriones et al. (2013) proposed that the effects of information and communication technology (ICT) has an indirect on the achievement of increasing the marketing capacity, profitability, and profit rates because of the advanced internal and external communication channels; just like the developed operational processes [START_REF] Bayo-Moriones | Perceived performance effects of ICT in manufacturing SMEs[END_REF]. Sawers et al. concluded that the intrinsic advancement level of SMEs has inverse correlation with the enthusiasm for collaboration, where the effect of extrinsic level has direct correlation [START_REF] Sawers | Safeguarding SMEs dynamic capabilities in technology innovative SME-large company partnerships in South Africa[END_REF]. Rocha showed that empowering the business and the high level of competition at market area will lead to a larger SME industry [START_REF] Rocha | The impact of the business environment on the size of the micro, small and medium enterprise sector; preliminary findings from a cross-country comparison[END_REF]. Franco and Haase introduced various collaborating businesses and stated their general policies, and tendencies to cooperation together with their courses of action [START_REF] Franco | Interfirm alliances: a taxonomy for SMEs[END_REF]. Antlová et al. insisted that the companies and their knowledge assets depending on the basic qualifications exhibit the developing businesses [START_REF] Antlová | Long term growth of SME from the view of ICT competencies and web presentations[END_REF].

Knowledge management and data mining overviews [START_REF] Tsai | Knowledge management vs. data mining: Research trend, forecast and citation approach[END_REF] and Knowledge Management performance studies [START_REF] Tae Hun | Understanding the effect of knowledge management strategies on knowledge management performance: A contingency perspective[END_REF] realized recently do not show any association rule study for the collaborative innovation success and failure.

Methodologies

Association Rules

Given a set of transactions, rules are defined that will exhibit that the occurrence of an item based on the occurrences of other items in the transaction. This is the association analysis. It is useful to explore the interesting relations, which are embedded in the huge data sets. These hidden interactions can be stated in the form of association rules [START_REF] Jackson | Data mining: A conceptual overview[END_REF]. The strength of an association rule is measured with its support and confidence values. Support shows the how often that rule is applicable to a given dataset. The Confidence is the occurrence frequency of the item in that transaction [START_REF]Parallel mining of association rules[END_REF].

Support (s) is the fraction of transactions that contain an itemset

N Y X Y X s ) ( ) (     (1)
Confidence (c) measures how often items in Y appear in transactions that contain X ) (

) ( ) ( X Y X Y X c      (2)
The item set patterns are found in various methods which could be apriori or aposteriori. The overview of all the methods used in association srule studies are given in Figure 3.

Support Vector Machines

It is a machine learning technique, which is mainly introduced for classification in two classes [START_REF] Cortes | Support Vector Networks[END_REF] but further used in clustering [START_REF] Finley | Supervised clustering with support vector machines[END_REF].

It can be analyzed as an optimization problem as in Eq. 3 [START_REF] Haykin | Neural Networks: A Comprehensive Foundation[END_REF] relaxed with Lagrange multipliers in objective function as in equation 4.

 

1 . . 2 1 min 0 2    w x w r t s w z i i i (3)
Data is separated with a hyper-plane multiplied by -1 or +1.

        i i i i i i i p w x w r w L   0 2 2 1 (4)
In 1990s Vapnik pioneered the development of support vector machines specifically for classification in two classes [START_REF] Cortes | Support Vector Networks[END_REF]. This method aims to develop an optimal hyper plane as a decision function using the maximum distance among two class vectors. Support Vector Using a Gaussian Kernel as defined in Eq. 5. will increase the reliability on dissimilarities [START_REF] Cherkassky | Practical selection of SVM parameters and noise estimation for SVM regression[END_REF].

           2 exp ) , ( 2 i i x x x x K (5)

Self Organized Maps

Self-Organizing Map (SOM) is a widely used artificial neural network technique in clustering with unsupervised learning algorithm. This technique clusters according to the similarities to the input data [START_REF] Leopold | Data Mining and Text Mining for Science & Technology Research[END_REF]. SOMs structure the output with individual node similarity as well as cluster center distance. This technique is based on competitive learning, where the output nodes are made of the winning node activated by one input node. The output nodes would have scoring values using a function, most commonly Euclidean distance between the inputs and weights. For each input vector x, and for each output node j, the value D (w j , x n ) of the scoring function. Euclidean distance function is shown in Eq. 6.

2 ) ( ,

ni ij n j x w x w D   (6)
The winning node therefore becomes the center of a neighborhood of excited nodes. In selforganizing maps, all nodes in the given neighborhood share competition. Therefore, even if the nodes in the output layer are not connected directly to the input layer, they tend to share common features, of the neighborhood [START_REF] Larose | Discovering Knowledge in Data: An Introduction to Data Mining[END_REF]. The nodes in the neighborhood of the winning node participate in adaptation, which is, learning. The weights of these nodes are adjusted to improve the weights defined in Eq. 7., until a threshold is reached.

current w x current w new w ij ni ij ij     (7)
In Eq 7. α is the learning rate. If it is necessary, the learning rate and neighborhood size are adjusted.

Application

A survey is run with the technology firms sited in Techno-parks of linked, 5 are about competences and 4 the technology choices. 130 firms responded but only 105 are included in the analysis. 14% of the companies were medium size and 37 % of them were aged more than 10 years. They have chosen the type of collaboration among the SME and Big firms as well as among the customers, suppliers and competitors as shown in Figure 4. The reason for innovative collaboration is stated as shown in Figure 5. SVM is applied to classify the collaboration made specifically for innovation, validated by Cronbach alpha, that resulted as more than 0.7. SOM is used to cluster those to show the supporting frequencies of choices. Then the cross tables are achieved as a basis for the association rules. The first achievement was micro companies would like all the collaborators to have innovation culture which is less important for the bigger companies. Some other samples of cross tables are given as below.

Table 1 gives the fact that majority of responders prefer for at least one collaborator to have design competence.

Table 1 Neglig. 1

In the firms collaborating for innovation for 1 to 3 years for the success of innovation it is necessary understanding the market requirements by all the firms together with the well-developed innovation culture.

Results

The Reliability analyzes have been done on the results of the questionnaires, the Cornbach's Alpha value is 0.602; this value is in the acceptable range. In the logistic regression model, the determined significance level is 0.100. The values of attributes of innovation are neglected, since they are greater than 0.100. Innovation related criteria in this study have no significance based on firm size and collaborator types of these firms. In other words, firms care the technological features but ignore the innovative attributes. In the table below, the significance values are shown for the other 3 attributes and the results of analyzes for features of the questions. The significance values less than 0.100 are used here (Figure 6).

Fig. 6. Significance values for each clusters based on demographic properties

After the multinomial logistic regression analysis coefficients for all attributes are obtained. The statistically significant attributes are used. Coefficients of financially related criteria are shown in Figure 7.

Fig. 7. Regression Coefficients & Significancies

The cross relation tables have been constructed to define the rules obtained from the model.

Finance Related Criteria :

MEANING:

 Small SMEs emphasize that capital of the collaborators are to be more than the sector average and exportation facilities are done by only one collaborator for innovation (Figure 9).

Fig. 10. Crossrelations Table: Collaboration -Firm Size -Technological Criteria

Figure 10 shows the crossrelations between collaborator type (customersupplier -SMEslarge firms) and firms' size for the technology related criteria. Also this mentioned model makes sense statistically significant as a result of the logistic regression analysis.

RULE 2:

IF

(Firm Size = "Small" years old" AND Collaborate = "Customers" and "LFs")

THEN

(Communication technologies = "All opportunities" AND Change Management = "Individual")

MEANING:

 Small SMEs who are collaborating with the customers which are large firms (LFs) emphasize usage of all communication technologies opportunities and change management is can be individual choice (Figure 9 ).

Management Related Criteria :

MEANING:

 Medium size SMEs of age 5-10 years prefer to collaborate with companies which have effective organizational structure with both cooperation and coordination attitude; in the collaboration a single leader is preferred and professionalism can be taken only as the motivator.

RULE 2:

IF

(Firm Size = "Micro" AND Collaborate = "Customers" and "SMEs") THEN (Professionalism = "All" AND Business Experience = "All" AND Leadership = "All")

MEANING:

 The micro SMEs collaborating with the customers emphasize professionalism, business experience for the problem solving and they prefer all the collaborators to have the leadership features.

This study investigates the most preferred conditions for a successful collaboration for innovative SMEs. SVM and SOM are used to construct the basis for creating the association rules. As the result of a survey in Turkey, there are hundreds of relations depicted in the analysis.

The achievements are interesting enough to show that the technology companies are confused in differentiating the technology and innovation concepts. It was interesting to observe micro and young companies not willing to collaborate with the big and overwhelming companies. Everybody asks for full communication technology, but only small SME with 5-10 years of experience ask for the collaborators to have effective organization and full professionalism.

The validation by logistic regression on the same data is in process. All the results achieved using logistic regression will be cross-validated with machine learning application results. Future survey will be aiming to improve the innovation concept of the technology firms in detail.

Fig. 3 .

 3 Fig. 3. Overview of Association rule techniques

Fig. 4 .Fig. 5 .

 45 Fig. 4. Choice of collaboration according to size and relation



  = "Micro" AND Firm Age ≤ "1" ) THEN (Innovation operation expenditure = "proportionally shared" AND Price = "important")MEANING: Preferences for the initiating micro SMEs emphasize the innovation operation expendituresaccording to the collaborator sharings and market value (price) of the innovated product (Figure8).

Fig. 8 .

 8 Fig. 8. Crossrelations Table: Firm Size -Firm Age -Finance Related Criteria

Fig. 9 .

 9 Fig. 9. Crossrelations Table: Collaboration -Firm Size -Finance Related Criteria

  = "Medium" AND Firm Age = "5-10 years old") THEN (Professionalism = "Motivation" AND Organizational Structure = "Effective" AND Cooperation & Coordination = "All" AND Leadership = "Only one")

. Business Understanding and Design Competence

					DESIGN
					At least	Neglig
	INNOV			All	one	e
	Collaboration for others	Bus-Under	All	4	14
			At least one	2	15
	Collaboration for innovation	Bus-Under	All	14	26	4
			At least
			one	7	13	2

Table 2 . Age-Innovation Culture and Market Requirements Relation

 2

					INNO_CULT
					At least
	COLLAB_Age		All	one	Neglige
	0	UNDERSTD_REQ	All	13	8
			More than one	6	6
			At least one	0	1
	<1 year	UNDERSTD_REQ	All	9	1	0
			More than one	3	1	1
	1-3 years UNDERSTD_REQ	All	13	1
			More than one	1	1
			At least one	4	2
	3-5 years UNDERSTD_REQ	All	6	1
			More than one	3	2
			At least one	0	1
	>5	UNDERSTD_REQ	All	8	2	0
			More than one	3	2	2
			At least one
				2	2	0