Jacek Kabziński
email: jacek.kabzinski@p.lodz.pl

Jarosław Kacerka
email: jaroslaw.kacerka@p.lodz.pl

TSK Fuzzy Modeling with Nonlinear Consequences

Keywords: fuzzy modeling, TSK fuzzy model, fuzzy model training

We propose to generalize TSK fuzzy model applying nonlinear functions in the rule consequences. We provide the model description and parameterization and discus the problem of model training and we recommend PSO for tuning parameters in membership functions and in nonlinear part of a rule consequence. We also propose some more or less formalized approach to nonlinear consequence selection and construction. Several examples demonstrate the main features of the proposed fuzzy models. The proposed approach reduces the average obtained model Root Mean Square Error (RMSE) with regard to the linear fuzzy model, as well that it allows to reduce the model complexity preserving the desired accuracy.

Introduction

Among various fuzzy modeling techniques the approach proposed by Takagi, Sugeno and Kang [START_REF] Takagi | Fuzzy Identification on Systems and its Applications to Modeling and Control[END_REF][START_REF] Sugeno | Structure Identification of Fuzzy Model[END_REF], so called TSK model, remains one of the most popular and effective. The standard TSK model employs affine functions in consequences of fuzzy rules. So the idea of modeling is based on local linear models dominating if the rule activation strength is high. The model possesses the universal approximation property [START_REF] Nguyen | Fuzzy Systems: Modeling and Control[END_REF] but in many practical problems the number of rules to achieve the desired accuracy is really high. Motivated by many experiences from modeling mechatronic systems (see example 2 below) we claim that it is possible to propose local nonlinear models in many practical applications. Having some knowledge on the nature of the modeled phenomenon we may foresee nonlinear functions that should appear in the model, even if we are not sure about the exact parameters of these functions.

In this contribution we propose to generalize TSK model by accepting nonlinear functions of model inputs in fuzzy rule consequences. We propose a formal model description and parameterization. Next we consider the problem of model training from the numerical data, which is more difficult than in the standard case. We also propose some formal methods to develop nonlinear consequences. The aim of the proposed modification is to reduce the model complexity preserving the desired accuracy, and we demonstrate on several examples that that it really happens.

The idea of using nonlinear functions in fuzzy rule consequences was already mentioned in literature, but only particular cases were considered. In [START_REF] Chen | Fuzzy Control Design for Switched Nonlinear Systems[END_REF] switched Takagi-Sugeno models with an affine nonlinear consequent part are used to control switched nonlinear dynamical systems and in [START_REF] Rajeshm | T-S fuzzy model with nonlinear consequence and PDC controller for a class of nonlinear control systems[END_REF] it is proved that under some special assumptions such fuzzy model can approximate a particular class of nonlinear functions, nonlinear dynamic systems and nonlinear control systems.

Fuzzy TSK System with Nonlinear Consequences

We consider a fuzzy, single output, multi input model given by the rules

: … = , … , , (1)
where = , , … , is the vector of inputs, = 1, … , is the rule number, µ ij are membership functions, each defined by three parameters ̅ , ̿ , ̌ , and y i are consequences = , , … , = , " , # , $ , % + , " , # , $ , % + ⋯ + ,( ) " ,( ) # , $ ,( ) %.

(

) 2
Parameters of the consequences are organized as follows: p i,j is a s i,j -dimensional vector parameterizing nonlinear function φ i,j while a i,j are scalars. For each rule we define its activation level as

+ = ∏ # % - (3)
and denote

. = ∑ + 0 - (4)
The model output is calculated from

= ∑ 1 ) 2 3 )45 ∑ 6+ 7. 0 - (5)
If we represent the vector of linear combination parameters as = 8 , , … , ,( 5 , , , … , ,( 9 , … , 0, , … , 0,( 3 :

or

; = 8 , , … , ,( ) : , = < ; ⋮ ; 0 > (7)
and we refer to the vector of activated consequences: ? = + 8" , # , $ , %, … , " ,( ) # , $ ,( ) %:, ? = 6? , … , ? 0 7,

we get a short description of the system output:

= @ 2 ? , (9)
which stresses that the model is linearly parameterized by a. This important feature will be utilized during model training and makes such model attractive in adaptive control applications.

If the same set of functions appears in each of the consequences, i.e.

A = A = ⋯ = A 0 = A, " , # , $ , % = " # , $ %, B = 1,2, … , A (10)
we may describe the model output as

= @ 2 6" , $ , … , " ( , $ ( 76 ; , … , ; ( 7 < + ⋮ + 0 > (11)
and after accepting notation

D = 6" , $ " , $ ⋯ " ( , $ ( 7, E F = < , ⋯ 0, ⋮ ⋱ ⋮ ,( ⋯ 0,( > H = < + ⋮ + 0 > 1 .
(12) we have another short output description

= D E F H. (13)
Similarly to the classical TSK model with linear consequences our model may be represented as a multi-layer neural network. In the first layer actual values of the membership functions are calculated, in the second the activation level for the each rule is calculated, in the third layer normalized activation strength

+ I J = 1 ) 2 @ 2
of each rule is computed, in the fourth products + I J are obtained and the final node performs the summation = ∑ + I J .

Model Training

The task to construct a TSK fuzzy model can be divided into two steps: first we have to design the model structure -recognize relevant inputs, decide about membership functions, plan the rules and the consequences etc. While the structure is established we optimize the system parameters based on some numerical data representing the desired behavior of the model -this stage of modeling is usually called model training or tuning.

In case of the proposed model with nonlinear consequences the model structure selection may be performed using any standard approach as placing membership functions on a grid, by clustering, or others [START_REF] Kabziński | One-Dimensional Linear Local Prototypes for Effective Selection of Neuro-Fuzzy Sugeno Model Initial Structure[END_REF]. The problem of nonlinear consequences selection is discussed separately. Here we concentrate on the model training methods.

The proposed model parameters may be divided into three sets: parameters of membership functions (parameters µ), parameters that appear nonlinearly in the consequences (parameters p) and parameters that are coefficients of linear combination in consequences (parameters a). The training data is collected in a M-dimensional target output vector K ; and each entry ; corresponds to an input vector while the model output for the same input is . The aim of training is to find model parameters minimizing the RMSE

L = M N ∑ ; - N - → A . ( 14
)
Calculation of optimal parameters a, for given parameters µ and p, is trivial since according to (9) a is the solution in the least squares sense to system of equations

K ; = Φ , Φ = R @ 2 5 ? ⋮ @#2 S % ? N T , = Φ\K ; . ( 15
)
Unfortunately the target function J in ( 14) is heavily nonlinear function of parameters µ and p. It may be nonsmooth and possess many local minima. So we propose to apply evolutionary optimization to find optimal model parameters. We have experimented with many evolutional optimization algorithms and finally we recommend PSO as well suitable for the problem. We use a well-known version as described in [START_REF] Shi | A modified particle swarm optimizer[END_REF]. Parameters [µ , p] are coded as particles X. Each calculation of the fitting function for the given [µ , p] is done as follows: calculate Φ and a from (15), next calculate J from (14).

Examples

The proposed model training approach was tested on several modeling problems. Example 1. Let us consider the following nonlinear functions defined for , ∈ 6-1,17 and plotted in fig. 1 and2 = --⋅ cos#13 ⋅ + %,

= -- ⋅ \ | | | | -0.2 ⋅ \ | | | | -0.2 (16)
and assume that we know the general form of these functions as

= + , ⋅ cos $ ⋅ + , ( 18
) = + , ⋅ \ | | | | -$ ⋅ \ | | | | -$ . ( 19
)
This assumption motivates the following choice of consequences in rule number i

y = `+ ⋅ + ⋅ + a ⋅ cos $ ⋅ + (20) y = `+ ⋅ + ⋅ + a ⋅ \ | | | | -$ ⋅ \ | | | | -$ (21)
The fuzzy model structure was designed by selecting membership functions

# % = bc d e fg )e h )e i 9j )e (22)
initialized on an uniform grid. We have tested the performance of modeling by: standard TSK model with linear consequences trained by ANFIS (adaptive-networkbased fuzzy inference system [START_REF] Jang | ANFIS: Adaptive-network-based fuzzy inference system[END_REF] Additional experiments demenstrated that both GA and PSO benefit from estimating the value of nonlinear p parameters prior to optimizations. Such estimation was performed by minimizing the value of the same RMSE objective with constant parameters of membership functions, i.e. membership functions were initialized as a uniform grid and remained unchanged -only p parameters were tuned by GA or PSO for a low number of iterations. During the main optimization the search space for p was limited to the neighborhood of the estimated value. Such two stage optimization results in further model improvement, e.g. for a 4 rule model the average RMSE is 0.029 for GA tuning and 0.006 for PSO.

Example 2. The data presented in fig. 3 describes the q-axis flux component of a permanent magnet synchronous motor as function of current and rotor position. Acceptable fuzzy model with affine consequences requires at least N=11 rules. It is visible that for currents bigger than 1A we observe flux oscillations with rotor position. The frequency of these oscillations is 6 times bigger than the rotor angular speed. Hence we propose nonlinear consequences

= + + `+ a 6 + l mn 6 , = 1 … (23)
where x 1 is a rotor position and x 2 is q-axis current. With this it was possible to reduce the number of rules and the model with 4 rules only was equivalent to 11 rules model with linear consequences, as it is demonstrated in table 2. Presented error is calculated as RMSE for the whole set of collected data (80 000 points), while model training was based on points equal to local average of measurements over a grid of 900 points. strated in table 3 and shown in figures 5 and 6 presenting modeling error of a sample linear and nonlinear model. To illustrate the complexity of linear model required to match the accuracy of nonlinear models we continue constructing of linear models for as many as 144 rules trained by ANFIS. The resulting model's RMSE is similar to error of a 4-rule nonlinear model.

Conclusions

We propose to generalize TSK fuzzy model applying nonlinear functions in the rule consequences. We provide the model description and parameterization and discus the problem of model training and we recommend PSO for tuning parameters in membership functions and in nonlinear part of a rule consequence. We also propose some more or less formalized approach to nonlinear consequence selection and construction. Several examples demonstrate the main features of the proposed fuzzy models.

The proposed approach allows to reduce the model complexity preserving the desired accuracy. A model with smaller number of rules is simpler for real time application and easier interpretable. Nonlinear consequences enable modeling of function imperfectly approximated by standard TSK models, for example discontinuous functions [START_REF] Jastrzebski | A generalized Takagi-Sugeno-Kang model designed for approximation of discontinuous dependeces[END_REF]. As the constructed models are linear in some of parameters they may be efficiently utilized in adaptive control of dynamical systems, as it was proposed for example in [START_REF] Kabzinski | Fuzzy Friction Modeling for Adaptive Control of Mechatronic Systems[END_REF]. It is also important that the proposed model may be described as a neural network and so NN-training techniques may be utilized for the model tuning.

Fig. 1 .Fig. 2 .

 12 Fig. 1. Example function f 1 (x 1 ,x 2 ) Fig. 2. Example function f 2 (x 1 ,x 2 )

Fig. 3 .

 3 Fig. 3. The q-axis motor flux (Ψ q [Wb]) as function of motor current (i q [A]) and rotor position (θ [rad]). The data collected from the flux observer.

Fig. 4 .

 4 Fig. 4. Fuzzy model output corresponding to the data from fig. 3. The model with 4 rules and consequences (23) trained by PSO.0

Fig. 6 .

 6 Fig. 6. Modeling error of function (16) for a 9-rule PSO trained model with nonlinear consequents given by eq. (34)

Table 1 .

 1 RMSE of linear model tuned by ANFIS and nonlinear models tuned by GA and PSO. Presented RMSE of PSO and GA tuned models is an average of 10 algorithm executions.

	Rules	ANFIS	GA	PSO
	4	0.5643	0.5369	0.0271
	9	0.5611	0.5149	0.0017
	16	0.5281	0.4962	5.5e-4
	25	0.4845	0.1165	2.0e-5
	36	0.4071	0.0523	3.7e-5
	49	0.3266	0.0345	6.3e-5

, the widely accepted standard for TSK systems training), TSK model with nonlinear consequences tuned by genetic algorithm (GA implemented as in Matlab Global Optimization Toolbox), TSK model with nonlinear consequences tuned by PSO. Results of modeling of function (14) in terms of average obtained model RMSE are presented in table 1. It may be observed, that modeling error for nonlinear models is lower regardless of model complexity, i.e. number of rules. PSO tuning results in a much more precise model -error is lower by 1 to 3 orders of magnitude than for GA tuned model.

Table 3 .

 3 RMSE of linear (ANFIS) and nonlinear models: PSO additive (eq. (32)), PSO constant (33), PSO lin.apx. (34) for 4 and 9 rules. Higher number of rules modeled for comparison for linear model only. Presented RMSE of PSO tuned models is an average of 10 algorithm executions.

	Rules	ANFIS	PSO additive	PSO constant	PSO lin.apx.
	4	0.5643	0.0271	0.0461	0.0024
	9	0.5611	0.0017	0.0014	0.0003
	81	0.1921	-	-
	100	0.0693	-	-	-
	144	0.0254	-	-	-

Fig. 5. Modeling error of function (16) for a 144-rule ANFIS trained model with linear consequents.

Nonlinear consequences construction

The obvious problem we have to solve constructing a TSK model with nonlinear consequences is how to select and parameterize nonlinear functions that appear in the consequences. Usually, in practical problems, the expert knowledge is sufficient to propose the consequences, as it was demonstrated in example 2. We may also propose a more formal approach.

Additive correction approach If we may assume (from an expert knowledge for example) what kind of nonlinearities may be present in the system model we add these nonlinearities to the consequences. Finally we get the consequence for rule number i

The first part of ( 24) is a standard affine function as in classical TSK model and the last one is selected to cope with expected nonlinearities. This approach is especially useful if we expect discontinuities in the data -for example friction is discontinuous at zero velocity. Consequents similar to (24) were applied in all examples presented above and prove to be effective.

Constant coefficient approach

Let us consider a function : → to be modelled and assume that it may be represented on a compact set A as

Assume that \ , $ is known, although the actual values of constant parameters p may be unknown. Assume that A is covered by a grid defined by selection of points for each axis:

We denote the middle points of the grid by

and define the rules

Of course the initial guess for 5 ,…, t is # ̅ 5 ,…, t %, ̅ 5 ,…, t = ̅ , 5 , … , ̅ , t , or the average value of the data representing + in the part of the grid around ̅ 5 ,…, t , if such information is available.

Linear approximation approach

Assuming that + is sufficiently smooth, it may be approximated by

what motivates the rules

Example 3. Proposed methods of constructing rule consequences were applied to modeling of example function ( 16). The tested model consisted of 4 or 9 rules with consequences defined as:

• standard affine:

• nonlinear -consisting of linear part and added nonlinear function:

• nonlinear with constant coefficients:

• nonlinear constructed according to the linear approximation approach:

Model with linear consequences was trained by ANFIS, while nonlinear models were trained by PSO. The objective of training was to minimize the RMSE model error. Nonlinear models -gained much better accuracy than the linear model as is demon-


Juhyun Oh
email: jhoh@kbs.co.kr

Seonggyu Jeon

Minho Kim

Hyukchul Kwon
email: hckwon@pnu.ac.kr

Iktae Kim

An Avatar-Based Weather Forecast Sign Language System for the Hearing-Impaired

Keywords: sign language, avatar animation, closed caption, machine translation, wordnet

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Closed caption and sign language broadcasts are provided with the terrestrial digital television (DTV) services in Korea for the hearing-impaired people. Even though closed captions are provided for almost the whole broadcast time, sign language broadcasting covers only about 5% of it. We propose a system that translates the closed captions of weather forecast programs into Korean sign language (KSL) and present it with three-dimensional (3D) avatar animation. The translated sign language data are sent via the Internet for the receivers such as personal computer (PC) and mobile devices to show the corresponding sign language animation. The system consists of the Korean-KSL translator, the sign language avatar animation system, and the server system that provides the closed caption and video for the most recent weather forecast in an on-demand manner.

A similar system for Japanese sign language (JSL) was proposed to use TV program making language (TVML) [START_REF] Kaneko | Sign language animation using TVML[END_REF][START_REF] Kato | Machine translation to Sign Language with CG-animation[END_REF] for high-quality computer animation. However the translation is not automatic in their work and the user should input the sign language words in the order of JSL. For Brazilian sign language (LIBRAS), automatic translation and middleware structure for DTV was also proposed in [START_REF] Araújo | Automatic generation of Brazilian sign language windows for digital TV systems[END_REF][START_REF] Ferreira | Providing support for sign language in middlewares compliant with ITU J[END_REF]. Similarly to the existing studies, the purpose of the proposed system in this paper is to provide information when human interpreters are not available and not to replace human interpreters.

This paper is organized as follows. In Section 2, the Korean-KSL translation method is described. In Section 3, the 3D avatar animation scheme is introduced. In Section 4, the system to provide the weather forecast sign language service is described. The experimental results and conclusion are given in Section 5 and 6 respectively.

Korean-KSL Translation

Table 1 shows an example of Korean-KSL translation in a couple of weather forecast sentences by a professional sign language interpreter. Our goal is to implement an automatic translator that outputs similar KSL results when the same Korean weather forecast input is given. Table 1. An example of manual Korean-KSL translation in weather forecast scripts, expressed in English.

Korean-KSL Dictionary

There are about 10,000 words in KSL Dictionary [5]. It is very difficult to build the Korean-KSL dictionary and capture the motions for all these 10,000 words. Thus we investigated the weather forecast scripts for the past three years, from Korean Broadcasting System (KBS) and a few other sources. After some preprocessing of the weather forecast scripts, the scripts are divided by part of speech (POS). For this purpose we use the POS tagger of Pusan National University with about 1 million registered words [START_REF] Kang | Variable-Length Category-Based N-Gram Part-of-Speech Tagging Model Considering Morph-Syntactic Characteristics of Korean[END_REF]. The accuracy of the POS tagger is about 98%. Since the basic word orders of Korean and KSL are similar as subject-object-verb (S-O-V), direct word-toword translation rule is applied in the proposed system.

Table 2 shows ten most frequent words for noun, adjective, verb, and adverb in the analyzed scripts. Notice that the POS in Korean is different from that of English for some words. In Table 2, the most frequently used noun and adjective are 'temperature' and 'high' respectively. The cumulative frequency in Table 2 is important because it shows the translation capability when that word and the above ones in the table were registered in the dictionary. For example, if only ten most frequently used nouns were registered in the dictionary and motion-captured, the system can probabilistically process 33% of the nouns in the script.

Korean

Recently it is cold at every weekend.

KSL week + end + cold Korean

It is raining from the early morning in the southern area and Jeju Island. KSL warm + place + and + Jeju Island + place + morning + from + rain

KSL Synonym Dictionary

Synonyms are translated into same sign language words. For example, 'house', 'housing' and 'abode' are synonyms and all translated into one sign language word. The synonym dictionary is built based on KorLex [START_REF] Yoon | Construction of Korean Wordnet KorLex 1.5[END_REF], as shown in Fig. 1. Using this synonym dictionary, a word that is absent in the KSL dictionary can be translated to a synonym, increasing the translation success rate. Without the synonym dictionary, non-registered words would have been represented with finger spelling, decreasing the usefulness of the system.

Word Sense Disambiguation with KorLex

In Korean, the word '눈' (pronounced as 'nun') can mean 'eye' or 'snow'. This kind of ambiguity should be resolved in the context. A simple rule to determine the word '눈' as 'snow' is shown in Table 3. If the context words (CW) exists around the ambiguous word, then it is translated to 'snow'. Table 3. A simple rule for the ambiguous word '눈'

눈 CW → snow (R1) CW = [ 내리다(fall) | 날리다(fly) | 흩날리다(flutter)]
This kind of rules suppose the exact match to the CWs, therefore its recall is low. We replace the CWs with the synonym set number in KorLex, to increase the recall. Case particle restriction (R3) and CW verb conjugation restriction (R4) are also applied, as shown in Table 4.

Sign Language Avatar Animation

We use about 2,700 sign language motions for daily life owned by Primpo Inc [8]. Additionally, we captured 507 words dedicated for weather forecast at a studio. 15 Vicon motion capture cameras [9] and 41 infra-red markers were used to capture the motion of a professional sign language interpreter, as shown in Fig. 2. Cyberglove [START_REF]CyberGlove II brochure[END_REF] was used to capture the hand motion.

Fig. 2. The sign language interpreter (left) and the corresponding captured motion (right).

A 3D female avatar model was built for sign language animation. Fig. 3 shows the avatar model and bones associated to it. When animating a sign language sentence, two contiguous sign words are overlapped to each other similarly in human sign language. 15 frames at the end of each sign word are blended with those of the following word, using a linear motion blending equation [START_REF] Slot | Motion Blending[END_REF]. Let two sign word motions to blend be M i and M j , and the k-th joint of concern J ik and J jk . If the two motions are blended with the weights w i and w j respectively, the blended position of the k-th joint is given as

k k blended i i j j k i j w w w w ⋅ + ⋅ = + J J J (1)
where 0≤w i ≤1 and w j =1-w i in this case. Similarly the blended quaternion q blended is given as ( )

1 j w blended i i j q q q q - = (2)
where q i and q j represent the quaternions to be blended [START_REF] Shoemake | Animating rotation with quaternion curves[END_REF]. Fig. 4 shows a blended motion frame using the original motions of two different sign words.

The Weather Forecast Sign Language System

In the service scenario, when a hearing-impaired user launches the 'weather forecast' application on a PC or mobile devices, the latest weather forecast video is provided with sign language animation. Fig. 5 shows the system for the weather forecast sign language service. The 'caption processing system' asks the news server for the latest weather forecast information. If there is a new weather forecast, it inquires the corresponding closed caption from the production server, using the program code of the weather forecast. The weather forecast video in Flash Video (FLV) format originally targeted for real-time message protocol (RTMP) streaming is transcoded to H.264 (MP4). The closed caption is translated into KSL with the method described in section 2 in the sign language broadcasting server. The closed caption in KSL is distributed to clients using an XML web service. In a client device, weather forecast video and sign language animation are composited together and presented to the hearing-impaired user. Fig. 6 shows the simplified XML schema for the KSL translation result.

Experimental Results

Fig. 7 shows the result screens on a PC and an Android phone. We use Unity3D engine [13] to develop an application for both PC and mobile devices. The avatar can be moved to left or right, and can be enlarged or reduced, as the hearing-impaired users want the sign language broadcasting to be. To our best knowledge, there has been no similar research that translates weather forecast texts into KSL to generate sign language animation. Hence we quantitatively compare the translation results in Table 5, with and without the Korlex-based synonym dictionary and word sense disambiguation. In Table 5 the word translation rate is given as the ratio of the number of the words for which correct translation words were chosen, to the number of total words in the corpus. By applying KorLex the translation rates were increased, especially in terms of the translation failures. For example, the translation failure rate for the external corpus was reduced from 11.4% to 9.32% by applying KorLex. Notice that this is only a simplified measure for the evaluation purpose, because the characteristics of sign language such as non-manual signals and spatial representation are difficult to include in the evaluation, and were not considered in this paper.

Conclusion

The system proposed in this paper generates 3D sign language animation by translating the closed captions in DTV, for the hearing-impaired people to view the weather forecast with sign language. In order to find the frequency of each word, we analyzed the last three years' weather forecast scripts from several sources. We built sign language synonym dictionary using KorLex, to improve the translation performance. KorLex was also used for the word sense disambiguation process. We captured the motions of the additional 500 words for the weather forecast purpose. The motions were applied to a 3D avatar with motion blending. The servers and applications were developed to show weather forecast video and avatar animation in an on-demand manner.

Although the proposed system is targeted only for KSL users, we believe that it will be helpful to the other similar researches, especially if the source language and target sign language have a similar word order as in Korean and KSL.

In the future work, we will expand the translator and motion database for all kind of TV programs other than the weather forecast, and develop a real-time sign language broadcasting system. Constructing a sign language corpus using daily TV programs can be another future work.

Fig. 1 .

 1 Fig. 1. The word '집(house)' found in KorLex.

Table 4 .

 4 Modified rule for the ambiguous word '눈'

Fig. 3 .

 3 Fig. 3. 3D avatar and bones.

Fig. 4 .

 4 Fig. 4. Motion blending.

Fig. 5 .

 5 Fig. 5. Weather forecast sign language system

Fig. 6 .

 6 Fig. 6. Simplified XML schema for the KSL translation result

Fig. 7 .

 7 Fig. 7. Sign language animation on a PC (top) and an Android (bottom) phone.

Table 2 .

 2 Ten most frequently used noun, adjective, verb, adverb words (expressed in English) in the weather forecast scripts. C.F. means the cumulative frequency of each POS.

	Noun		Adjective	Verb	Adverb
	Word (Kor/Eng)	Frequency (times)	C.F. (%)	Word Freq. C.F. Word Freq. C.F. Word Freq. C.F.
	temperature 4,180	6 high 1,277 14 go down 1,451 11	a little	742 11
	tomorrow	3,295 11 many 1,173 27 rise 854 18 gradually 702 21
	day	2,664 15 exist 1,125 40 come 844 24	mostly 628 30
	today	2,602 19 clear 841 49 rise 821 31	again	576 38
	rain	2,092 22 low 798 58 continue 737 36 somewhat 470 45
	Seoul	1,747 24 strong 609 65 hang 712 42	more	417 51
	morning	1,611 27 similar 480 70 take 628 46 occasionally 284 55
	region	1,585 29 large 376 74 pass by 493 50	again	269 59
	central area	1,426 31 dark 343 78 fall 440 54 especially 204 61
	whole country 1,293 33 cold 313 81 be seen 430 57	but	185 64

Table 5 .

 5 Quantatative comparison of the word translation rate.

		Corpus size	Word translation rate
		(words)	without KorLex	with KorLex
	Internal corpus	82,303	95.87%	96.17%
	External corpus	1,448	88.60%	90.68%


Velislava Stoykova
email: vstoykova@yahoo.com

Interpretation of Possessive and Reflexive-possessive Pronouns of Bulgarian Language in DATR

Keywords: Non-monotonic Reasoning, Knowledge Acquisition and Representation, Natural Language Processing

The paper presents a formal interpretation of possessive and reflexivepossessive pronouns in Bulgarian language using non-monotonic approach to formal logical representation. It offers a semantic network to model inflectional morphology of definite article for related pronouns using DATR language for lexical knowledge representation. The proposed encoding combines non-monotonic orthogonal semantic networks hierarchical representation and relational database approach. Finally, some conclusions about presenting inflectional morphology using non-monotonic semantic networks approach are presented.

Introduction

Non-monotonic approach has been widely used for formal representation in various Natural Language Processing tasks. Its main advantage is that it allows formal account for both regular and irregular cases and is suitable for modeling any type of grammar feature. Also, some knowledge representation languages which allow non-monotonic interpretation have been successfully used to model various types of language features (like inflection, syntax, etc.) for related and non-related natural languages. Further, we are going to present a formal interpretation of inflectional morphology of possessive and reflexive-possessive pronouns in Bulgarian language by using DATR language for lexical knowledge representation as a suitable formal framework.

The DATR language

The DATR language for lexical knowledge representation is a non-monotonic language for defining the inheritance networks through path/value equations [START_REF] Evans | DATR: A language for lexical knowledge representation[END_REF]. It has both an explicit declarative semantics and an explicit theory of inference allowing efficient implementation, and at the same time, it has necessary expressive power to encode lexical entries presupposed by the work in the unification grammar tradition [START_REF] Evans | Inference in DATR[END_REF][START_REF] Evans | The semantics of DATR[END_REF].

In DATR information is organized as a network of nodes, where a node is a collection of related information. Each node has associated with it a set of equations that define partial functions from paths to values where paths and values are both sequences of atoms. Atoms in paths are sometimes referred to as attributes.

DATR is functional, it defines mapping which assigns unique values to node attributepath pair, and the recovery of these values is deterministic. With respect to its universality, DATR's formal properties and techniques underlay both rule-based inference and non-monotonic inference by default, and allow to account for language phenomena such as regularity, irregularity, and subregularity by using deterministic parsing.

The semantics of DATR uses non-monotonic inference and default inheritance, and allows generalization-capturing representation of inflectional morphology. DATR has expressive power which is capable to encode and process both syntactic and morphological rules and it allows representation of grammar knowledge by using semantic networks.

The DATR language for lexical knowledge representation offers syntagmatic operators (which can be used to define concatenation procedures) and paradigmatic operators (which can be used to define specific structure of the inflecting rules, and for further development of part-of-speech interpretations).

The DATR language has a lot of implementations, however, our application was made by using QDATR 2.0 (consult URL http://www.cogs.susx.ac.uk/lab/ /nlp/datr/datrnode49.html for a related file bul_det.dtr). This PRO-LOG encoding uses Sussex DATR notation [START_REF]PossPron: <third sing_p masc_p plur def_1> == negovite[END_REF].

DATR allows construction of various types of language models (language theories), however, the analysed interpretations underlay architecture of a rule-based formal grammar and a lexical database. The particular queries to be evaluated are all related inflected word forms, and the implementation allows to process words in Cyrillic alphabet.

The DATR language has been applied for developing inflectional morphology for a lot of languages including Russian [START_REF] Corbett | Network Morphology: a DATR account of Russian nominal inflection[END_REF] and Polish [START_REF] Czuba | The DATR Web Pages at Sussex[END_REF]. The ideas used for Russian nominal inflection interpretation underlay that of a paradigm and the encoding presents a resolving of a tabular conceptualization encoding task.

Bulgarian possessive pronouns in DATR

The semantics of possessive pronouns in Bulgarian language includes various types of relation like: possession (depending whether it is an object or a subject of possession), part-of-whole, relational, etc. The main semantic relationship of possession varies depending whether it is referred to the possessor or to the thing being possessed [START_REF] Nicolova | The Bulgarian Pronouns Nauka i izkustvo[END_REF]. Only the full forms of possessive pronouns have inflection. They are: 'moj' (my), 'tvoj' (your), 'negov' (his), 'nein' (her), 'nash' (our), 'vash' (your), 'tehen' (their).

They have grammar features of person, number, gender and definiteness. The grammar feature of person is not inflectional and expresses information both at the level of syntax and at the hypertext level through agreement. The full forms imply information both about the possessor and the object being possessed using agreement in number and gender.

The grammar feature of definiteness implies information about the possession at the syntactic level using agreement and is expressed by a formal morphological marker which is an ending morpheme [START_REF]Gramatika na suvremennia bulgarski knizoven ezik[END_REF]. It is different for genders however, for masculine gender two types of definite morphemes exist -to determine a defined in a different way entities, which have two phonetic alternations, respectively. For feminine and for neuter gender only one definite morpheme exists, respectively. For plural, two definite morphemes are used depending on the ending vocal of the main plural form. The features of gender and number of definite article are different from the gender and number features of possessive pronouns, themselves. The former are inflectional whereas the later are not inflectional, even both they can express agreement and both they are significant for part-of-speech parsing. The relation between them is important starting point for our interpretation. The inflectional forms of possessive pronouns are presented at Fig 1 .  Our approach to inflectional morphology of possessive pronouns is based on the encoding already published in [START_REF] Stoykova | Bulgarian possessive and reflexive-possessive pronouns in DATR[END_REF]. It presents an inheritance semantic network consisting of different inflectional hierarchical type nodes of rule-based formal grammar and a lexical database (the pronouns). The particular queries to be evaluated are related inflected forms. It also offers an account of sound alternations.

The interpretation is based on adjectives' encoding [START_REF] Stoykova | The definite article of bulgarian adjectives and numerals in DATR[END_REF] and takes as a starting point linguistic motivation, in particular, the priority of one or another grammar feature. Thus, it uses definite article's grammar features of gender and number to triger the value of related inflected forms. It also is similar to the approach presented in [START_REF] Stoykova | The Inflectional Morphology of Bulgarian Possessive and Reflexivepossessive Pronouns in Universal Networking Language[END_REF][START_REF] Stoykova | Representation of Possessive Pronouns in Universal Networking Language[END_REF] to possessive pronouns formal representation using frameworks of Universal Networking Language [START_REF] Uchida | Universal Networking Language[END_REF] which present another type of semantic network.

Further, we are going to extend that approach by relating inflectional rules and inflectional type hierarchy to grammar features of possessive pronouns themselves.

The inflectional type hierarchy

The DATR analysis of possessive pronouns starts with node DET which defines all inflecting morphemes for definite article as follows: == _te. The basic node AdjG defines grammar rules for generating all inflected forms for the feature of gender, number, and definiteness and is exactly the same as for the adjectives.

AdjG:

<sing undef masc> == "<root>" <sing undef femn> == "<root gend>" _a <sing undef neut> == "<root gend>" _o <sing def_2 masc> == "<plur undef masc>" DET <sing def_1 masc> == "<plur undef masc>" DET <sing def_1> == "<sing undef>" DET <plur undef> == "<root gend>" _i <plur def_1>

== "<plur undef>" DET.

Node Adj inherits all grammar rules from node AdjG but employs grammar rules for generating inflected forms for the feature of comparison of degree. It also is the same as for the adjectives. We employ it to make encoding as short as possible regardless the fact that possessive pronouns generates compared inflected forms very rare. The pronouns 'negov' (his) and 'nein' (her) uses this type of inflectional rules.

Adj: <> == AdjG <compar> == po_ "<>" <superl> == naj_ "<>". Node Adj_2 is defined as it is employed for the adjectives and it presents inflectional rules of possessive pronoun 'tehen' (their), which realises two types of phonetic alternation during the process of inflection. At this node an additional inflectional base form <root plur> is introduced to account for the complexity. Adj_2:

<> == Adj <plur undef> == "<root plur>" _i. Node Adj_4 is defined also for adjectives. It encodes inflectional rules of possessive pronouns 'nash' (our) and 'vash' (your) and inherits all grammar rules of node Adj. The new employed grammar rule changes inflectional morpheme for neuter gender into -e.

Adj_4:

<> == Adj <sing undef neut> == "<root gend>" _e.

The new node Adj_5 is employed and it consists of inflectional rules for possessive pronouns 'moj' (my) and 'tvoj' (your). It defines feminine and neuter gender forms by using palatal morphemes, and generates defined inflected forms of masculine gender by using the <root gend> base.

Adj_5:

<> == Adj <sing undef femn> == "<root gend>" _ja <sing undef neut> == "<root gend>" _e <sing def_1 masc> == "<root gend>" DET <sing def_2 masc> == "<root gend>" DET.

The grammar features of gender and number of possessive pronouns themselves can be presented as relational information introducing node PossPron which consists of rules that relate lexical database information presented at the nodes of possessive pronouns themselves ('Moj', 'Tvoj', 'Negov', 'Nein', 'Nash', 'Vash' and 'Tehen') to inflectional grammar rules nodes AdjG, Adj_2, Adj_4 and Adj_5. The node is as follows: The new introduced paths <first>, <second> and <third> refer to the feature of person, paths <masc_p> and <femn_p> refer to the feature of gender, and paths <sing_p> and <plur_p> refer to the feature of number of possessive pronouns themselves. The generated inflected forms of possessive pronoun 'Negov' are given at the Appendix.

Bulgarian reflexive-possessive pronoun in DATR

The semantics of reflexive-possessive pronoun in Bulgarian language combines semantics of possession relationship and that of reflexivity. It expresses possession relationship between the possessor (defined by the subject in the sentence, and agreed with it in gender and number) and the thing being possessed (to which the pronoun is referred to, and agrees in gender and number). full and short form, and both they can be used with respect to agreement. However, only its full form 'svoj' (-self) has inflectional grammar features of person, gender, number, and definiteness, which are similar to that of adjectives and of possessive pronouns. Its inflected forms are given at Fig. 2.

The inflectional type hierarchy

The DATR formal account of reflexive-possessive pronoun inflectional morphology [START_REF] Stoykova | Bulgarian possessive and reflexive-possessive pronouns in DATR[END_REF] uses inflectional rules already defined at node Adj_5 and is part of it. It also uses the same principle as for possessive pronouns. The entire interpretation is extended whith rules which relate reflexive-possessive pronoun to its grammar features of person and gender (the person and gender of the subject in the sentense which are reflexive) and the gender and number grammar features of its definite article (the gender and number of the complement in the sentence). The node RefPoss is introduced which is as follows: The encoding differ with that for possessive pronouns 'moj' and 'tvoj' because the features of person and gender of pronoun 'svoj' are reflexive and should be defined by the person and gender of the subject in the sentense. Thus, entire interpretation gives a relation also to syntactic interpretation for part-of-speech analysis. The related generated inflected forms of reflexive-possessive pronoun 'svoj' are given at the Appendix.

Conclusion

The entire formal semantic network interpretation of inflectional morphology of possessive and reflexive-possessive pronouns in Bulgarian language uses non-monotonic approach to account for both regular and irregular word forms by introducing different nodes with related inflectional rules. It also uses the features of gender and number as a trigger to change the values of related inflected forms and a relational database approach to relate the grammar features of definite article to grammar features of pronouns themselves allowing further syntactic interpretation. Thus, it is good as a future works to extend that approach by enlarging the application offering syntactic rules.

  PossPron: <first sing_p sing undef masc> == <sing undef masc> Moj <first sing_p sing undef femn> == <sing undef femn> Moj <first sing_p sing undef neut> == <sing undef neut> Moj <first sing_p sing def_2 masc> == <sing def_2 masc> Moj <first sing_p sing def_1 masc> == <sing def_1 masc> Moj <first sing_p sing def_1> == <sing def_1> Moj <first sing_p plur undef> == <plur undef> Moj <first sing_p plur def_1> == <plur def_1> Moj <secound sing_p sing undef masc> == <sing undef masc> Tvoj <secound sing_p sing undef femn> == <sing undef femn> Tvoj <secound sing_p sing undef neut> == <sing undef neut> Tvoj <secound sing_p sing def_2 masc> == <sing def_2 masc> Tvoj <secound sing_p sing def_1 masc> == <sing def_1 masc> Tvoj <secound sing_p sing def_1> == <sing def_1> Tvoj <secound sing_p plur undef> == <plur undef> Tvoj <secound sing_p plur def_1> == <plur def_1> Tvoj <third sing_p masc_p sing undef masc> == <sing undef masc> Negov <third sing_p masc_p sing undef femn> == <sing undef femn> Negov <third sing_p masc_p sing undef neut> == <sing undef neut> Negov <third sing_p masc_p sing def_2 masc> == <sing def_2 masc> Negov <third sing_p masc_p sing def_1 masc> == <sing def_1 masc> Negov <third sing_p masc_p sing def_1> == <sing def_1> Negov <third sing_p masc_p plur undef> == <plur undef> Negov <third sing_p masc_p plur def_1> == <plur def_1> Negov <third sing_p femn_p sing undef masc> == <sing undef masc> Nein <third sing_p femn_p sing undef femn> == <sing undef femn> Nein <third sing_p femn_p sing undef neut> == <sing undef neut> Nein <third sing_p femn_p sing def_2 masc> == <sing def_2 masc> Nein <third sing_p femn_p sing def_1 masc> == <sing def_1 masc> Nein <third sing_p femn_p sing def_1> == <sing def_1> Nein <third sing_p femn_p plur undef> == <plur undef> Nein <third sing_p femn_p plur def_1> == <plur def_1> Nein <first plur_p sing undef masc> == <sing undef masc> Nash <first plur_p sing undef femn> == <sing undef femn> Nash <first plur_p sing undef neut> == <sing undef neut> Nash <first plur_p sing def_2 masc> == <sing def_2 masc> Nash <first plur_p sing def_1 masc> == <sing def_1 masc> Nash <first plur_p sing def_1> == <sing def_1> Nash <first plur_p plur undef> == <plur undef> Nash <first plur_p plur def_1> == <plur def_1> Nash <second plur_p sing undef masc> == <sing undef masc> Vash <second plur_p sing undef femn> == <sing undef femn> Vash <second plur_p sing undef neut> == <sing undef neut> Vash <second plur_p sing def_2 masc> == <sing def_2 masc> Vash <second plur_p sing def_1 masc> == <sing def_1 masc> Vash <second plur_p sing def_1> == <sing def_1> Vash <second plur_p plur undef> == <plur undef> Vash <second plur_p plur def_1> == <plur def_1> Vash <third plur_p sing undef masc> == <sing undef masc> Tehen <third plur_p sing undef femn> == <sing undef femn> Tehen <third plur_p sing undef neut> == <sing undef neut> Tehen <third plur_p sing def_2 masc> == <sing def_2 masc> Tehen <third plur_p sing def_1 masc> == <sing def_1 masc> Tehen <third plur_p sing def_1> == <sing def_1> Tehen <third plur_p plur undef> == <plur undef> Tehen <third plur_p plur def_1> == <plur def_1> Tehen.

  st pers. 2 nd pers. 3 rd person 1 st pers. 2 nd pers. 3 rd pers.

	possessed			POSSESSOR
	number	gender	singular 1 gender person male female neuter	plural	definiteness
	sg. male sg. female moja moj sg. neuter moe pl. moi	tvoj tvoja tvoe tvoi	negov negova nejna negova nasha nein negov nash negovo nejno negovo nashe negovi nejni negovi nashi	vash vasha vashe vashi	tehen tjahna tjahno tehni	undefined
	sg. male moja	tvoja negovija nejnija negovija nashija vashija tehnija
	sg. male mojat	tvojat negovijat nejnijat negovijat nashijat vashijat tehnijat
	sg. female mojata tvojata negovata nejnata negovata nashata vashata tjahnata
	sg. neuter moeto tvoeto negovoto nejnoto negovoto nasheto vasheto tjahnoto
	pl.		moite	tvoite negovite nejnite negovite nashite vashite tehnite

defined Fig.

1

. The Bulgarian possessive pronouns inflection table representation.

  1

  The reflexive-possessive pronoun is one and it has The Bulgarian reflexive-possessive pronoun's inflection table representation.

	number		singular		plural
	gender	male	female neuter
	undefined	svoj	svoja	svoe	svoi
	defined	svoja(t) svojata svoeto svoite
		svojat
	Fig. 2.

Here and elsewhere in the description we use Latin alphabet to present morphemes instead of Cyrillic. Because of the mismatching between both some typically Bulgarian phonological alternations are marked by two letters, whereas in Cyrillic alphabet they are marked by one.


Kolyo Onkov

Georgios Tegos
email: gtegos@gen.teithe.gr

Forecasting algorithm adaptive automatically to time series length

Keywords: varying length time series, automatic model fitting, criterion, adaptive algorithm 1

The developed forecasting algorithm creates trend models based on varying length time series by eliminating its oldest member. The constructed criterion evaluates the attained models through estimating the ratio between the average of the stochastic errors for the forecasted period and the average of real values. The best model and forecasting are automatically achieved in contrast to statistical software systems SPSS, STATISTICA, etc. where this process is accomplished progressively by the user. Therefore, this forecasting algorithm is adaptive to the length of time series. Component oriented approach has been used for software implementation. Simulation experiments have been carried out to test the forecasting algorithm using the multidimensional time series database on fishery in Greece. This algorithm is more efficient in case forecasting is applied on large number of time series because it saves time and efforts.

INTRODUCTION

Time series forecasting refers to the process of identifying past relationships and trends in historical data for predicting future values [START_REF] Singh | Pattern Modelling in Time-Series Forecasting[END_REF]. Forecasting is important in time series analysis because it plays a central role in management since it precedes decision making [START_REF] Makridakis | Forecasting: Methods and Applications[END_REF]. There are many time series analysis techniques related to forecasting [START_REF] Leonard | Large-Scale Automatic Forecasting: Millions of Forecasts[END_REF]. Trend modelling can be used for forecasting if it is assumed that the studied event will follow the same rules during the historical and the forecasted period. The attained forecasts are a simple consequence of the trend extrapolation. The advantages of forecasts based on trend modelling compared to other forecasting methods are: a) the preliminary evaluation of the forecast stochastic error by trend models and b) the determination of the confidence intervals. As a result, the critical limits, on which the real values of the studied event fluctuate during the forecasted period, can be evaluated. While modeling the trend of time series for forecasting purposes, the issue concerning the length of the period, within which the trends will be studied, frequently arises. Here there are two tendencies opposing each other. The stochastic error of the forecast decreases in longer historical periods. However, by covering a longer historical period, there is a risk for the model to include the influence of factors that used to function in the past, but later disappeared.

Forecasting time series based on trend modeling is often implemented through using statistical packages such as SPSS, Statistica etc. However, for reapplying trend modeling to different length time series and achieving new forecasted values the same procedure is followed manually. In general, in case of forecasting based on large numbers of time series, automatic forecasting is more efficient [START_REF] Makridakis | Forecasting: Methods and Applications[END_REF], [START_REF] Marahaj | Forecasting Time Series from Clusters[END_REF]. Two automatic forecasting time series algorithms based on state space and ARIMA models have been implemented as R packages for statistical computing [START_REF] Hyndman | Automatic Time Series Forecasting: The forecast Package for R[END_REF]. Fuzzy time series models and automatic forecasting techniques are developed to improve forecasting accuracy [START_REF] Wong | Adaptive Time-Variant Models for Fuzzy-Time-Series Forecasting[END_REF], [START_REF] Chen | Multivariate fuzzy forecasting based on fuzzy time series and automatic clustering techniques[END_REF].

This work aims to present an adaptive to time series length algorithm in order to achieve the best models and forecasted values automatically. The idea is to reapply forecasting by decreasing time series length and testing different trend models. As a result, different forecasts are achieved giving the opportunity to choose the most proper one based on objective criterions.

MATERIALS AND METHODS

The algorithm is based on the gradual elimination of data from the start of the historical time series by applying the same trend model to each resultant time series. The confidence intervals are evaluated according to the different types of trend models on the grounds of which the forecast is achieved. Let's denote: y i (i=1, 2,…, n) -time series with length n, j n y + ˆ -forecasted values, j = 1, 2, ……, L, L -length of the forecasted period The confidence interval for the linear and polynomial second degree models is evaluated as follows:

S y = ∑ = - n i i i y y n 1 2 ) ( 1 ˆ -standard error, I i y ˆ(i=1,2,…, n) -smoothed
IV j n y + ~
-the attained value of y for the j th year of the forecasted period; t [1-α] -Student's test at a level of significance α and n-2 degrees of freedom.

From equations II and III it is noticed that the value of the stochastic error depends directly on the length of time series n to which the trend model is applied and on the length of the forecasted period j. When n increases the value of stochastic error decreases while when j increases the stochastic error also does. Stated in other words, the greater the historical period is, the smaller the stochastic error becomes and on the other hand, the longer the forecasted period is, the greater the stochastic error becomes.

A criterion is created in order to evaluate the forecasts attained from time series with different length. The criterion is related to the stochastic error (equation II and III). During the elimination of data of time series by decreasing each time the value of n by 1, the ratio between the average of the stochastic errors for the forecasted period and the average of real values are computed according to the next equation:

              = ∑ ∑ = = + n i i L j y y n e L d C j d 1 1 1 1 1 ) ( , d=n, n-1,…,m
V m is the minimal value of time series length, (m < n). The best trend model and forecasting are achieved for the minimal value of C 1 (d) criterion when varying the length time series.

The created forecasting algorithm, based on trend models applied to varying length time series, is presented here for one time series by the following steps: The algorithm is adaptive to time series length and as a result it gives the ability to achieve automatically forecasts based on trend modelling. The perspective of this algorithm is to apply different weights on the time series members used for trend modeling. The study of this approach will reveal abilities for attaining more precise forecasting values.

RESULTS AND DISCUSSION

Software and simulation experiment

Component oriented approach and VB programming language have been used for software implementation according to the algorithm. The procedures are placed in two modules:

• The first of them is consisted of procedures for estimating the descriptive statistics (mean value, standard deviation, standard error etc) as well as for evaluating the Ftest and Student test; • The second one contains the procedures for trend modelling, managing the length of time series, stochastic error, confidence intervals, application of the criterion and presentation of the results.

Generally, the procedures in the second module, "call" the procedures from the first one. VB programming code is also created for managing Excel-sheets which are proper for storing tables containing automatically accessed theoretical values of F-test and Student test by the trend modelling and forecasting and forecasting results as well.

A simulation experiment has been carried out to test the forecasting software using multidimensional time series database on sea fishery in Greece. Data in database concerns quantities and values of fish catch by areas, fish species, fishing tools and category as well as months, kinds of fishery and employment for the period 1990-2011 [START_REF] Tegos | PC-Information System concerning sea fishery time series in Greece[END_REF], [START_REF] Onkov | Management solutions for fish resources in Greece based on analysis of multi-dimensional database[END_REF]. Each time series extracted from FTS database consists of 22 members. For the purpose of forecasting it is divided into two parts: the first twenty members are used as historical time series and the last two (2010, 2011) are compared with the obtained forecasted values for the same years (proportion 20:2).

Forecasting results

The forecasting results presented here are oriented to time series on catch quantity of 71 sea fish species statistically registered in Greece. Table1 presents the part of the obtained results concerning only 10 fish species. Generally, for forecasting purposes the optimal time series length on the particular fish species is different. Besides the type of trend model used for the forecasts is also different. From 71 fish species there are 6 without adequate trend because of big random factor and as a result no forecasts are proposed by the algorithm. Almost all these cases concern fish species with small amount of catches -mean value for the studied period of time do not exceed 200 metric tons (Sprat, Skipjack etc). Table 2 presents more detailed results of the forecasting algorithm applied on quantity of catch for fish species "Bogue", based on linear trend modeling. For each varying length historical time series (n = 15, 14, 13, 12, 11) the forecasted values for the years 2010 and 2011 as well as the confidence interval (equation IV) and the C1 criterion (equation V) are also presented. The relative error shows the percentage deviation between real and forecasted values. The best forecast is obtained for time series length 12 when C1 criterion has the minimum value (Figure1). The obtained relative error values are considered satisfactory -10.87% and 2.47% for the years 2010 and 2011 respectively. Table 3 presents the results of the forecasting subroutine application, based on polynomial second degree trend model, on the "total fish quantity" time series. It is evident that the best forecasting result is achieved for time series length 9, because the value of criterion C 1 is the smallest -2.96 and 3.75, for years 2010 and 2011, respectively. Figure2. Forecasting results for varying time series length n

The obtained results show that the algorithm is adaptive to the dynamic of the fish catch quantity process and automatically achieves forecasting results.

CONCLUSION

This work presents an algorithm applied on time series, capable to achieving optimum trend models and forecasting with minimum human intervention. The most significant characteristic of the developed algorithm is the varying length time series used for automatic model fitting. A criterion related to the stochastic error of the forecasted values is constructed for the estimation of the best forecast. When analyzing the set of time series the algorithm behaviour is adaptive to time series length and trend model. As a result, the forecasts are achieved automatically. A component oriented approach has been used for the implementation of the software. The forecasting algorithm has been tested on time series sets concerning sea fishery in Greece and acceptable forecasting results are attained. The same algorithm can be applied in appropriate time series of other branches giving equivalent results as well. More efficient results are achieved in case forecasting is applied on large numbers of time series because it saves time and efforts.

  is the standard error by equation I.

  If n < m then go to step 10 else go to step 9; 9. Elimination of the most distant year from time series and go to step 1; 10. Application of the criterion for the best forecasting -MIN (C 1 (n), C 1 (n-1), …, C 1 (m)) for both trend models; 11. Evaluation and printing the best forecasting results based on both trend models.

	) 1 1 ) -3. Procedure for trend model adequacy by F-test. If the model is adequate then go to ( 2 ( 3 1 1 2 2 -+ + + ⋅ n n j n j n y e + = 1. Implement steps 2-6 for both trend models; S y 2. Procedure for trend modeling; ˆ1 -n n step 4 else go to step 6;
	n y ˆ, i=1, 2, …n and Standard error S y ; t n t t t 1 1 2 2 4 2 1 2 1 4. Computation of smoothed values i t t 4 2 1 2 ) . ). 2 ( . 1 1 1 -5. Procedure for estimation and storing of: S y = n • Forecasted values 1 ˆ+ n y , 2 ˆ+ n y , …, L n ˆ; y + ( • Stochastic errors 1 ˆ+ n y e , 2 ˆ+ n y e …, L n y e + ˆ;	4
	• Confidence intervals by equation IV applying Student's test;
	• C 1 (n) value.
	6. Next operation;
	7. n=n -1;
	8.

  Forecast of length 11 is optimistic because trend model uses much bigger values of catch quantity at the beginning of the period 1999-2009 than at the end. For the opposite reason forecast of length 7 spanning the period 2003-2009 is pessimistic.• The most important result consists of the fact that the algorithm automatically finds the optimal time series length 9 and the polynomial second degree model for achieving the best forecast.

	Time series length 70000 85000 100000 115000 • 55000	Table3. Forecasting results for total quantity of fish catches in Greece Forecasted values Confidence interval Real values C 1 Relative error % Year Total quantity of Year catch /m 3 tons/ Real values n = 11 n = 9 n = 7
	1999	2001	2003	2005	2007	2009	2011


Xanthoula Eirini Pantazi
email: renepantazi@gmail.com

Dimitrios Moshou
email: dmoshou@agro.auth.gr

Abdul Mounem Mouazen

Boyan Kuang

Thomas Alexandridis

Application of Supervised Self Organising Models for Wheat Yield Prediction

Keywords: neural networks, data mining, precision agriculture, machine learning

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

A large number of approaches, models, algorithms, and statistical tools have been proposed and used for assessing the yield prediction in agriculture. Many authors used simple linear correlations of yield with soil properties but the results varying from field to field and year to year [START_REF] Drummond | Analysis and correlation methods for spatial data[END_REF][START_REF] Khakural | Variability of corn/soybean yield and soil/landscape properties across a southwestern Minnesota landscape[END_REF]. Many other studies, contain complex linear methods like multiple linear regression, were accomplished with similar results [START_REF] Drummond | Analysis and correlation methods for spatial data[END_REF][START_REF] Khakural | Variability of corn/soybean yield and soil/landscape properties across a southwestern Minnesota landscape[END_REF][START_REF] Kravchenko | Correlation of corn and soybean grain yield with topography and soil properties[END_REF]. Some authors proposed non-linear statistical methods to investigate the yield response [START_REF] Adams | Machine learning methods in site-specific management research: An Australian case study[END_REF][START_REF] Wendroth | Spatial crop yield prediction from soil and land surface state variables using an autoregressive state-space approach[END_REF]). Expert systems and artificial intelligent algorithms are a relatively new subset of nonlinear techniques. They have been proposed in agriculture for decision making and decision support tasks. More specifically, expert systems [START_REF] Plant | Knowledge based systems in agriculture[END_REF][START_REF] Rao | Expert systems in agriculture[END_REF] have been developed and applied in different fields in agriculture to give advices and make management decisions. In this context many studies have been reported using artificial intelligence techniques and a few of them focused on the spatial analysis of produced data in precision agriculture. The most of them use artificial neural networks (ANNs) and machine learning algorithms for setting target yields which is one of the problems in PA [START_REF] Canteri | Application of data mining in automatic description of yield behavior in agricultural areas[END_REF][START_REF] Liu | A neural network for setting target corn yields[END_REF]Miao et al., 2006). Schultz, et al. (2000) summarized the advantages of applying neural networks in agro ecological modeling, including the ability of ANN to handle both quantitative and qualitative data, merge information and combine both linear and non-linear responses.

Neural networks have been proposed for identifying important factors influencing corn yield and grain quality variability (Miao et al., 2006), for data analysis [START_REF] Irmak | Artificial neural network model as a data analysis tool in precision farming[END_REF], for prediction crop yield based on soil properties [START_REF] Drummond | Statistical and neural methods for site-specific yield prediction[END_REF], for setting target corn yields [START_REF] Liu | A neural network for setting target corn yields[END_REF]. [START_REF] Shearer | Yield prediction using a neural network classifier trained using soil landscape features and soil fertility data[END_REF] studied a large number of variables, including fertility, satellite imagery, and soil conductivity, for a relatively small number of observations in one site-year of data. Self-Organizing Maps (SOMs) are one of the most well-known among the several Artificial Neural Networks architectures proposed in literature [START_REF] Kohonen | Self-Organization and Associative Memory[END_REF].

Their applications have increased during the last decade and they have been applied in several different fields and nowadays they are considered as one of the foremost machine learning tools and an important tool for multivariate statistics [START_REF] Marini | Artificial neural networks in food analysis: trends and perspectives Analytica[END_REF] Self-Organizing Maps (SOMs) are self-organizing systems able to solve problems in an unsupervised way, without needing target data. In order to cover certain needs, unsupervised models have been extended in order to be able to work in a supervised framework. To this end, methods like counterpropagation Artificial Neural Networks (CP-ANNs), which are very similar to SOMs, since an output layer is added to the SOM layer [START_REF] Zupan | Neural networks with counter-propagation learning strategy used for modelling[END_REF], have been introduced. When dealing with classification issues, CP-ANNs are generally efficient methods for achieving class separation in non-linear boundaries. Recent modifications to CP-ANNs have led to the introduction of new supervised neural network architectures and relevant learning algorithms such as Supervised Kohonen Networks (SKNs) and XY-fused Networks (XY-Fs) [START_REF] Melssen | Supervised Kohonen networks for classification problems[END_REF]. The aim of the work reported is to present a methodology that can determine wheat yield behavior in precision farming, based on Machine Learning techniques and particularly based on aspects related to cluster visualization. In the current paper several Self Organizing Map models using supervised learning approach and algorithm are used to classify precision agriculture data in order to predict the yield productivity. To achieve this, soil physical and chemical parameters have been fused together with biomass indicators.

Materials and Methods

Crop Parameters affecting yield

For the calculation of crop cover the Normalized Different Vegetation Index (NDVI) was used. The NDVI is (NIR-RED)/ (NIR+RED) where NIR is the Near Infrared Radiation (0,725 to 1μm) and RED is the Red Radiation (0, 58 to 0, 68 μm). The NDVI was calculated based on satellite images that were collected two times on the 2nd May and 3rd June in the spring of 2013 (Fig. 1). These images were provided by DMCii (http://dmcii.tumblr.com/) for the TF1 in the Duck End farm in the UK. Below is the processing workflow chain for crop NDVI, based on satellite imagery:

1. We received L1R (geo-rectified from DMCii) or L1T (ortho-rectified imagery).

With the L1R we had to complete the ortho-rectification using software called 'keystone workstation', otherwise the L1T ortho product should be purchased directly from DMCii (the preferred method), 2. We performed in-band reflectance calibration using ArcGIS, 3. A map of NDVI was created with ArcGIS, 4. The NDVI was calculated for 5mX5m grid resulting 16500 values.

The yield was calculated for 5mX5m grid by a combine harvester that able to measure automatically the yield during harvest resulting in 16500 values. On-line measurements of soil were carried out in the selected field of Horn's End.

The measurement was carried out by Cranfield University (CU). The raw soil spectra was recorded and stored for time on analysis. Soil samples collected from the field were sent to laboratory analysis with standard methods.

The following soil parameters were estimated from the spectra in a 5mX 5m grid: soil Organic Carbon (OC), pH, Mg, Total N, Ca, Cation Exchange Capacity (CEC) and Moisture Content (MC). This resulted in 16500 vectors of soil parameters for the whole field matching the number of values obtained from the NDVI calculation.

Counterpropagation Artificial Neural Networks

Counterpropagation Artificial Neural Networks (CP-ANNs) are modeling methods which combine features from both supervised and unsupervised learning [START_REF] Zupan | Neural networks with counter-propagation learning strategy used for modelling[END_REF] CP-ANNs consist of two layers, a Kohonen layer and an output layer, whose neurons have as many weights as the number of classes to be modelled .The class vector is used to define a matrix C, with I rows and G columns, where I is the number of samples and G the total number of classes; each entry cig of C represents the membership of the i-th sample to the g-th class expressed with a binary code (0 or 1). When the sequential training is adopted, the weights of the r th neuron in the output layer (y r ) are updated in a supervised manner on the basis of the winning neuron selected in the Kohonen layer. Considering the class of each sample i, the update is calculated as follows:

  max 1 1 old n r i r d y c y d             (1)
where d ri is the topological distance between the considered neuron r and the winning neuron selected in the Kohonen layer; c i is the i th row of the unfolded class matrix C, that is, a G-dimensional binary vector representing the class membership of the i th sample. At the end of the network training, each neuron of the Kohonen layer can be assigned to a class on the basis of the output weights and all the samples placed in that neuron are automatically assigned to the corresponding class.

XY-fused Networks

XY-fused Networks (XY-Fs) [START_REF] Melssen | Supervised Kohonen networks for classification problems[END_REF] are supervised neural networks for building classification models derived from Self-Organizing Maps (SOMs). In XY-fused Networks, the winning neuron is selected by calculating Euclidean distances between a) sample (xi) and weights of the Kohonen layer, b) class membership vector (ci) and weights of the output layer. These two Euclidean distances are then combined together to form a fused similarity, that is used to find the winning neuron.

The influence of distances calculated on the Kohonen layer decreases linearly during the training epochs, while the influence of distances calculated on the output layer increases.

Supervised Kohonen Networks (SKNs)

As in the case for CP-ANNs and XY-Fs, Supervised Kohonen Networks (SKNs) [START_REF] Melssen | Supervised Kohonen networks for classification problems[END_REF] are supervised neural networks derived from Self-Organizing Maps (SOMs) and used to calculate classification models. In Supervised Kohonen Networks, Kohonen and output layers are glued together to give a combined layer that is updated according to the training scheme of Self-Organizing Maps. Each sample (xi) and its corresponding class vector (ci) are combined together and act as input for the network. In order to achieve classification models with good predictive performances, xi and ci must be scaled properly. Therefore, a scaling coefficient for ci is introduced for tuning the influence of class vector in the model calculation.

3

Results and Discussion.

The values of the eight soil parameters were concatenated with the NDVI values so as to form 16500 feature vectors which correspond to fusion of both soil and crop parameters. Aiming to avoid bias during clustering the fusion vectors were preprocessed so that they had zero mean and start a deviation equal to unity. In order to predict the yield fusion vectors were used as inputs while the yield values were divided in three classes with equal number of samples containing 5500 each in ascending order, thus corresponding to low medium and high yield.

The Supervised Map models Xyf, Skn and Cpann were trained with the fusion vectors as input and the yield classes as output. In order to be able to test the generalization capability of the neural networks cross validation was applied by leaving one out of ten samples randomly so that after training on nine samples the prediction was tested on the tenth. The results of the cross validation are shown in Tables 1, 2, 3 for Xyf, SKn and Cpann. The best overall result is obtained from the Skn network. The Skn gives better results than the other two methods because in the case of Skn the clustering of input layers and output yield is performed using one combined vector and this reduces the possibility of deviating values of yield from affecting the result of classification. In the other two architectures there are values of yield that can affect the clustering in a negative way since it is possible that other non-measured parameters are capable of affecting the yield. In all cases the best prediction is obtained for the low category of yield which is advantageous since the low yield spots in the field require additional fertilization. The SOM clusters of the components of the training vectors are shown in Figures 2,3,4. The first subplot corresponds to NDVI component while the second to ninth correspond to Ca, CEC, MC, Mg, OC, P, pH, TN. The last two correspond to target yield and normalized target yield. From this can be seen that the NDVI is the first component and shows partial correlation to the yield map which is shown in the tenth subplot. The other components correspond to soil parameters and provide complementary information to the yield. The Component Maps are useful in interpreting the correlations between the soil factors and NDVI as related to the yield. In the case where tendency of the soil factors is similar to the yield this factor is important for higher yield while in the opposite case, a high value in a soil factor could limit the yield. Generally the NDVI has a positive correlation to the yield. The target yield is shown in Fig. 5. in three classes labeled as red for high yield, blue for low yield and yellow for medium yield. By looking at Fig. 6 one can see that there is a high correlation between the target yield and the predicted yield. The NDVI Shown in Fig. 7 demonstrates partial correlation to the yield in the left part of the field, thus explaining the cluster distribution of the Neural Network models. This discrepancy in the right part of the field can be explained by the distribution of the other nutrients which counterbalance the effect on yield. In Figure 8, the spatial variation of total N is shown divided in three classes corresponding to different levels of total N.

Conclusions.

In this paper, three Self Organizing models were developed that consisted of input nodes representing the main factors in wheat crop production such as biomass indicators, organic carbon (OC), pH, Mg, total N, Ca, Cation Exchange Capacity (CEC), moisture content (MC) and the output weights represented the class labels corresponding to the predicted wheat yield. The results indicate that yield prediction is possible with a very high accuracy reaching 93% and can be used in order to improve fertilizer administration by using the yield prediction models that were presented. Visualization of component maps could reveal extra information that is useful in order to interpret the relations between the soil and crop parameters and the yield.

Fig. 1 .

 1 Fig. 1. False colour infrared (left) and NDVI (right) of 3rd Jun 2013 obtained from DMCii UK-DMC2 satellite imagery for TF1 (e.g. Hawnes End field) in the Duck End farm in the UK

Fig. 2 .Fig. 3 .Fig. 4 .

 234 Fig. 2. Skn Component Maps for NDVI and soil parameters

Fig. 5 .Fig. 6 .Fig. 7 .Fig. 8 .

 5678 Fig. 5. Target Yield in three classes produced by ArcGIS Fig. 6. Prediction in three classes produced by ArcGIS

  Precision farming requires development of on-line sensors for real-time measurement of soil properties, because these sensors can lead to reducing labor and time cost of soil sampling and analysis. With the emergence of commercial NIR spectrophotometers and multivariate calibration software packages, the vis-NIR spectroscopy has been adopted much widely for soil analysis. Numerous researchers have extended the vis-NIR spectroscopy applications from the measurement of key soil properties (Moisture Content, pH, Soil Organic Matter, Total N, and soil Organic Carbon) with high accuracy to almost all other micro and macro elements with less accuracy. The analysis of soil with this technique was also extended to soil biological, physical, and engineering properties. Multivariate calibration techniques allowed for simultaneous measurements of several soil properties under consideration.[START_REF] Shibusawa | Soil mapping using the real-time soil spectrometer[END_REF] developed an on-line vis-NIR (400-1700 nm) sensor to predict Moisture Content, pH, Soil Organic Matter, and NO 3 -N. A simpler design to the one of[START_REF] Shibusawa | Soil mapping using the real-time soil spectrometer[END_REF] without sapphire window optical configuration was developed by[START_REF] Mouazen | Towards development of on-line soil moisture content sensor using a fibre-type NIR spectrophotometer[END_REF]. The system was successfully calibrated for Moisture Content, Soil Organic Matter, Total N, pH, and available P in different soils in Belgium and northern France[START_REF] Mouazen | Towards development of on-line soil moisture content sensor using a fibre-type NIR spectrophotometer[END_REF][START_REF] Mouazen | On-line measurement of some selected soil properties using a VIS-NIR sensor[END_REF][START_REF] Mouazen | The future of for on-line measurement of soil properties with sensor fusion[END_REF].

	2.2	Soil Parameters affecting yield

Table 1 .

 1 Results of cross validation for Xyf Network

Table 2 .

 2 Results of cross validation for Skn Network

Table 3 .

 3 Results of cross validation for Cpann Network

	Real Network		Network Prediction (%)
	Estimation	low	medium	high
	low	95.62	3.89	0.49
	medium	5.45	89.95	4.6
	high	5.35	11.29	83.36
	Real Network		Network Prediction (%)
	Estimation	low	medium	high
	low	93.11	5.69	1.20
	medium	4.09	89.05	6.85
	high	2.33	6.64	91.04
	Real Network		Network Prediction (%)
	Estimation	low	medium	high
	low	92.40	4.36	3.24
	medium	8.07	81.22	10.71
	high	4.18	6.02	89.80

Acknowledgement

The presented research was carried out in the framework of project FARMFUSE of ICT AGRI 2 ERANET.


Ramin Amali
email: ramin2.amali@uwe.ac.uk

Siamak Noroozi
email: snoroozi@bournemouth.ac.uk

Application of Artificial Neural Network to Predict Static Loads on an Aircraft Rib

Keywords: Static load, Finite Element Analysis, Artificial Neural Network, Aircraft Rib, MATLAB 1.0 INTRODUCTION

Aircraft wing structures are subjected to different types of loads such as static and dynamic loads throughout their life span. A methodology was developed to predict the static load applied on a wing rib without load cells using Artificial Neural Network (ANN). In conjunction with the finite element modelling of the rib, a classic two layer feed-forward networks were created and trained on MATLAB using the backpropagation algorithm. The strain values obtained from the static loading experiment was used as the input data for the network training and the applied load was set as the output. The results obtained from the ANN showed that this method can be used to predict the static load applied on the wing rib to an accuracy of 92%.

dominated loads, the majority of which tend to occur at frequencies of less than a few Hertz (Hz) and in the case of a rotary-wing aircraft, the loading spectrum experienced by the airframe structure is significantly more complex. There have been a number of attempts in the last few decades at estimating these loads on the helicopter indirectly from flight state parameters or fixed points on the airframe with varying success [START_REF] Polanco | Estimation of Structural Component Loads in Helicopters: A Review of Current Methodologies[END_REF].

In this work, several sets of data obtained from the static test conducted on a wing rib were analysed. This paper presents the implementation details of how accurate an artificial neural network has been used to predict the static load applied on the wing rib, the data sets are standard strain values recorded from the experimental test.

EXPERIMENT

The main aim of this research section was to measure the strain values which will be used as the inputs for the neural network training on MATLAB. The strain values are measured by subjecting the rib to a range of static loads using a powered hydraulic system.

Experiment Design

The rib is to be tested in two different positions due to the number of strain gauges and the location that they are attached on the rib. The testing equipment was set up as shown in Figure 1; the rib is supported in the frame with the aid of four steel rods bolted from the top surface of the shear tie to the top frame and another four steel rods connecting the bottom surface of the cap to the ground. The fifteen strain gauges attached to the rib are channelled to the smart strain scanner which is in turn connected to a computer to run and record the strain values during the test process.

The cylinder is attached to the top part of the frame with a steel block bolted to the frame as shown in the experiment set-up; the cylinder is then connected to the manual hand pump with two high pressure hydraulic hoses as shown in Figure 1.

To conduct the test the manual hydraulic pump was used to apply a gradual load on the shear tie of the rib as shown above in Figure 1. The aim was to apply the specified load to the rib as quick as possible without physically damaging any part of the rib, while the smart strain scanner records the strain output of the rib due to the applied load over a time period. The software used for recording the strain output is precise enough to measure the applied load and strain to the nearest one percent.

However due to the fact that the test is a non-destructive test, the maximum load that could be potentially applied to the rib without causing any damage has been set to 3000N.

The strain data was gathered using the smart strain software, the overall test process was repeated three times on several locations on the rib where the cylinder comes in contact with the surface of the shear tie to get as many data and accurate results from the experiment.

RESULTS

The results obtained were analysed by plotting a graph to show the relationship between the force applied on the rib over a period of time and the output strain against time. Figure 2 shows the relationship between the applied force and the measured strain from the experiment, it can be observed that the strain increases linearly as the force applied increases. Therefore it can be concluded that the relationship between the output and the input which would be used later in the neural network training are linearly related, this result also ascertain the assumption which was made before conducting the experiment.

3.0 DATA ANALYSIS Before the network can be created it requires some data sets as inputs and targets obtained from the experiment, the data obtained from the experiment are in ordinary form of forces and strains. For the network to be trained it requires data from which it can learn the appropriate weight and biases that will help the network to produce the desired result. The data is obtained from the linear relationship between the force and strain graphs obtained from the experiment.

To calculate the strain values which are used as the set of inputs for the network training, a line of best fit was plotted on each strain line in Figure 2 to obtain the linear relationship between the force and the strain. Figure 3 shows some examples of the line of best fit on the graph which also generates the equation that relates the parameters together.

The linear equation was attained for the fifteen strain lines present in each experiment which means that for each experiment there are fifteen linear equations relating the force and the strain which have two constant values.

In general the linear equation can be written in broad terms of the applied force and the strain as shown in the equation ( 1)

𝐹 = 𝑎 𝑛 𝜀 𝑛 + 𝑏 𝑛 , 𝑛 = 1 -15 (1)
Where F is the force which indicates y in the equations in figure 3, 𝜀 𝑛 is the strain indicating x on the graph, a and b are the intercept and gradient values in mathematical terms. By repeating the same step for the experimental test that was conducted, a number of equations are generated in generic terms as shown in equation ( 2)

𝐹 1 = 𝑎 𝑛 𝜀 𝑛 + 𝑏 𝑛 , 𝐹 2 = 𝑐 𝑛 𝜀 𝑛 + 𝑑 𝑛 (2
) Where𝐹 1 and 𝐹 2 are the forces applied for two different experiments 𝜀 𝑛 is the corresponding strains and n represents strains 1-15 and 𝑎 𝑛 𝑏 𝑛 , 𝑐 𝑛 ,and 𝑑 𝑛 , 𝑒𝑡𝑐 are constants for each corresponding strain line. Equation 2 represents a generic format of how the linear relationship between the force and the strain obtained from each experiment can be written.

By adding up 𝐹 1 and 𝐹 2 in the previous equation and rearranging to make 𝜀 𝑛 the subject, a set of strain values can be calculated from the equation which is used as the inputs for the training data. The generic form of the equation is written below:

2𝜀 𝑛 = � 𝐹 1 -𝑏 𝑛 𝑎 𝑛 � + � 𝐹 2 -𝑑 𝑛 𝑐 𝑛 � (3)
Finally, for the network to be trained accurately it requires a large number of data for the inputs and target. To achieve this, a set of random numbers ranging between zero and three were generated and then used to represent the targets which indicates the force in equation [START_REF] Polanco | Estimation of Structural Component Loads in Helicopters: A Review of Current Methodologies[END_REF]. Using the relationship constant obtained from each strain line graph of the experiment, the corresponding 𝜀 𝑛 values were calculated.

NEURAL NETWORK IMPLEMENTATION

This part of the investigation aims to use all the materials and results obtained from the previous section to create an ANN that will be able to predict the load applied to the rib under static conditions as shown in the experimental investigation. A number of neural networks were created for the different experiments carried out on the rib. The network was trained using the calculated strain values obtained through the strain relationship from the experiments as inputs and the corresponding applied load range as the training targets. The trained network was then used to predict the actual experimental load that was applied on the rib by using the measured strain values from the experiments as inputs, the results from the network were also compared with the exper-imental result to define how well the network was able to predict the applied load.

TRAINING DATA

The training data comprises of two sets which are the input sets and the targets sets, as previously mention the inputs are the calculated strains also referred to as 𝜀 𝑛 and the targets are the random numbers ranging from zero to three.

For the network training the inputs was set to a 100x15 matrix configuration and the corresponding set targets had a 100x2 matrix configuration, the 100x2 matrix represents the random forces ranging from zero to three and the 100x15 represents the corresponding calculated strain.

NEURAL NETWORK CREATION

The simulation and analysis of the network was executed on MATLAB using the neural network toolbox function. The first designed network is a two layer feed forward network with sigmoid functions, ten hidden neurons and two output functions feed forward network is mainly used for fitting multi-dimensional mapping problems when given consistent data. Figure 4 shows an illustration of the network configuration from MATLAB. This type of network is mainly used for input and output curve fitting which is required for this analysis. The network is trained with Levenberg-Marquardt back propagation algorithm.

With the number of hidden neurons chosen, the network must allocate the data sets into three different parts which are training, validation and testing. The training data sets are presented to the network during training and the network is adjusted according to its error. The validation sets are used to measure network generalisation and to end training when generalisation stops improving. While the testing data sets which has no effect on the training is used to provide an independent measure of the network performance during and after training. For this neural network development, the default settings was selected on MATLAB which allocates 70% of the given data for training and 15% each for validation and testing.

NETWORK VALIDATION

The network created above uses a data set that its previously seen, therefore it is important to access the performance and capability of the trained network and how closely it would match other outputs values when a new set of inputs are introduced to the network. To validate the network a new set data calculated from the measured strains was set as the input in a 15x2 matrix format to obtain a predicted load as the new output using the trained network. The output from the network is the predicted force or load which is then compared with the actual load that was applied to the rib during the experiment. Figure 5 shows an illustration of a bar chart comparing the predicted load obtained from the network with the experimental load.

The bar chart shows how well the network has been able to predict the load from the set of new strain data which was set as the new inputs. From the chart it can be observed that the error between the first predicted load and the experimental load is as low as 0.25, which in percentage terms is approximately 8% and the second predicted load is 0.36. The calculated error shows that the initial setting that was designed in the network before training the data and the selection of ten hidden neurons produces an appropriate result from the validation aspect of the network. To obtain a better accurate prediction an analysis was carried out on how the variation of the number of hidden neurons would affect the predicted load which is the output of a network.

HIDDEN NEURON ANALYSIS

For this analysis the inputs data was in a 300x15 matrix form and the target was in a 300x2 matrix form, the default settings was chosen on tool box for the data allocation where 70% of the data set was allocated to training and 15% each was allocated to both validation and testing.

The only value that is being varied in this analysis is the number of hidden neurons assigned to each network; however the number of hidden neurons for the analysis will range from two to fifteen which is a combination of the new set of inputs that would be used to validate each network created.

For all the networks trained for this analysis, a new set of data points was introduced to each network to obtain a new set of output which represents the predicted load, a graph was then plotted to show the relationship between the number of hidden neurons and the predicted load as shown in the graph plotted in figure 6. It is expected that an increase in the number of neurons in the hidden layer should initially improve the performance of the network, however there is a downside to increasing the hidden neurons. This causes the network to over fit the training data which will therefore reduce the effectiveness of the network when a new set of data is introduced to the network.

Figure 6 shows the results obtained from the number of hidden neuron analysis, from the graph it can be observed that the network performance initially improves as the number of hidden neurons increases. The graph also shows that the network performs best in the range of 6 to 10 hidden neurons as the predicted load obtained from those set of networks lie within the experiment threshold.

The results obtained from this analysis also indicates that there is a risk with increasing the number of hidden neurons where the network can begin to over fit the training data as shown in figure 6. From eleven hidden neurons the predicted load starts to increases again, however to counter this effect it is possible to increase the number of data samples that is being introduced to the network for training.

NEURAL NETWORK RESULT DISSCUSSION

In conclusion the results for the entire neural network tests were positive as indicated from the range of several networks that was created. The results show that it is possible to predict or estimate the static load applied on the rib to an accuracy of 92%.

However there is a requirement that must be met when training a network, this requirement is related to the number of hidden neurons used in training the samples and also the amount of data used to train and validate the network. To improve the accuracy of a neural network it is important to use as many data sets as possible, this will give the network more data samples to allocate for training, validation and testing which should lead to a better result.

Following the success of the prediction part of the networks, attempts were made to produce a method by which the networks would predict lower force values, this was achieved by changing the targets output and the targets of the network from the usually matrix format of [500x2] to a [500x1]. This new matrix will allow a new set of inputs in the matrix form of [15x1] to be introduced to the network to validate the performance of the network. Also the new set of inputs values that was used for the training of this network has a range of strain values from correspond load applied between 0 and 1.1KN, it is expected that the network should be able to predict the load as it different from the initial loads that have been predicted. The result obtained from this final validation shows that the network is able to predict any static load applied to the rib to an accuracy of 95% provided a consistent set of strain inputs are introduced to the network.

CONCLUSION

This paper presents the use of an ANN to predict the static load applied on a wing rib. It can be stated that this research has shown that it is possible to design a set of neural networks that will correctly predict or estimate the load applied on the rib provided a consistent set of strain inputs values are introduced to the network.

Although a large portion of the analysis has been carried out on the location of the rib where strain gauge rosettes have been attached, it is also possible to use the same technique to analyse other areas or the overall rib by attaching more strain gauges to the rib to capture accurate data when conducting similar static test.

The percentage error between the neural network prediction and the experiment can be classified as the error obtained from using some noisy data to train the network. This however does not affect the performance of the network or how well the network is able to predict other force values when a corresponding set of strain inputs are introduced to the network.

It should be stated that during the data analysis stage of the research, not all the fifteen strain gauges had a perfect linear relationship between the applied force and the measured strain, which could have also some error in the new data set that was used as inputs. Further improvement in the experimental procedure and more allocation of strain gauges to the rib will allow the measuring equipment to capture strain values more accurately which would lead to a better neural network creation and load prediction in the future.

Figure 1 :

 1 Figure 1: Static Experiment Set-up

Figure 2 :

 2 Figure 2: Strain against Force Graph-Experiment Result

Figure 3 :

 3 Figure 3: Force against Strain-Strain 13 and 15

Figure 4 :

 4 Figure 4: Neural Network Result comparison

Figure 5 :

 5 Figure 5: Hidden Neuron Analysis