Ammar Mohemmed
email: amohemme@aut.ac.nz

Stefan Schliebs
email: sschlieb@aut.ac.nz

Satoshi Matsuda
email: matsuda.satoshi@nihon-u.ac.jp

Nikola Kasabov
email: nkasabov@aut.ac.nz

Method for training a spiking neuron to associate input-output spike trains

Keywords: Spiking Neural Networks, Supervised Learning, Spatio-temporal patterns

We propose a novel supervised learning rule allowing the training of a precise input-output behavior to a spiking neuron. A single neuron can be trained to associate (map) different output spike trains to different multiple input spike trains. Spike trains are transformed into continuous functions through appropriate kernels and then Delta rule is applied. The main advantage of the method is its algorithmic simplicity promoting its straightforward application to building spiking neural networks (SNN) for engineering problems. We experimentally demonstrate on a synthetic benchmark problem the suitability of the method for spatio-temporal classification. The obtained results show promising efficiency and precision of the proposed method.

Introduction

The desire to better understand the remarkable information processing capabilities of the mammalian brain has recently led to the development of more complex and biologically plausible connectionist models, namely spiking neural networks (SNN). See e.g. [START_REF] Gerstner | Spiking Neuron Models: Single Neurons, Populations, Plasticity[END_REF] for a comprehensive text on the material. These models use trains of spikes as internal information representation rather than continuous-value variables. Many studies investigate how to use SNN for practical applications, some of them demonstrating very promising results in solving complex real world problems. However, due to the increased complexity of SNN compared to traditional artificial neural networks, the training of these networks in a supervised fashion has proved to be difficult. See [START_REF] Kasiński | Comparison of supervised learning methods for spike time coding in spiking neural networks[END_REF] for a review on supervised learning methods for SNN.

Only few algorithms have been proposed that are able to impose a precise input-output mapping of spike trains to a SNN. One of the first supervised learning methods for SNN is SpikeProb [START_REF] Bohte | SpikeProp: backpropagation for networks of spiking neurons[END_REF]. It is a gradient descent approach that adjusts the synaptic weights in order to emit a single spike at specified time. The timing of the output spike encodes a specific information, e.g. the class label of the presented input sample. Using SpikeProp, the SNN can not be trained to emit a desired train of spikes that has more than one spike.

An interesting learning rule for spatio-temporal pattern recognition has been suggested in [START_REF] Gutig | The tempotron: a neuron that learns spike timingbased decisions[END_REF]. The so-called Tempotron enables a neuron to learn whether to fire or not to fire in response to a specific input stimulus. Consequently, the method allows the processing of binary classification problems. However, the neuron is not intended to learn a precise target output spike train, but instead whether to spike or not to spike in response to an input stimulus.

A Hebbian based supervised learning algorithm called Remote Supervised Method (ReSuMe) was proposed in [START_REF] Ponulak | ReSuMe -new supervised learning method for spiking neural networks[END_REF] and further studied in [START_REF] Ponulak | Analysis of the resume learning process for spiking neural networks[END_REF][START_REF] Ponulak | Supervised learning in spiking neural networks with ReSuMe: sequence learning, classification, and spike shifting[END_REF]. ReSuMe, similar to spike time dependent plasticity (STDP) [START_REF] Bell | Synaptic plasticity in a cerebellumlike structure depends on temporal order[END_REF][START_REF] Bi | Synaptic modifications in cultured hippocampal neurons: Dependence on spike timing, synaptic strength, and postsynaptic cell type[END_REF], is based on a learning window concept. Using a teaching signal specific desired output is imposed to the output neuron. With this method, a neuron is able to produce a spike train precisely matching a desired spike train.

Recently, a method called Chronotron was proposed in [START_REF] Florian | The chronotron: a neuron that learns to fire temporally-precise spike patterns[END_REF]. It is based on minimizing the error between the desired spike pattern and the actual one. The error is measured using the Victor-Purpura spike distance metric [START_REF] Victor | Metric-space analysis of spike trains: theory, algorithms and application[END_REF]. This metric produces discontinuities in the error landscape that must be overcome through approximation. E-Learning is compared with ReSuMe on some temporal classification tasks and its better performance in terms of the number of spike patterns that can be classified is shown.

In this study, we propose a new learning rule allowing a neuron to learn efficiently associations of input-output spike trains. Although being principally simpler 1 compared to the methods mentioned above, we demonstrate its efficiency on a synthetic benchmark problem. The method is based on an extended Widrow-Hoff or Delta rule. In order to define a suitable error metric between desired and actual output spike trains, we convolve each spike sequence with a kernel function. Minimizing the error allows the efficient training of an output neuron to respond to specific input spikes with a desired target spike train.

We first present the new learning method. In the experiment section, we demonstrate the method using arbitrary input-output spike sequences for training an output neuron. Furthermore, we apply the learning method on a synthetic spatio-temporal classification problem and discuss the results.

Learning method

In this section, we describe the neural and synaptic model used in this study, followed by a detailed description of the proposed learning algorithm.

Neural and synaptic model

The Leaky Integrate-and-Fire (LIF) neuron is arguably the best known model for simulating spiking networks. It is based on the idea of an electrical circuit 1 For example the method does not require the definition of learning windows. containing a capacitor with capacitance C and a resistor with a resistance R, where both C and R are assumed to be constant. The dynamics of a neuron i are then described by the following differential equation:

τ m du i dt = -u i (t) + R I syn i (t) (1)
The constant τ m = RC is called the membrane time constant of the neuron. Whenever the membrane potential u i crosses a threshold ϑ from below, the neuron fires a spike and its potential is reset to a reset potential u r . Following [START_REF] Gerstner | Spiking Neuron Models: Single Neurons, Populations, Plasticity[END_REF], we define t

(f ) i : u i (t (f ) ) = ϑ, f ∈ {0, . . . , n -1} (2)
as the firing times of neuron i where n is the number of spikes emitted by neuron i. It is noteworthy that the shape of the spike itself is not explicitly described in the traditional LIF model. Only the firing times are considered to be relevant. The synaptic current I syn i of neuron i is modeled using an α-kernel:

I syn i (t) = j w ij f α(t -t (f ) j ) (3)
where w ij ∈ R is the synaptic weight describing the strength of the connection between neuron i and its pre-synaptic neuron j. The α-kernel itself is defined as

α(t) = e τ -1 s t e -t/τs Θ(t) (4)
where Θ(t) refers to the Heaviside function and parameter τ s is the synaptic time constant. We describe the parametrization of the model equations in the experiments section of this paper.

Learning

Similar to other supervised training algorithms, the synaptic weights of the network are adjusted iteratively in order to impose a desired input/output mapping to the SNN. We start with the common Widrow-Hoff rule for modifying the weight of a synapse i:

∆w WH i = λx i (y d -y out ) (5)
where λ ∈ R is a real-valued positive learning rate, x i is the input transferred through synapse i, and y d and y out refer to the desired and the actual neural output, respectively. This rule was introduced for traditional neural networks with linear neurons. For these models, the input and output corresponds to real-valued vectors.

In SNN however, trains of spikes are passed between neurons and we have to define how to implement Equation 5 for this form of information exchange. In order to define the distance between spike trains, we introduce here an extended Delta rule. We transform each spike sequence into a continuous function using a kernel function. This is similar to the binless distance metric used to compare spike trains [START_REF] Van Rossum | A novel spike distance[END_REF]. In this study, we use an α-kernel, however many other kernels appear suitable in this context. The convolved input x i is defined as

x i (t) = f α(t -t (f ) i ) (6)
where α(t) refers to the α-function defined in Equation 4. Representing spikes as a continuous function allows us to define the difference between spike sequences as the difference between their representing functions. Similar to the neural input, we define the desired and actual outputs of a neuron:

y d (t) = f α(t -t (f ) d ) (7)
y out (t) = f α(t -t (f ) out ) (8)
As a consequence of the spike representation defined above, ∆w WH i itself is a function over time. By integrating ∆w WH i we obtain a scalar ∆w i that is used to update the weight of synapse i:

∆w i = λ x i (t) (y d (t) -y out (t)) dt (9)
Weights are updated in an iterative process. In each iteration (or epoch), all training samples are presented sequentially to the system. For each sample the ∆w i are computed and accumulated. After the presentation of all samples, the weights are updated to w i (e + 1) = w i (e) + ∆w i , where e is the current epoch of the learning process.

We note that the algorithm is capable of training the weights of a single neural layer only. Related methods such as ReSuMe [START_REF] Ponulak | ReSuMe -new supervised learning method for spiking neural networks[END_REF] and the Chronotron [START_REF] Florian | The chronotron: a neuron that learns to fire temporally-precise spike patterns[END_REF] exhibit similar restrictions. Therefore, a combination with the well-known Liquid State Machine (LSM) approach [START_REF] Maass | Real-time computing without stable states: A new framework for neural computation based on perturbations[END_REF] was suggested in these studies. By transforming the input into a higher-dimensional space, the output of the LSM can potentially be mapped to any desired spike train.

Figure 1 illustrates the functioning of the learning method. An output neuron is connected to three input neurons through three excitatory synapses with randomly initialized weights. For the sake of simplicity, each input sequence consists of a single spike only. However, the learning method can also deal with more than one spike per input neuron. The inputs t (f ) i are visualized in Figure 1A. In this example, we intend to train the output neuron to emit a single spike at a pre-defined time t (0) d . Assume that, as shown in Figure 1B, the presented stimulus causes the excitation of the output neuron resulting in the generation of two output spikes at times t 9. Using Equations 6 and 7, the input, actual and desired spikes trains are convolved with the α-kernel (Figure 1B andC). We define the area under the curve of the difference y d (t)y out (t) as an error between actual and desired output:

E = |y d (t) -y out (t)| dt (10)
Although this error is not used in the computation of the weight updates ∆w i , this metric is an informative measure of the achieved training status of the output neuron. Figure 1E shows the weight updates ∆w i . We especially note the large decrease of weight w 0 . The spike train t (0) 0 of the first input neuron causes an undesired spike at t (0) out and lowering the corresponding synaptic efficacy potentially suppresses this behavior. On the other hand, the synaptic weight w 2 is increased promoting the triggering of spike t [START_REF] Bell | Synaptic plasticity in a cerebellumlike structure depends on temporal order[END_REF] out at an earlier time. We note that, unlike related methods such as ReSuMe [START_REF] Ponulak | ReSuMe -new supervised learning method for spiking neural networks[END_REF], the defined learning rule does not employ any learning windows making the method easy to comprehend and to implement.

Experimental results

Two experiments are conducted to demonstrate the functioning and the efficiency of the proposed learning algorithm. The first experiment intends to demonstrate the concept of the proposed learning algorithm, while the second

Neural Model Type

Leaky integrate-and-fire (LIF) neuron Description Dynamics of membrane potential u(t):

-Spike times: Table 1: Tabular description of the experimental setup as suggested in [START_REF] Nordlie | Towards reproducible descriptions of neuronal network models[END_REF].

t (f ) : u(t (f ) ) = ϑ -Sub-threshold dynamics: τm du dt = -u(t) + R I syn (t) -Reset & refractoriness: u(t) = ur∀f : t ∈ (t (f ) , t (f ) + τ ref ) -exact
experiment implements the more practical scenario of a spatio-temporal pattern classification problem. We follow the initiative recently proposed in [START_REF] Nordlie | Towards reproducible descriptions of neuronal network models[END_REF] for promoting reproducible descriptions of neural network models and experiments. The initiative suggests the use of specifically formatted tables explaining neural and synaptic models along with their parametrization, cf. Table 1. For both experiments, we use 200 input neurons that stimulate the synapses of a single output neuron, cf. Figure 2. The spike trains for each input neuron are sampled from a uniform random distribution in the interval [0, 200]ms. For the sake of simplicity of this demonstration, we allow only a single spike for each input neuron (multiple spikes in a spike train are processed in the same way). A similar experimental setup was used in [START_REF] Florian | The chronotron: a neuron that learns to fire temporally-precise spike patterns[END_REF]. The single output neuron is fully connected to all 200 input neurons with randomly initialized connection weights. All of our experiments employ the SNN simulator NEST [START_REF] Gewaltig | Nest (neural simulation tool)[END_REF].

Demonstration of the learning rule

The purpose of the first experiment is to demonstrate the concept of the proposed learning method. The task is to learn a mapping from a random input spike Fig. 2: A single output neuron is trained to respond with a temporally precise output spike train to a specific spatio-temporal input. The organization of the figure is inspired by [START_REF] Florian | The chronotron: a neuron that learns to fire temporally-precise spike patterns[END_REF].

pattern to specific target output spike train. This target consists of five spikes occurring at the times t

(0) d = 33, t (1)
d = 66, t (2)
d = 99, t (3)
d = 132 and t

d = 165ms. Initially, the synaptic weights are randomly generated uniformly in the range (0, 25pA). In 100 epochs, we apply the new learning rule to allow the output neuron to adjust its connection weights in order to produce the desired output spike train. The experiment is repeated for 100 runs each of them initialized with different random weights in order to guarantee statistical significance.

In Figure 2, the experimental setup of a typical run is illustrated. The left side of the diagram shows the network architecture as defined in the experimental setup above. The right side shows the desired target spike train (top) along with the produced spike trains by the output neuron over a number of learning epochs (bottom). We note that the output spike trains in early epochs are very different from the desired target spike sequence. In later epochs the output spike converges towards the desired sequence. Consequently, the error as defined in Equation 10decreases in succeeding epochs (right part of Figure 2). We note that the neuron is able to reproduce the desired spike output pattern very precisely in less than 30 learning epochs.

Figure 3 shows the evolution of the average error over the performed 100 runs. We note the logarithmic scale of the y-axis and the exponential decrease of the error. The slope of the curve suggests further improvement of the error, if the neuron was trained longer than the 100 performed learning epochs.

From this simple experiment, we conclude that the proposed learning method is indeed able to train a input-output behavior to a spiking neuron.

Classification of spatio-temporal data

The second experiment is a spatio-temporal classification task. The objective is to learn to classify five classes of input spike patterns. The pattern for each class Average Error Evolution of Average Error Fig. 3: Evolution of average error obtained in 100 independent runs is given as a random input spike pattern that was created in a similar fashion as for the first experiment. Fifteen copies for each of the five pattern are then generated by perturbing each pattern using a Gaussian jitter with a standard deviation of 3ms resulting in a total of 15 × 5 = 75 samples in the training data set. Additionally, we create 25 × 5 = 125 testing samples using the same procedure. The output neuron is then trained to emit a single spike at a specific time for each class. Only the training set is used during training, while the testing set is used to determine the generalization ability of the trained neuron. The spike time of the output neuron encodes the class label of the presented input pattern. The neuron is trained to spike at the time instances 33, 66, 99, 132, and 165ms respectively, each spike time corresponding to one of the five class labels. We allow 200 epochs for the learning method and we repeat the experiment in 30 independent runs. For each run we chose a different set of random initial weights.

Figure 4a shows the evolution of the average error for each of the five classes. In the first few epochs, the value of the error oscillates and then starts to stabilize and decrease slowly. The learning error decreases for some classes faster than for others, e.g. class 3. We also note that the class reporting the highest error is class 1. This behavior is expected and confirms a quite similar finding in [START_REF] Florian | The chronotron: a neuron that learns to fire temporally-precise spike patterns[END_REF]. In order to classify samples of class 1 correctly, the output neuron has to emit a very early spike at t ≈ 33ms. Consequently, the neuron needs to be stimulated by input spikes occurring at times before t = 33ms. However, due to the random generation of the input data, only few input spikes occur before t = 33ms. Most input spikes arrive after that time at the output neuron and therefore do not contribute to the correct classification of class 1 samples. The relationship between the accuracy and the output spike time was also noted in [START_REF] Florian | The chronotron: a neuron that learns to fire temporally-precise spike patterns[END_REF]. Future studies will further investigate this interesting observation.

In order to report the classification accuracy of the trained neuron, we define a simple error metric. We consider a pattern as correctly classified, if the neuron d . Any other output is considered as incorrect. It is noteworthy to mention that using this definition, an untrained neuron is very likely to produce incorrect outputs resulting in accuracies close to zero. Figure 4b shows the average classification error for each class in the training and testing phase. As mentioned above, for testing, the 125 unseen patterns of the test set are used. The neuron is able to learn to classify the 75 training patterns with an average accuracy of 94.8% across all classes. Once more, we note the comparatively poor classification performance of samples belonging to the first class. For the test patterns, the neuron is able to achieve average accuracy of 79.6% across all classes which demonstrates a satisfying generalization ability.

Conclusion

In this paper, we have proposed a novel learning algorithm that allows the training of a precise input-output behavior to a SNN. Besides the benefits of an efficient supervised training method for a spiking neuron, we see the main advantage of the proposed approach in its algorithmic simplicity promoting its straightforward application to engineering problems. As a proof of concept, we have demonstrated the suitability of the method for classification problems on a small synthetic data set. Although the method can only train a single layer of the SNN, it was argued that in combination with the LSM approach [START_REF] Maass | Real-time computing without stable states: A new framework for neural computation based on perturbations[END_REF], the method allows the processing of complex non-linearly separable classification problems. Considering the difficulty of the spike-based classification task, the results are both satisfying and encouraging.

Fig. 1 :

 1 Fig. 1: Illustration of the proposed training algorithm. See text for a detailed explanation of the figure.

  Figure 1B. The lower part in the figure (Figure 1C,D,E) depicts a graphical illustration of Equation

  integration with temporal resolution dt Parameters Membrane time constant τm = 10ms Membrane resistance R = 333.33MΩ Spike threshold ϑ = 20mV, reset potential ur = 0mV Refractory period τ ref = 3ms Time resolution dt = 0.1ms, simulation time T = 200ms Synaptic Model Type Current synapses with α-function shaped post-synaptic currents (PSCs) Description Synaptic input current I syn(t) = w f α(tt (f ) ) α(t) = e τ -1 s t e -t/τs , if t > 0 0, otherwise Parameters Synaptic weight w ∈ R, uniformly randomly initialized in [0, 25] Synaptic time constant τs = 5ms Input Model Type Random input Details Population of 200 input neurons each firing a single spike at a randomly chosen time in the period (0, T )

Fig. 4 :d + 3 ]

 43 Fig. 4: Evolution of the average errors obtained in 30 independent trails for each class of the training samples (a). The average accuracies obtained in the training and testing phase (b).

Acknowledgements

The work on this paper has been supported by the Knowledge Engineering and Discovery Research Institute (KEDRI, www.kedri.info). One of the authors, NK, has been supported by a Marie Curie International Incoming Fellowship with the 7 th European Framework Programme under the project 'EvoSpike', hosted by the Neuromorphic Cognitive Systems Group of the Institute for Neuroinformatics of the ETH and the University of Zurich.


Karim Ouazzane
email: k.ouazzane@londonmet.ac.uk

Jun Li

Hassan B Kazemian
email: h.kazemian@londonmet.ac.uk

An Intelligent Keyboard Framework for Improving Disabled People Computer Accessibility

Keywords:

Computer text entry may be full of noises -for example, computer keyboard users inevitably make typing mistakes and their typing stream implies all users' self rectification actions. These may produce a great negative influence on the accessibility and usability of applications. This research develops an original Intelligent Keyboard hybrid framework, which can be used to analyze users' typing stream, and accordingly correct typing mistakes and predict users typing intention. An extendable Focused Time-Delay Neural Network (FTDNN) n-gram prediction algorithm is developed to learn from the users' typing history and produce text entry prediction and correction based on historical typing data. The results show that FTDNN is an efficient tool to model typing stream. Also, the computer simulation results demonstrate that the proposed framework performs better than using the conventional keyboard.

Introduction

It is inevitable that users will make typing mistakes, which is particularly the case for disabled users. These are different kinds of mistakes such as spelling errors and adjacent key press errors etc. [START_REF] Ouazzane | A hybrid framework towards the solution for people with disability effectively using computer keyboard[END_REF] [START_REF] Trewin | An invisible keyguard[END_REF]. A series of research based on words vocabulary which apply both, neural network and language modeling have been carried out; Bengio and Ducharme [START_REF] Bengio | A neural probabilistic language model[END_REF] suggested a model using neural network probabilistic language modeling to learn distributed representation for words that allow each training sentence to inform the model about exponential number of semantically neighboring sentences. Schwenk and Gauvain [START_REF] Schwenk | Connectionist Language Modeling for Large Vocabulary Continuous Speech Recognition[END_REF] addressed a related problem further by carrying out probability estimation in a continuous space and enabling a smooth interpolation of the probabilities. However, due to the curse of dimensionality in the discrete space representation, they still have to narrow the vocabulary by using a shortlist which damages the prediction accuracy and fail to learn a long-span language model with n >> 3 gram. An alternative way is to install filters which modify the control signals generated by the device. Such filters can have a significant effect on the speed and accuracy with which a device can be used. Attempts have also been made by IBM to devise intelligent mechanisms that could adjust the settings of the keyboard accessibility features by detecting the usage problems as introduced in Trewin and Pain [START_REF] Trewin | Pain: A Model of Keyboard Configuration Requirements[END_REF].

Dasher [START_REF]Inference Group of Cambridge[END_REF] is an information-efficient text-entry interface, driven by natural continuous pointing gestures [START_REF] Ward | Dasher -a Data Entry Interface Using Continuous Gestures and Language Model[END_REF]. It is based on language model prediction, through which the space of interface is determined to each piece of text. It is useful to the users who operate a computer onehanded, by joystick, touch screen, trackball or mouse, which might be an inspiration for QWERTY keyboard [START_REF][END_REF] tools development. Although the Dasher project is a good step forward, it still has some limitations. For example, Dyslexia can cause significant problems in remembering even short sequences of numbers in the correct order. Those types of disability frequently cause typing mistakes, which haven't been well solved. ProtoType [9] is a piece of software used to type text into other programs such as a word processor based o lists of words, which includes word prediction, spelling correction and word banks. ProtoType is designed to improve spelling for people with dyslexia or spelling difficulties, but it is dysfunctional to correct the keystrokes mistakes made by most motor disabled people. Research works have been carried out to address this problem, these include Metaphone [10] and n-grams [11], each of them may have its unique features, but they all have limitations.

Intelligent models such as neural network models have been implemented in various directions, however, they are hardly seen to apply to noisy text entry processing such as user typing stream. Moreover, although efforts have been made in multiple directions such as language modeling, natural language processing and user interface design, those technologies, if used alone, will fail to meet the user's particular needs which have been presented above. It is also worth arguing that combination of models such as Jianhua & Graeme word prediction model [START_REF] Li | Semantic Knowledge in Word Completion[END_REF] emphasizes excessively on providing a global method, and lack 'user-oriented' feature. Furthermore current models are short of self-adaptive ability (i.e. learning ability), and fail to fully recognize the right patterns from user's distinct performance.

Ouazzane et al. [START_REF] Ouazzane | A hybrid framework towards the solution for people with disability effectively using computer keyboard[END_REF] presented a hybrid framework based on machine learning model to offer an efficient solution for people having difficulties using QWERTY keyboard; it integrates neural network, language model and natural language processing technologies, to provide users with two fundamental functions, namely, word prediction and typing correction. Also, Li et al. [START_REF] Li | Evolutionary ranking on multiple word correction algorithms using neural network approach[END_REF] combined distinct word correction algorithms, using neural network approach, to produce an optimal prediction. It was demonstrated that neural network, as a learning means, can provide an optimum solution through combining distinct algorithms in order to achieve a high ranking performance. Hence, the main purpose of this research is to develop an Intelligent Keyboard hybrid novel framework which is based on the combination of multiple technologies (i.e. neural network and language modeling) and therefore to put all merits of those distinct solutions. It is desirable to develop a solution that is evolutionary and adjustable, which can learn from users' past mistakes and then to predict and/or correct these mistakes. In order to achieve the text correction and prediction, in this investigation a Focused Time-Delay Neural Network (FTDNN) [START_REF]Focused Time-Delay Neural Network (newfftd)[END_REF] is chosen, which can represent the unclear and complex relationship between current typed sequence and its precedent one. In this paper, the Intelligent Keyboard hybrid system's case study is developed and the conclusion of the work alongside future recommendations is provided at the end.

As to cognitive tasks it is shown that rather than seek solutions based on the symbolic Artificial Intelligence, a more potentially useful approach is to build a structured connectionist model or a hybrid system. Then it is able to combine more functions for some specific purposes, which includes four fundamental elements, namely, Environment Analysis, Learning Elements, Knowledge Base and Performance element [START_REF] Haykin | Neural Networks -A comprehensive Foundation[END_REF]. All of these units could be divided into more subdivisions to form a highly efficient hybrid system. Therefore, an Intelligent Keyboard (IK) hybrid system which combines neural network, statistics and natural language model together, is designed and intends to provide users with fundamental functions such as typing prediction and correction. User's typing data stream can be checked, rectified, and predicted in sequence by going through each unit following user's typing process. Through this way, the typing stream's noises are filtered significantly and the language interaction between a user and a computer becomes smoother. Multiple units and a database separated by a long-term memory and a short-term memory have been presented according to the technologies. The architecture is shown in Figure 1, which includes four processing units: Text prediction, Inference, Natural Language Processing and Error correction units; and two additional modules: User interface module and Noise process module to enable the interaction of the user with the outside environment (i.e., computer keyboard).

The two additional modules function as data pre-processing, post-processing and interaction interface. They correspond to machine learning model's Environment Element and part of Performance Element respectively. The Knowledge Base element is represented by Long-term Memory and Short-term Memory. The rules inferred from Inference Engine and some other facts such as user profile and frequently used texts, are saved in the Long-term Memory. Other facts such as recently used new words are stored in the Short-term Memory which will be transferred to the long-term memory if a certain threshold is reached. The framework is invoked by user's key strokes. As much of data stream typing could be un-preprocessed, incomplete and noisy, for example, a long key press generates more than one Window's message, the data stream needs to undergo Noises Processing module first. Through this module the input vectors are further exploited, which would include the key-up signals, keydown signals, the time difference between two consecutive strokes and so on. The definition of noises can be given according to the user profile. Subsequently, a representation vector which includes time stamp and Virtual Key Code (VKC) [16] message is chosen to be sent to the processing units, namely, Text Prediction unit and Error Correction unit. Both units process the vectors based on the association rules, dictionaries and some other facts retrieved from the memory.

Text prediction unit is composed of different algorithms developed based on different scientific methods such as statistics and phonemics while Error Correction unit is designed based on users' performance. Firstly, a spell checker function is used to detect if a mistake had occurred. In the case of no mistakes being traced, the unit processing is stopped and the result is passed on to Inference Engine. Otherwise, the function such as motor checker, to process motor disability errors, would be evoked if spell checker fails to present a result. These two units (i.e. Text Prediction unit and Error Correction unit) can process data stream simultaneously. The typing mistakes which are still under doubt are further checked by Natural Language Processing unit to check syntax and semantics' errors. Finally, the results are refurbished and shown to the user by User Interface unit.

The results (e.g. a list of words) generated from Text Prediction unit and Error Correction unit, which are usually more than one, are presented to Inference Engine unit. The Inference Engine unit ranks the results based on their probabilities to generate a word-list or directly presents a highest ranking presentation to the user. The user's feedback such as selections and correction actions is recorded by the inference algorithms (here is a neural network algorithm), and transferred to rules or rewards to be stored into the memory.

Shown as in Figure 1, an input of a sentence is a process passing through different structure status from letter, word to sentence, during which distinct units are evoked up according to the structure status' changes. 3 Intelligent Keyboard Framework Demonstration Using N-gram Focused Time-Delay Neural Network Modeling

Modeling Data Sets

As illustrated above, IK framework provides two fundamental functions namely, prediction and correction. Typing stream prediction and correction can be achieved based on historical data by using neural networks through designing two models; or can be developed using a single neural network architecture if the correction can be considered as one specific case of prediction -the model produces the right symbol based on the inaccurate historical data. At this point a model based on Focused Time-Delay neural network modeling associated with different datasets is designed and implemented in the following sections. The used datasets are shown below;

♦ Dataset one: a novel -'Far from the Madding Crowd' was written by Thomas Hardy [START_REF] Hardy | Wikipedia[END_REF]. It has been used as a testing sample by some compression algorithms such as PPM* [START_REF] Cleary | Unbounded length contexts for PPM[END_REF]. The version used here is extracted from Calgary Corpus [START_REF] Bloom | PPMZ-High Compression Markov Predictive Coder[END_REF].

♦ Dataset two: it is extracted from a charity keystroke log. From the reflected keystroke log, the typing mistakes are predominantly about adjacent key press and prolong key press errors. The keystroke recording tool used in this research is KeyCapture software [START_REF] Soukoreff | [END_REF], which has been modified and adjusted for the purpose of this research. It runs in background under Windows environment to collect keystrokes without interfering with user's work.

N-gram Focused Time-Delay Neural Network Modeling

Studying user's typing behavior would require the network to study user's history. FTDNN is suitable for time-series prediction. However, a comprehensive research on Focused Time-Delay Neural Network language modeling has never occurred. In the following sections an extendable FTDNN n-gram prediction is developed to predict noise-free and typing stream datasets.

♦

%a b c d e f g h i j k l m n o p q r s t u v w x y z ''

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 %'t' 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 %'h' 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 %'e'

First, all of the capital alphabets are converted to their corresponding lower case. The other symbols which do not belong to symbol set A are converted to space. Then each symbol is converted to a twenty seven length unary code. As shown in the example above, the word 'the' is successfully converted.

For this n-gram prediction, a three layer FTDNN network with 27 input neurons, 27 output neurons, extendible numbers of hidden layer neurons and extendible numbers of time delays is designed. Both input and output are encoded in unary code. A post processing function which ranks the twenty-seven output in a descending order has been used to produce the unary code results: the maximum value is converted into one and the rest of the values are converted into zeros.

Simulation with Plain Text Data Using Dataset One

In order to weight the computer simulation results, two concepts are introduced here, namely, Hitting Rate and First Three (FT) Hitting Rate. If given a testing metrics P, a target metrics T and a testing result metrics R with their numbers of lines and columns equal and expressed as n, m respectively, then the Hitting Rate is

{ | ( ) / , } i i i HR hr hr zeros T R n i m   
 , where i R is a vector of the th i Rank Conversion Value of R, and zeros() is the function to compute the numbers of zero vectors included in the metrics. For instance, the second Hitting Rate is the second best option for the symbol prediction in all output neurons of FTDNN, while the third Hitting Rate is the third best option etc. Obviously the sum of all Hitting Rates is 1 100%

k i i HR HR     .
During the FTDNN model training and testing using the dataset one, the numbers of grams - [START_REF] Ouazzane | A hybrid framework towards the solution for people with disability effectively using computer keyboard[END_REF][START_REF] Trewin | An invisible keyguard[END_REF][START_REF] Bengio | A neural probabilistic language model[END_REF][START_REF] Trewin | Pain: A Model of Keyboard Configuration Requirements[END_REF][START_REF] Ward | Dasher -a Data Entry Interface Using Continuous Gestures and Language Model[END_REF]9,11,[START_REF] Li | Evolutionary ranking on multiple word correction algorithms using neural network approach[END_REF] which are represented by time delays, and the numbers of hidden neurons - [START_REF] Ouazzane | A hybrid framework towards the solution for people with disability effectively using computer keyboard[END_REF][START_REF] Trewin | An invisible keyguard[END_REF][START_REF] Bengio | A neural probabilistic language model[END_REF][START_REF] Trewin | Pain: A Model of Keyboard Configuration Requirements[END_REF][START_REF] Ward | Dasher -a Data Entry Interface Using Continuous Gestures and Language Model[END_REF]9,[START_REF] Haykin | Neural Networks -A comprehensive Foundation[END_REF]25,50,100] are cross-designed and implemented. Thereinto, as the gram reaches 11 and the number of hidden neurons reaches 100, or as the gram reaches 13 and the number of hidden neurons reaches 15 or onwards, the memory of current system is beyond its limit. Therefore, the computer simulation results are abandoned from G-11 & H-100 onwards.

In order to demonstrate the effect of different grams on the hitting rate, a type of plots is produced based on the same computer simulation results, as shown in Figure 2. It has [START_REF] Ouazzane | A hybrid framework towards the solution for people with disability effectively using computer keyboard[END_REF][START_REF] Trewin | An invisible keyguard[END_REF][START_REF] Bengio | A neural probabilistic language model[END_REF][START_REF] Trewin | Pain: A Model of Keyboard Configuration Requirements[END_REF][START_REF] Ward | Dasher -a Data Entry Interface Using Continuous Gestures and Language Model[END_REF]9,11] grams associated with various number of hidden neurons. It is evident that 2, 3 & 5-gram give the best three First Hitting Rates while 1, 2 & 3gram give the best three FT Hitting Rates (Note: in a small margin 2-gram gives the best FT hitting rates and 3-gram gives the best First hitting rate). From both plots of Figure 2, the lower grams (1, 2 & 3) show a better convergence toward the maximum Hitting Rate (i.e. FT HR is around 56%, First HR is around 33%). Both figures illustrate that smaller Hitting Rates occur from 4-gram onward. This proves that the more historical dataset input the more learning neural network space is needed, and the more training is needed toward convergence. Under the current training sample the results suggest that there is a best gram with certain number of hidden units to suit the prediction best. Beyond a critical point of prediction rate, further increase of gram or hidden unit doesn't help to achieve a better performance. Figure 2 also shows that the number of neurons in hidden layer affects the model's learning ability and Hitting Rate. For example, the number of neurons in hidden layer should not be too small to a structured symbol set {a, … , z, space} distribution; otherwise, it would be difficult for neural network to reach a good hitting rate. The 11-gram testing stops at fifty neurons; a 27-100-27 three layer FTDNN model has failed to complete the training process under the current system environment.

N-gram Prediction with Typing Data using Dataset Two

As analyzed, the designed n-gram FTDNN models have shown that it can be applied to data prediction with a high capability. Here a user's typing data stream (i.e. dataset two) is used to further test the model. The users typing history is analyzed by the model to predict user's next typing intention. As the typing data stream is a typical noisy dataset which includes user's typing mistakes as well as self correction strokes such as symbols 'backspace' and 'delete', the model will not only learn the habits of user using language but also learn the self-correction actions which occurs in typing stream. For example, a self-correction action from a wrong typing word 'desj' to the right word 'desk' can be broken down in a typing data stream as,

d => e => s => j => backspace => k …
This is a typical adjacent-key-press error usually made by some people with motor disability or Parkinson disease. Through training, the FTDNN model is able to learn 2-gram prediction rules between the predecessor and successor, for instance,

d  e e  s
From the typing stream shown above, the model learns not only the existing noises such as 's'  'j', but also the correction actions such as 'j'  'backspace'. In practice, users just continue their typing without stop in spite of the mistakes, while the model should be able to correct the mistakes automatically or specify recommendations later on.

The collected data stream in dataset two is expressed in Virtual Key Codes [16]. In this research only editing virtual keys are adopted, other keys such as arrows are discarded. Then, the size of symbol set originally used by the model is extended into fifty three individual symbols, which apart from alphabet it also includes some other symbols such as, Based on the original design of FTDNN model, an extension to fifty-three units both at the input and output layer has been made. The dataset two has recorded both the key press 'down' status and 'up' status. Considering some disabled people specific typing behavior such as prolonged key press which would generate more 'down' keys corresponding to one 'up' keys, the keystrokes with 'down' status are selected by the pre-processing function for neural network training and testing. A comparison among the gram set [START_REF] Ouazzane | A hybrid framework towards the solution for people with disability effectively using computer keyboard[END_REF][START_REF] Bengio | A neural probabilistic language model[END_REF][START_REF] Trewin | Pain: A Model of Keyboard Configuration Requirements[END_REF][START_REF] Ward | Dasher -a Data Entry Interface Using Continuous Gestures and Language Model[END_REF]9] based on various numbers of hidden neurons - [START_REF] Bengio | A neural probabilistic language model[END_REF][START_REF] Trewin | Pain: A Model of Keyboard Configuration Requirements[END_REF][START_REF] Ward | Dasher -a Data Entry Interface Using Continuous Gestures and Language Model[END_REF]9,[START_REF] Haykin | Neural Networks -A comprehensive Foundation[END_REF]25,50,100] is shown in Figure 3. The first plot demonstrates a comparison of several grams' First Hitting Rates with an increase of hidden neurons. The second plot is a comparison of FT Hitting Rate between different grams. Figure 3 shows that 1-gram produces the maximum FT Hitting Rate of 53% whereas 3-gram with fifty hidden neurons produces the maximum First Hitting Rate of 38.1%. Similar results have been obtained using dataset one; the lower grams (1, 2 & 3-gram) show a better solution using FTDNN model prediction under current circumstances. Both datasets demonstrate a highly accurate prediction rate (FT Hitting Rate, approximately 50%) with FTDNN model.

Conclusion

This research work brings forth an original concept, Intelligent Keyboard hybrid framework for noisy language processing. It is developed as a hybrid solution based on FTDNN and n-gram technologies. Plain text dataset and user typing stream are tested in sequence based on extendable input and hidden neurons. Considering the user's typing history, a 38% First Hitting Rate and a 53% First Three Hitting Rate are obtained. The size of a training dataset, the occurrence of each symbol in a training dataset and the relationships among symbols of a training dataset all play an important role in the determination of a neural network language modeling prediction accuracy. The results further demonstrate that 1, 2 & 3-gram generate better outcomes than other grams.

For future work, a distributed representation method to preprocess the typing symbols, where each symbol is represented by several features such as key distance, time stamp and symbols can be applied to the FTDNN models. In such a case, the prediction will not be solely based on the symbols themselves but also on related ngram features.

Acknowledgments. The research is funded by Disability Essex [START_REF]Disability[END_REF] and Technology Strategy Board [START_REF]Knowledge Transfer Partnership[END_REF]. Thanks to Pete Collings for helpful advice and discussions.

Fig. 1 .

 1 Fig. 1. The Architecture of Intelligent Keyboard; blue boxes and their connections represent the system's input and output process. The processing units from left to right, which has been marked as light yellow (or light grey in the case of black/white printing), are named as Text prediction unit (No. 6), Inference engine unit (No. 7), Error correction unit (No. 8), and Natural Language Processing (NLP) unit (No. 9). Data storage (No. 12) is divided into short-term memory and long-term memory, where the temporary and permanent information are stored. There are two additional modules: Noise process module (No. 10) and User interface module (No. 11), which are responsible for the interaction with the outer environment such as keyboard.

Fig. 2 .

 2 Fig. 2. N-gram First and First Three Hitting Rate curves; both have [1, 2, 3, 5, 7, 9, 11] grams associated with various number of hidden neurons.

Fig. 3 .

 3 Fig.3.[START_REF] Ouazzane | A hybrid framework towards the solution for people with disability effectively using computer keyboard[END_REF][START_REF] Bengio | A neural probabilistic language model[END_REF][START_REF] Trewin | Pain: A Model of Keyboard Configuration Requirements[END_REF][START_REF] Ward | Dasher -a Data Entry Interface Using Continuous Gestures and Language Model[END_REF] 9] gram typing stream Hitting Rates; the first plot demonstrates a comparison of several grams' First Hitting Rates with an increase of hidden neurons. The second plot is a comparison of FT Hitting Rate between different grams.

FTDNN n-gram prediction definition: let

  Due to the system environment's limitations in this research, rather than adopting the whole dataset, a chunk of data ranging from zero to 100k is selected from the dataset one. The dataset is subsequently divided into training, validation and testing data. A symbol set with twenty-seven elements, A = {a…z, space}, is applied to simplify the dataset. The processing logic is shown as follows,

			♦ Symbol-distribution definition: Given a certain ranking level m and a symbol
	set	{ ,..., , A a z space } 	, one defines the n-gram Symbol-Distribution in ranking level m
	as	m n D		{ , | x y x i	 	i y	}	, where i x symbol set A, and i y is the level m Hitting Rate
	corresponding to each symbol.
						for each symbol i s  context C, where	1 { ... } n C s s 
								if 'a' < i
											's assume existing string
	1 { . . . . | i j k m S s s s s s i j k m }     and (	) j i n k j l , ( )     , where , , , i j k s s s s are symbols m
	and i, j, k, l, m, n are natural numbers, if one builds a
	relation	n R		{ , | x y x s s ( ... ) i j n 	( y s  	j	1 	 	) } k l s	, then the relation is defined as n-gram's
	l  prediction; if one considers the special case 1 l  , then the relation is called n-
	gram's one-prediction, or n-gram prediction for short. For example, given string S=
	{student}, some 2-gram prediction cases are,

s < 'z' then write unary code to file else if 'A' < i s < 'Z' then convert to { ,..., } a z and write unary code to file else convert to blank and write unary code to file end for


Xanthos Papakonstantinou
email: pxanthosster@gmail.com

Lazaros S Iliadis
email: liliadis@fmenr.duth.gr

Elias Pimenidis

Fotis Maris

Fuzzy Modeling of the Climate Change Effect to Drought and to Wild Fires in Cyprus

Keywords: Climate Change, Fuzzy Inference System, Fuzzy Drought Indices, Heat Index, Wild Fires

This is an intelligent modeling of the evolution of drought and forest fires, due to climate change in Cyprus. Original annual wild fire data records and data regarding meteorological parameters were used. A flexible modeling approach was proposed towards the determination of drought risk indices in all of the country. Cyprus was divided in eight polygons corresponding to eight meteorological stations. A Fuzzy Inference Rule Based System (FIRBS) was developed to produce the drought risk indices vectors for the forest regions under study. An analysis of the spatial distribution of the heat index vectors was performed. Forest fires distribution through the island was addressed. All of the results were stored by using an ArcGIS, (version 9.3) spatial data base that enables more comprehensive presentation of the most risky areas. There is a significant increase of drought in the island and this has a serious effect in the problems of forest fires and heat indices.

Introduction

According to Olej and Hardi [START_REF] Olej | Modelling of Selected Areas of Sustainable Development by Artificial Intelligence and Soft Computing[END_REF][START_REF] Hardi | Measuring Sustainable Development: Review of Current Practice[END_REF] sustainable development is understood as the one that ensures the fulfillment of the needs of contemporary society without jeopardizing the opportunity to meet the needs of future generations and at the same time it improves the quality of life while living within the carrying capacity of supporting ecosystems. Drought is an extreme local climate phenomenon, difficult to be defined scientifically and this is one of the reasons that make its rational management more complicated [START_REF] Nagref | Drought A dangerous extreme climate phenomenon with special management requirements[END_REF]. Regardless the efforts for a general definition the specialists did not manage so far to agree in one that would be globally acceptable. Some of the existing descriptions for the term refer to "Meteorological drought", "Agricultural drought" and "Hydrological drought" as a result of looking at the problem from different points of view. When the word drought is used commonly, the most often intended definition is meteorological drought. However, when the word is used by urban planners, it is more frequently in the sense of hydrological drought.

Sustainable development is and it will be one of the main objectives of the contemporary societies and thus this study concentrates not only in recording the situation in terms of climate change but also in producing new Artificial Intelligence tools that can provide significant assistance towards the design of effective natural disasters management policy. The drought in the island of Cyprus is slowly but continuously getting worse during the last decades due to the fact of the climate change. The Republic of Cyprus has recognized several years ago the need for the control and management of desertification with its complicated potential consequences [START_REF] Dorflinger | Assessment of Sedimentation and Erosion, Kalavasos Reservoir and Catchments, Cyprus[END_REF][START_REF] Kosmas | The Medalus project -Mediterranean desertification and land use: Manual on key indicators of desertification and mapping environmentally sensitive areas to desertification[END_REF][START_REF] Photiou | Wastewater Treatment and Reuse-Conference on: Integrated Water Management, Policy Aspects[END_REF]. The problem is influenced by natural phenomena and also by human activities. The United Nations contract on the problem became an official law of the Cyprus Republic (Act 23 (III)/1999) and it contains a special Annex for the Mediterranean countries (Annex IV) [START_REF]Cyprus Ministry of Agriculture, Natural Resources and Environment, National Plan for fighting the desertification[END_REF]. According to this Act it is obligation of each state to design a National planning towards facing the problem.

This research effort aims first in modeling establishing and testing new flexible and efficient drought risk indices (DRI) and also in investigating the effect of climate change to a major natural disaster like forest fires. A main achievement of this effort is the design and implementation of an Intelligent Fuzzy Rule based System under the MATLAB platform where the DRI model is embedded. It is a System that performs local analysis of the drought and forest fire and estimates the degree of risk. The results are stored visually in a spatial database using ARCGIS 9.3 using actual coordinates for all of the areas.

Area of Research, Changes in the Climate Conditions

The main characteristics of the Mediterranean Cyprus climate are hot and dry summer from mid May till mid September, the rainy but soft winter which lasts from mid November till March. There are two short transition periods spring and autumn. During the 20 th century the climate of Cyprus and especially the two main climate parameters namely rainfall and temperature have followed significant variations. More specifically in Cyprus during the last century the average rain height has dropped significantly whereas the average temperature has risen. The change rate of these meteorological features has been much higher in the second half of the 20 th century. The warmest cases of the last 100 years have been recorded during the last 20 years [START_REF]Cyprus Ministry of Agriculture, Natural Resources and Environment, National Plan for fighting the desertification[END_REF]. The problem of desertification in Cyprus is a crucial one.

Literature Review

Previous research efforts estimate risk mainly based on specific imposed instructions and they do not consider major aspects and features related to the problem. They use crisp boundaries in order to classify areas based on their risk. This approach is quite error prone because an area for example with 10mm of rain height can be classified as "dry", whereas an area with 10.0001 mm as "not dry". More specifically the following stochastic research efforts have been proposed recently for modeling drought [START_REF] Ghanbarpour | Groundwater for Sustainable Development Problems, Perspectives and Challenges Chapter 14[END_REF] and some research has been done in the direction of evaluating existing models [START_REF] Stockwell | Critique of drought models in the Australian drought exceptional circumstances report (DECR)[END_REF]. Fuzzy modeling efforts have been done recently towards forest fire risk classification in Greece [START_REF] Iliadis | Α decision support system unifying partial environmental risk indices by using fuzzy intersection operators: the case of forest fires[END_REF][START_REF] Iliadis | A Decision Support System Applying an Integrated Fuzzy Model for Long-Term Forest fire Risk Estimation[END_REF][START_REF] Iliadis | An intelligent system employing an enhanced fuzzy cmeans clustering model: Application in the case of Forest Fires[END_REF][START_REF] Iliadis | A Fuzzy Inference System Using Gaussian Distribution Curves for Forest Fire Risk Estimation[END_REF][START_REF] Iliadis | Performing Fuzzy Multi-feature Scenarios for the Determination of Forest Fire Risk[END_REF]. Finally a limited number of Soft Computing Approaches applied in drought modeling has been published in the literature [START_REF] Yang | Drought modeling and products of random variables with exponential kernel[END_REF].

Materials and Methods

Data Vectors and Data Gathering

The meteorological and cartographic data have been generously provided by the Cypriot public services. More specifically the meteorological data have been provided by the Cyprus Meteorological Service. Average monthly maximum and minimum temperatures and rainfall from eight meteorological stations located in the island for a period of thirty nine years 1979-2009 were gathered. The forest department of Cyprus provided detailed forest fire data (forest fire frequencies, cause of ignition, burned areas) and an ArcGIS shape file (.shp) with spatial and land data on the forest areas of Cyprus. Wild fire data are related to period 30 th of April 1979 to the 24 th of October 2009. Real coordinates of the center of each wild fire from 1994 till 2009 were gathered.

A Fuzzy Inference System for the Determination of DRI

In fuzzy algebra every piece of data belongs to a fuzzy set with a degree of membership (in the closed interval [0, 1]) that is determined by a fuzzy membership function (FMF) [START_REF] Iliadis | Intelligent Information Systems and Applications in Risk Estimation[END_REF][START_REF] Kecman | Learning and Soft Computing[END_REF]. Thus each case can belong to both of "Risky area" and "High Risk area" fuzzy sets with a distinct degree of membership [START_REF] Kecman | Learning and Soft Computing[END_REF]. This research aims in proposing a reliable, flexible and rational Drought Risk Index (DRI). This is achieved with the use of fuzzy sets in order to determine the partial risk indices (due to each risk parameter). These partial risk indices can be unified to a unique risk index by the employment of fuzzy relations and more specifically fuzzy conjunction Norms. Average monthly maximum temperature, average monthly minimum temperature, average monthly relative humidity, average total rainfall were considered in this research.

Fuzzy Degree Membership Functions

Semi-Triangular and Triangular membership functions (functions 1 and 2 respectively) were used to determine the degree of membership (DOM) of each area under study to each corresponding fuzzy risk set ( [START_REF] Iliadis | Α decision support system unifying partial environmental risk indices by using fuzzy intersection operators: the case of forest fires[END_REF], [START_REF] Iliadis | An intelligent system employing an enhanced fuzzy cmeans clustering model: Application in the case of Forest Fires[END_REF], [START_REF] Iliadis | Intelligent Information Systems and Applications in Risk Estimation[END_REF]). The choice of these functions was based on the fact that they offer only one single peak point where the DOM equals 1 and thus they distinguish the areas in a more straightforward manner.

⎪ ⎪ ⎩ ⎪ ⎪ ⎨ ⎧ > ∈ ∈ < = b X if 0 b] [c, X if c) - X)/(b - (b c] [a, X if a) - a)/(c - (X a X if 0 (Χ) μ s (1) ⎩ ⎨ ⎧ ∈ < = c] [a, X if a) - a)/(c - (X a X if 0 (Χ) μ s (2)
Unification of Partial Risk Indices. After the determination of the partial degrees of risk due to each separate feature, the final target was the unification of all indices to a unique risk index (the DRI) for each of the eight areas under evaluation. In fact the DRI would express the degree of membership of each area to the fuzzy set "Area of high Drought". A special case of fuzzy relation namely the Minimum T-Norm was applied in order to perform the fuzzy conjunction. The following function 3 presents the Minimum T-Norm approach where μΑ(Χ) and μB(X) are the DOM of element X to the fuzzy sets A and B respectively ( [START_REF] Iliadis | An intelligent system employing an enhanced fuzzy cmeans clustering model: Application in the case of Forest Fires[END_REF], [START_REF] Iliadis | A Fuzzy Inference System Using Gaussian Distribution Curves for Forest Fire Risk Estimation[END_REF], [START_REF] Kecman | Learning and Soft Computing[END_REF], [START_REF] Cox | Fuzzy Modeling and Genetic Algorithms for Data Mining and Exploration[END_REF]). DRI=

)

) ( μ (Χ), MIN(μ Β Α Χ (3)
The fuzzy conjunction performs the fuzzy "AND" operation between two fuzzy sets. This means that the minimum T-Norm determines the DOM of an element x to the fuzzy set A AND B. After the determination of the fuzzy sets that correspond to each parameter, the next step was the design and construction of the "Linguistic" fuzzy rules to be used for the operation of the fuzzy controller. Thus a Mamdani FIS ( [START_REF] Cox | Fuzzy Modeling and Genetic Algorithms for Data Mining and Exploration[END_REF][START_REF] Kuncheva | How good are Fuzzy IF-THEN Classifiers[END_REF][START_REF] Pedrycz | Fuzzy Control and Fuzzy Systems. 2nd Extended Edition[END_REF][START_REF] Vascak | Automatic Adaptation of Fuzzy Controllers[END_REF]) was designed built and tested. The System was designed to use four risk parameters and three FMF (corresponding to "Low", "High", "Middle" linguistics) for each parameter. So according to Olej and Hajek ([1], [START_REF] Kecman | Learning and Soft Computing[END_REF]) totally 81 3 4 = fuzzy rules should be built, which leads to an acceptable degree of complexity. Thus, the System was built to contain totally 81 fuzzy rules, derived from properly designed Decision Tables. It is a fact that fuzzy sets μ 1 (y), μ 2 (y),…. μ pk (y),…. μ o (y) represent the actual assignment of linguistic variables values to set Y. Let [x 1 ,x 2 ,…x i ,…x m ] be the input variables vector defined on the reference sets X 1 ,X 2 ,….,X i ,….X m and let y be an input variable defined on the reference set Y. In this case the FIS has m input variables and one output and each set X i (i=1,2,..,m) can be divided in p j (where j=1,2,…,n) fuzzy sets.

In a Mamdani FIS the fuzzy rules can be written as follows: IF x 1 is A 1 and x 2 is A 2 AND …..AND x i is A pj AND…AND x m is A n THEN y is B (where i=1,2,….,m j=1,2,….n) The fuzzy rules that were actually applied had the following form:

IF (H is LOW) and (N is HIGH) and (J is MIDDLE) and (C is LOW) THEN (DRI is HIGH).

The developed FIS outputs a matrix that contains pure numbers in the closed interval [0, 1] that express the degree of membership of each area to the fuzzy set "Area of high Drought".

Of course this can be considered as the matrix of the drought risk indices. Each single index is produced after the consideration of all involved meteorological parameters for each area of study and for a specific temporal value. Due to the fact that the obtained indices matrix has quite low values that are not easy to follow and to use, the risk indices were multiplied by 100.

Average Monthly Maximum Heat Index

The determination of the average maximum Heat Index (HI) value for each area was done by using the respective average monthly maximum temperature and relative humidity values from all of the meteorological stations used. For this reason the temperatures were converted from Celsius °C to Fahrenheit °F. The most widely used deterministic approach was employed for the estimation of the HI, namely the methodology proposed by the Unite States meteorological service which can be seen in the following function 4 The obtained values were manipulated by a FIS to determine the risk due HI.

Using Geographic Information Systems ArcGIS

Geographic Information Systems as dynamic tools of spatial data analysis and manipulation can be combined with fuzzy systems. They can join their forces in order to reach decisions in complex non linear and non deterministic problems such as risk modeling.

Defining Zone Areas Using GIS

Using ArcGIS (version 9.3) a spatial database has been developed towards the analysis and spatial representation of the most risky areas in terms of drought and according to the estimated average heat index. Also the cases were studied based on their average burned area during the last fifteen years (1994)(1995)(1996)(1997)(1998)(1999)(2000)(2001)(2002)(2003)(2004)(2005)(2006)(2007)(2008)(2009). A shape ARCGIS file (.shp) of Cyrpus was employed and the free part of Cyprus (as a polygon) was divided from the occupied one, by using the cut-polygons approach. Before this division our research team performed "georeport" of the scanned map. This was done because scanned maps do not contain any piece of real coordinate data. Thus in all cases a spatial report had to be performed to the raster data in order to define the application of the data to the coordinates of the map. In other words the geo-report offered real world coordinates to each part of the map. The fact that the actual coordinates of each point of each station were known enabled the division of the limits of each meteorological station. In this way several polygons were formed in the free part of the island. This task was performed by using the ArcGIS Spatial Analyst which is an actual Extension of the System. Spatial Analyst enables the construction, analysis and mapping of raster data and the performance of an integrated raster-vector analysis [START_REF]ArcGIS Desktop 9.3 Help 2009 Spatial Analysts[END_REF][START_REF] Hillier | Working with ArcView 9[END_REF]. More specifically the Path Distance Allocation methodology was employed [START_REF]ArcGIS Desktop 9.2 Help 2008 Path Distance Allocation[END_REF]. This approach recognizes for each cell the zone of the source cell which may be approached with the minimum cost. More specifically, it calculates the nearest source for each cell based on the least accumulative cost over a cost surface, while accounting for surface distance and horizontal and vertical cost factors. Finally a cut of the raster files was performed (cut polygons) and the new data were stored to correspond to the new polygons-regions.

Results

Results related to Drought Indices

Figure 4 shows that the area corresponding to the meteorological station of "Athalassa" is assigned with the highest and the most stable values of the Average Maximum Drought Index (AMDI) for the period 1979-2009.

For the other areas the AMDI does not have a stable rate and it has the highest peaks during the decade 1999-2009. The highest AMDI value ever, has been recorded in the "Athalassa" meteorological station in 2008 and it is equal to 86.62706905. It should be clarified that though the obtained AMDI indices are fuzzy numbers and thus they take values in the closed interval [0, 1] they have been multiplied by 100 in order to be more easily understood. The following graph (Figure 6), the area of "Athalassa" has also the highest AAVDI for the period 1979-2009. This area includes the central part of the country where the capital Nicosia is located and where is the residence and the working place of the majority of the population. This area has the highest CO 2 emissions by vehicles and the smallest forest cover as a percentage of its surface. However the most significant result is that the AAVDI values are constantly increasing for all of the areas under study during the last decade. Even in areas such as "Prodromos" and "Platania" with the lowest AAVDI values and with the 2/3 of the island's forested area, the problem of drought has been increasing constantly during the last decade, without any significant decrease as it used to happen during the previous two decades. From table 1 it is concluded that for the areas of "Prodromos" "Polis Chrysochous" and "Platania" which are located mainly in the center of the island the most risky month due to drought is July followed by August whereas for "Larnaka", "Athalassa", "Pafos", "Lemesos" and "Paralimni" the worst month is August followed by July. The most important result is the fact that the area of "Athalassa" which is far from the seaside has the highest average drought risk value equal to 86.6270 during August. The areas of "Prodromos" "Platania" and "Larnaka" appear 31 times to have the highest values of the average maximum drought risk index whereas "Paralimni" 26 times, "Athalassa" and "Pafos" 27 "Lemesos" 17 and "Polis Crysochous" 10. This means that though the area of "Athalassa" has the highest average values of drought risk index, the areas of "Prodromos", "Platania" and "Larnaka" have more often extreme maximum values of drought.

Results related to the Heat Index

From the following graph (Figure 7) it is concluded that the area of "Athalassa" has the highest values of average maximum heat index for the period 1979-2009. This is probably due to the fact that the area of "Athalassa" is located in the central part of the country and it contains major urban centers like the capital of Cyprus "Nicosia" where the biggest part of the country's population lives. This area has the highest levels of air pollutants due to high traffic and the smallest ratio of forested area to the total surface. The highest value was recorded in this area during August 1998 and July 2000. In all of the other areas, the average maximum heat index risk appears to be stable for the period 1979-2009. From the following graph (Figure 8) it is concluded that the most risky area regarding the average heat index for the period 1979-2009 is "Athalassa". This is easy to be explained if we consider that this area has the highest Average Actual Values of Drought Indices and it is also mainly an urban area densely inhabited with the smallest proportion of forest cover compared to the rest of the island. The most important obtained result is that during the last five years the value of the AAVDI increases constantly even in areas with the smallest heat index values. It is impressive that the areas with the smallest average heat index values namely "Prodromos" and "Platania" which have a very high proportion of forest cover (they contain almost 66% of the whole forested area of Cyprus) appear to have a continuously increasing heat index values in the last five years.

From the following table 1 it can be seen that the most risky month regarding the heat index for the areas of "Prodromos" "Polis Chrysochous" and "Platania" is July followed by August whereas for "Larnaka" "Paralimni", "Pafos" and "Lemesos" the most risky month is August followed by July. For "Athalassa" the two months are equally risky regarding the heat index. Again the area of "Athalasa" has the highest heat index values which have been recorded in July and in August. The areas of "Prodromos" "Larnaka" and "Platania" have the highest average heat index risk values 31 times, "Paralimni" 26 times, "Athalassa" and "Pafos" 27, "Lemesos" 17 and "Polis Chrysochous" 10 times. The area of "Pafos" has the least number of forest fire breakouts (only 21 cases). "Athalassa" appears to be significantly risky with a high number of wild fires although only a small percentage of this area is forested. This finding is very important and it shows that the urbanization and land value together with the high heat index influence the problem significantly. The influence of the human factor is obvious [28].

Totally 917 wild fires incidents appear in Figure 10, with 211 incidents in "Platania", 158 in "Prodromos" 140 in "Polis Chrysochous", 131 in "Lemesos" and 123 in "Athalassa". The area of "Pafos" has the least number of forest fire breakouts (only 21 cases). "Athalassa" appears to be significantly risky with a high number of wild fires although only a small percentage of this area is forested.

Discussion

For the first time Cyprus was divided in eight polygons based on the Path Distance Allocation approach. This enabled the more comprehensive and reliable analysis of each area that demonstrated different and distinct special characteristics and properties. These areas were ranked based of the estimated DRI and on their HI values. It is worth mentioning that the DRI was estimated based on the equal contribution of several meteorological features something that offers a high degree of innovation to this effort. It is obvious that in Cyprus the climate conditions are changing to the worse especially during the last decade and this has an influence to the burning of forest ecosystems and to the obtained heat indices vectors that are continuously growing to higher maximum and average values A very interesting result of this research is related to the urban area of "Athalassa" where the capital Nicosia is located. Though this region has a small ratio of forest cover (forested area divided to its area) it has a very high number of wild fire incidents something that was not expected initially. This shows the influence of the drought increase and perhaps the influence of the urbanization and land value to the problem. Another crucial point is the conclusion that the DRI and the heat index are both increasing significantly during the last decade even in the areas which are located near the cost yard. This should alarm the authorities towards the design of a more effective planning. This research will continue in the future with more sophisticated Soft Computing techniques and of course with more available data.

Fig. 1 .

 1 Fig. 1. Structure of the fuzzy inference system

Fig. 2 .

 2 Fig. 2. Design of the fuzzy Ruleset in MATLAB

  [23]. T HI = 16,923 + 1,85212*10 -1 *T + 5,37941*RH -1,00254*10 -1 *T*RΗ + 9,41695* 10 -3 *T 2 + 7,28898*10 -3 *RH 2 + 3,45372*10 -4 *T 2 *RH -8,14971*10 -4 *T*RH 2 + 1,02102*10 -5 *T 2 *RH 2 -3,8646*10 -5 *T -3 + 2,91583*10 -5 *RH 3 + 1,42721*10 -6 *T 3 * RH + 1,97483*10 -7 *Τ*RH 3 -2,18429*10 -8 *RH 2 + 8,43296*10 -10 *T 2 *RH 3 -4,81975*10 -11 *Τ 3 *RH 3 (4)

Fig. 3 .

 3 Fig. 3. Division in polygons based on the Path Distance Allocation method

Fig. 4 .

 4 Fig. 4. Average Maximum Drought risk indices 1979-2009The following maps (Figure5a, 5b) are indicative graphical displays of the AMDI and of the Average Actual Values of Drought Indices (AAVDI) for the specific year 2006 and they confirm that the driest area is the one of "Athalassa".

Fig. 5a .

 5a Fig. 5a. (AMDI) for 2006 Fig. 5b. (AAVDI) for 2006

Fig. 6 .

 6 Fig. 6. AAVDI risk indices values for the period 1979-2009

Fig. 7 .

 7 Fig. 7. Mean Maximum Heat Index 1979-2009

Fig. 8 .

 8 Fig. 8. Average heat index values for the period 1979-2009

4. 3

 3 Results related to Forest Fire RiskA few reports are available in the literature on the forest fire problem of Cyprus[START_REF][END_REF]. The problem has not been analyzed by the use of spatial decision support systems like ARCGIS or Intelligent Systems. From Figure9it is concluded that the majority of forest fire incidents were recorded during the last decade whereas the year 2008 was the worse so far. It should be mentioned that year 2008 had the highest average maximum drought risk index for the period 1979-2009. Obviously these two findings are related. Totally from the 1226 forest fires of the period 1979-2009 the 804 were recorded during the last decade 1999-2009.

Fig. 9 .

 9 Fig. 9. Forest fire frequencies of Cyprus (1979-2009)

Fig. 10 .

 10 Fig. 10. Number of forest fires for each area of study This finding is very important and it shows that the urbanization and land value together with the high heat index influence the problem significantly. The influence of the human factor is obvious [28].

Table 1 .

 1 Average Maximum Heat Index and AMDI X100.

			Frequency		Frequency
			of inclusion	Maximum	of inclusion	Average
			in highest	Average	in highest	Maximum
	Area	Month	DRI	DRI	HI risk	HI risk
	Prodromos	July August	61,29% 38,71%	682,651 657,319	64,52% 35,48	26,7 25,4
	Platania	July August	64,52% 35,48%	658,913 673,197	64,52% 35,48	27,8 27,3
	Larnaka	August July	61,29% 38,71%	789,141 775,406	64,52% 35,48	29,8 29,6
	Paralimni	August July	61,29% 38,71%	810,530 829,690	69,23% 30,72	30,3 30,1
	Athalassa	August July	66,67% 29,63%	866,270 855,613	48,15 51,85%	34 34
	Pafos	August	77,78%	643,002	77,78%	27,1
	Lemesos	August	76,47%	723,669	70,59%	29
	PolisChrisochous July	100%	770,234	100%	29