Maytham Alabbas
email: alabbasm@cs.man.ac.uk

Allan Ramsay
email: ramsay@cs.man.ac.uk

Improved POS-tagging for Arabic by combining diverse taggers

Keywords:

A number of POS-taggers for Arabic have been presented in the literature. These taggers are not in general 100% accurate, and any errors in tagging are likely to lead to errors in the next step of natural language processing. The current work shows an investigation of how the best taggers available today can be improved by combining them. Experimental results show that a very simple approach to combining taggers can lead to significant improvements over the best individual tagger.

Introduction

The process of assigning a correct POS tag (i.e. noun, verb, adverb or others) to each word of a sentence is called part-of-speech(POS) tagging. This process is considered an essential step for most natural language applications. In general, however, POS-taggers make mistakes, and since tagging is the first step in most natural language processing (NLP) systems these mistakes will lead to problems in all subsequent stages of analysis. It is thus important to obtain the highest possible accuracy at this initial stage of processing. One popular technique for improving tagging accuracy is tagger combination. This approach involves combining different taggers to exploit the unique properties of each tagger and reduce some of the random errors. This technique has been applied for different languages such as English [START_REF] Brill | Classifier combination for improved lexical disambiguation[END_REF], Swedish [START_REF] Sjöbergh | Combining pos-taggers for improved accuracy on swedish text[END_REF],Telugu [START_REF] Ramasree | Combining pos taggers for improved accuracy to create telugu annotated texts for information retrieval[END_REF], Italian [START_REF] Søgaard | Ensemble-based pos tagging of italian[END_REF] and Polish [START_REF] Śniatowski | Combining polish morphosyntactic taggers[END_REF] and the results were encouraging, but has not to our knowledge been applied for Arabic. The current work is step forward in this regard. We evaluate different techniques for combining POS-taggers for Arabic.

The key problem for Arabic is that it is more ambiguous than many other languages (e.g. English) for many reasons. Firstly, Arabic is written without diacritics (short vowels and a range of other phonological effects), often leading to multiple ambiguities [START_REF] Nelken | Arabic diacritization using weighted finite-state transducers[END_REF]. This is particularly problematic because the diacritics are often the only difference between different words (especially derived forms) and between inflected forms of the same word. This matter makes analysis of the language morphologically very challenging. This is because a certain lemma (or lexeme) in Arabic can be interpreted in various ways. Hence, a single word can have various senses, where determining the sense is based on the context in which the word is used. Furthermore, a noun in Arabic can be diacritised in three different ways for the nominative, accusative and genitive cases, which can be even more ambiguous at the structural or grammatical level.

In addition, Arabic is highly syntactically flexible [START_REF] Daimi | Identifying syntactic ambiguities in single-parse arabic sentence[END_REF]:

-It has a comparatively free word order, where sentence components can be exchanged without affecting the core meaning. This results in structural ambiguity, with each morphological analysis having more than a single meaning. So, besides the regular sentence of verb, subject and object (VSO), Arabic allows other potential surface forms such as SVO and VOS constructions.

The potential of allowing variations on the canonical order leads to a large amount of ambiguity. -Furthermore, Arabic (like Spanish, Italian and Japanese) is a pro-drop language [START_REF] Farghaly | Subject pronoun deletion rule[END_REF]. The pro-drop can lead to structural ambiguity by leaving any syntactic parser with the challenge to determine if there is a dropped pronoun or not in the subject position, which is made worse by the fact that lots of Arabic verbs can have both transitive and intransitive forms, and further that it is generally impossible to tell the difference between active and passive forms by inspecting the surface form. In case that just one noun phrase (NP) follows one of these verbs, the ambiguity appears. In contrast with English because its canonical order is SVO, losing the subject (S) does not cause confusion about the object (O). -In addition the copula is omitted in simple positive equational sentences (the sentences that did not contain verb), so that a sequence of a noun and a predicative item (i.e. another noun, an adjective or a prepositional phrase (PP)) may make a sentence. -Finally, Arabic nouns can be linked together without any overt marker, whereas two English nouns are joined together by different markers, such as the suffix "-'s" on possessing noun, a possessive phrase "of" or a pronoun as "his/her". In the construct phrase the first noun must be indefinite solely (which can be in any case: nominative, genitive or accusative). The second noun may be either definite or indefinite (which is always in genitive case).

While these are strictly speaking syntactic issues, they have serious consequences for tagging, since they mean that the local context often fails to supply constraints on tags-to take a simple example, if you do not know whether the item following a verb is its subject or its object then you cannot use the proximity of the verb to determine the case marker.

The taggers

We are interested in improving tagging by combine different taggers. We have carried out a number of experiments with state-of-the-art taggers (AMIRA 2.0 [START_REF] Diab | Second Generation Tools (AMIRA 2.0): Fast and Robust Tokenization, POS Tagging, and Base Phrase Chunking[END_REF], MADA 3.1 [START_REF] Habash | Introduction to Arabic Natural Language Processing[END_REF] and a home-grown tagger, MXL [START_REF] Ramsay | Bootstrapping a lexicon-free tagger for arabic[END_REF], with comparable accuracy), using the Penn Arabic Treebank (PATB) [START_REF] Maamouri | Developing an Arabic treebank: Methods, guidelines, procedures, and tools[END_REF] as a gold-standard corpus.

Gold-standard tags

We used PATB Part 1 v3.0 as a resource for our experiments. The words in the PATB are already tagged. This provides us with a benchmark for tagger evaluation. Even PATB tagging is not guaranteed to be 100% accurate, but it nonetheless provides as good a reference set as can be found.

The PATB uses a very fine-grained set of tags, which carry a great deal of syntactically relevant information (particularly case-marking). This tagset contains 305 tags, with for instance 47 tags for different kinds of verb and 44 for different kinds of noun. We carried out our experiments with a variant of this original tagset, and also with a coarser-grained version obtained by deleting details such as case-and agreement-markers.

AMIRA

The first tagger we use is AMIRA. This tagger is reported to achieve around 97% accuracy on its target tagset, which is about as good as any reported system.

Using AMIRA, however, highlights one of the problems that arise when we try to compare its retagged corpus with other corpora. The PATB is tagged, but with different tags from the ones used by AMIRA. 1 In order to compare AMIRA tags with other corpora tags, we will have to translate between the two tagsets. This is a non-trivial task. The two tagsets have different numbers of tags (e.g. AMIRA has 130 fine-grained tags compared with 305 in PATB), and more importantly they make different kinds of distinctions. The AMIRA tagset, for instance, uses one tag (RP) to cover a range of particles which are subdivided into eight subclasses (EMPH PART, EXCEPT PART, FOCUS PART, INTERROG PART, RC PART, NEG PART, PART, VERB PART) in the PATB; and it uses several tags to describe different kinds of verbs (VB, VBG, VBD, VBN, VBP) where the PATB just uses three (IV, PV, CV).

In order to overcome these problems, we use transformation-based retagging (TBR) [START_REF] Brill | Transformation-based error-driven learning and natural language processing: a case study in part of speech tagging[END_REF][START_REF] Lager | µ-tbl lite: a small, extendible transformation-based learner[END_REF] to recover from the mismatches between the two tagsets. TBR collects statistics about the local context in which erroneous tags have been assigned, and attempts to find rules based on this information to apply after the original tagger has been run. This technique will produce a small improvement in the performance of almost any tagger. Typically, taggers that achieve scores in the mid 90s are boosted by 2-3%-the lower the original accuracy, the greater the typical improvement. When we used it for comparing the original tags produced by AMIRA and the gold tags the score improved from around 89% to just over 95%. Some of this improvement arises simply from rules that spot that the two tagsets use different names for the same things (e.g. that things that are called JJ are called ADJ in the PATB) but some of it comes from learning how to split coarse-grained AMIRA tags into fine-grained PATB ones.

There is a further problem with using AMIRA. The fact that Arabic allows a range of items to be cliticised (conjunctions, prepositions, pronouns) makes it difficult even to tokenise text reliably. This means that not only does AMIRA sometimes assign different tags from the PATB, it sometimes even splits the text into different numbers of tokens (i.e. AMIRA's tokeniser segments the text differently from the way that it is segmented in the PATB-AMIRA's tokeniser gives us around 97% agreement with gold-standard tokenisation).

In order to see whether this was the cause of the difference, we constructed a version of the corpus where we replaced PATB tags by a coarse copy of the AMIRA tags, using hand-coded substitution rules, and then replaced these by fine-grained AMIRA tags where the substituted tags were compatible with tags actually assigned by AMIRA. Thus if the PATB assigned a word the tag ADJ we replaced it by the AMIRA tag JJ. We then inspected the tags assigned by AMIRA itself: if the tag assigned to this word was one of AMIRA's fine-grained adjective tags, e.g. JJR, then we used this instead. If, on the other hand, the handcoded replacement for the PATB tag was incompatible with the one assigned by AMIRA then we retained the hand-coded one. This gave us a version of the treebank that had the same number of tokens as the original PATB, with as many items as possible given the tags assigned by AMIRA and the others given hand-coded AMIRA equivalents of the original PATB tags.

MADA

MADA [START_REF] Habash | MADA+TOKAN: A Toolkit for Arabic Tokenization, Diacritization, Morphological Disambiguation, POS Tagging, Stemming and Lemmatization[END_REF] uses a slightly extended version of the PATB Part 1 v3.0 tagset, with some extra classification of nouns. The fine-grained MADA retagged version contains 352 tags compared to 305 tags in the PATB corpus. Fortunately the MADA tags are a strict superset of the standard PATB set, and hence can be reduced to either the standard fine-grained version or our coarse version by omitting the extra classification of nouns, so we do not have the same problems using MADA with the PATB as we have with AMIRA.

We also applied TBR to the output of MADA, because although we were not faced with mapping incompatible tagsets in the same way as with AMIRA, using TBR nearly always provides a small improvement, amounting in this case to an increase from 94% to 96.7%. We applied the same technique that we used in AMIRA to get a MADA version of the corpus that has the same size as the gold-standard one because MADA's tokeniser also has small (1%) variance from the gold-standard.

Maximum-likelihood tagger

We also use an in-house maximum-likelihood tagger, which we will refer to as MXL. We have described this tagger in detail in [START_REF] Ramsay | Bootstrapping a lexicon-free tagger for arabic[END_REF]: we will simply outline the basic principles that it is based on and note its accuracy here.

MXL operates in two stages, as follows:

-In the first stage we use two simple kinds of statistic: (i) the conditional likelihood that a word which starts with the same three letters or ends with the same three letters as the one we are trying to tag has a given tag, and (ii) the transition probabilities between tags. We use a weighted combination of these to produce a maximum-likelihood guess at the current tag. This process produces about 95.2% accuracy. -We then use TBR to refine the original set of hypotheses, leading to a final accuracy of 95.6%.

The advantage of this tagger is that because it was trained on the PATB, the tags it uses are exactly the PATB tags and the tokenisation is exactly the PATB tokenisation. We therefore do not need to overcome problems associated with mismatches between tag sets.

Improving POS-tagging accuracy

In the current work, we are interested in both fine-grained and coarse-grained tagsets. We therefore collapsed the original fine-grained set, by deleting inflectional markers (case, number, gender), to the coarse-grained set show in 2 summarises the fine-grained and coarse-grained tags for PATB corpus and the three retagged corpora by the taggers: AMIRA, MADA and MXL. Table 3 summarises the accuracy of the three taggers on our gold standard set using the built-in tagsets and coarse versions of each, and shows the improvements that are obtained in each case by applying TBR. This provides a reference point: the best of the three taggers is MADA, which achieves 96.7% on the coarsegrained version of its built-in tagset if we also apply TBR and 93.6% on the fine-grained version, again after applying TBR. The goal of the current paper is to see whether we can improve on this by utilising the other taggers, despite the fact that their individual performance is worse. The first observation is that when the taggers agree on how to tag a given item they are more likely to be right than when they disagree. This is fairly obvious-if you have a set of taggers which assign different tags to an item then at least one of them must be wrong! Table 4 substantiates this observation-each column shows the precision (P) and recall (R) for a particular pair of taggers simply taking cases where they agree and leaving words on which they disagree untagged. Thus the combination of MXL and MADA achieves a precision of 99.5% on the coarsegrained tagset and 99% on the fine-grained one. Table 5 shows what happens when we combine all three taggers, either taking the majority view when at least two of them agree or demanding that all three agree. In the latter case we obtain a precision of 99.9% for the coarse-grained tagset and 99.2% for the fine-grained one. The cost, of course, is that the recall goes down, because there are places where they disagree, and in these cases no tag is assigned. What should we do in such cases?

POS

Metrics

Backoff strategies

Our first proposal was to take the majority view when at least two of the taggers agreed, and to backoff to one or other when they there was no common view [START_REF] Zeman | Improving parsing accuracy by combining diverse dependency parsers[END_REF]. The results of this are shown in Table 6 where column one shows the result of backing off to MXL when there is no majority view and column two shows the result of backing off to MADA. The results for the coarse-grained tagset are markedly better than for any of the taggers individually, though there is only a very slight improvement over the original MADA scores for the fine-grained set. Interestingly, the best results are obtained by backing off to MXL rather than to MADA (98.2% vs. 97.9%, despite the fact that MADAs individual performance is better than MADA's (96.7% vs. 95.6%) (the results in this table and in Table 8 are for accuracy rather than precision or recall). The fact that backing-off to either MADA or MXL improved the performance suggested that it was worth investigating other backoff strategies. If majority vote + backoff to an arbitrary tagger is better than any single tagger in isolation, then perhaps there is something we can backoff to which will do even better.

We return to the observation that in cases where all three taggers disagree, at most one of them can be right. Given that they each employ different information about the material being tagged, it is likely that they are systematically prone to different kinds of errors.

We therefore collected statistics about the kinds of things they each get right. How likely, for instance, is MXL to be right when it assigns the tag NEG PART? Table 7 shows an extract of this data, showing the likelihood that each tagger is right for a given assigned tag, e.g. for the given instance of the word gyr2 "other than" the correct tag was NEG PART (as in gold standard): MADA suggested NOUN, MXL suggested NEG PART, AMIRA suggested RP. Because MXL is right 98.2% of the time when it suggests NEG PART, whereas MADA is right 97.9% of the time when it suggests NOUN and AMIRA is right only 8.1% of the time when it suggests RP, NEG PART was chosen. For <lA "to", MADA's suggestion of EXCEPT PART was accepted because MADA is right 100% of the time when it suggests EXCEPT PART, which is better than the reliability of either of the other suggestions. Using this strategy for choosing what to do when all three taggers make different suggestions produces the results in Table 8. 3 The 'default' column reports the results when we simply chose the most confident proposal, whereas for 'backoff unless two agree' we took the majority verdict if two of the taggers agreed and the most confident if all three gave different results. The results in Table 8 are both surprising and compelling. Simply taking the most confident of the three taggers produces 99.5% accuracy for the coarsegrained set, which is going to be hard to beat by much, and even for the finegrained set it produces 96%. This improves over taking the majority verdict when at least two of the contributing taggers agree and backing off to the most confident one where there is no agreement,4 and it beats each of the individual taggers by a fairly wide margin-the original error by MADA of 3.3% for the coarse-grained set has been reduced to 0.5%, a nearly sevenfold reduction.

Conclusions

We have shown a rather simple mechanism for combining taggers which can provide considerable improvements in accuracy. If you measure the likelihood that a tagger is right when it suggests a particular tag, and then take the suggestion with highest score in each case then you can decrease the error rate substantially.

The key is that the different taggers tend to make different systematic mistakes. The accuracy statistics capture these systematic mistakes, so a low score is likely to reflect a case where the tagger is making one of its characteristic errors, and in such cases we take the output of one of the others. This simple approach outperforms strategies involving more subtle ways of combining the individual taggers, e.g. by taking the majority preference in cases where one exists. This is likely to be because two of the taggers (MADA and AMIRA) use very similar information, and hence where they make systematic mistakes they are likely to make the same systematic mistakes. In such cases they will tend to agree, and hence would outvote MXL if allowed the majority view to win, as suggested by the precision results in Table 5. The simple mechanism we have used provides a built-in resilience against this tendency.

In every case, including TBR makes a useful contribution. Again, TBR is most effective when the base tagger makes systematic errors. The version of TBR that we are using includes extra templates looking at the first three and last three letters of words in addition to the standard word-based templates. These extra templates pay attention to prefixes and suffixes, which carry much more information than is the case for English (where TBR has been most extensively applied).

The bottom line is that for the tagset in Table 1 we can obtain 99.5% accuracy when tagging freely occurring Arabic. It is going to be hard to improve substantially on this score. Given that Marton et al. [START_REF] Marton | Improving arabic dependency parsing with lexical and inflectional morphological features[END_REF] have argued that coarsegrained tagsets are actually more useful than fine-grained ones for parsing, which is the usual next step in the chain, we are fairly satisfied with this result.

Table 1 (

 1 e.g. PATB has 305 fine-grained tags which become 39 coarse-grained tags, for instance, the fine-grained tags NOUN+CASE DEF ACC, NOUN+CASE DEF GEN, NOUN+CASE DEF GEN+POSS PRON 3MS, NOUN+CASE DEF NOM are grouped to NOUN).

	ABBREV	DET+NOUN PROP	LATIN	PUNC
	ADJ	DET+NUM	NEG PART	PV
	ADV	EMPH PART	NOUN	PVSUFF DO
	CONJ	EXCEPT PART	NOUN PROP	RC PART
	CV	FOCUS PART	NO FUNC	REL ADV
	CVSUFF DO	FUT+IV	NUM	REL PRON
	DEM PRON	INTERJ	PART	SUB
	DET	INTERROG PART	POSS PRON	SUB CONJ
	DET+ADJ	IV	PREP	VERB PART
	DET+NOUN	IVSUFF DO	PRON

Table 1 .

 1 Coarse-grained tagset

	Table

Table 2 .

 2 Coarse-grained and Fine-grained tags numbers, single tagger

		PATB/MXL AMIRA MADA
	Coarse-grained	39	29	56
	Fine-grained	305	130	351
	POS	TBR AMIRA MXL MADA
	Coarse-grained	× 89.6% 95.2% 94.1% √ 95.3% 95.6% 96.7%
	Fine-grained	× 84.3% 89.7% 91.7% √ 88.8% 91.2% 93.6%

Table 3 .

 3 Tagger accuracies in isolation, with and without TBR

Table 4 .

 4 Precision

		POS	TBR MXL-AMIRA MXL-MADA MADA-AMIRA
	P	Coarse-grained Fine-grained	× √ × √	97.7% 98.8% 92.9% 94.3%	99.3% 99.5% 98.9% 99%	96.3% 99.5% 94.7% 95.8%
	R	Coarse-grained Fine-grained	× √ × √	84.3% 94.1% 79.8% 81.6%	90.1% 94.6% 83.3% 86.4%	86.1% 91.9% 84.9% 85.6%
	F-SCORE	Coarse-grained Fine-grained	× √ × √	0.905 0.964 0.858 0.875	0.945 0.97 0.904 0.923	0.909 0.955 0.895 0.904
		(P) and Recall (R) and F-score for combinations of pairs of taggers,
	with and without TBR

Table 5 .

 5 Precision (P) and Recall (R) and F-score for combinations of three taggers, with and without TBR

Table 6 .

 6 Backoff to MXL or MADA when there is no agreement

	POS	TBR Otherwise, MXL tag Otherwise, MADA tag
	Coarse-grained	× √	94.9% 98.2%	93.8% 97.9%
	Fine-grained	× √	89.6% 91.8%	88.8% 91.5%

Table 7 .

 7 Confidence levels for individual tags

	Word Gold standard MADA	MXL	AMIRA TAG
	gyr NEG PART	NOUN (97.9)	NEG PART (98.2) RP (8.1) NEG PART
	<lA EXCEPT PART EXCEPT PART (100.0) SUB CONJ (96.5) RP (7.9) EXCEPT PART
	. . .	. . .	. . .	. . .	. . .	. . .

Table 8 .

 8 Backoff to most confident tagger

	POS	TBR default backoff unless two agree
	Coarse-grained	× 97.3% √ 99.5%	95.7% 99.2%
	Fine-grained	× 95.6% √ 96.0%	93.2% 94.5%

We used the Extended Reduced Tag Set (ERTS) setting for AMIRA and then removed inflectional markers. This produced a set of tags that is very similar to the

25 tags in the Bies/RTS tagset, but with distinctions between nouns, adjectives and cardinal numbers.

The transcription of Arabic examples follows Buckwalter's system for transcribing Arabic symbols. Available at: http://www.qamus.org/transliteration.htm.

Note that taking the majority verdict when all three agree and backing off to the most confident when there is not complete unanimity is exactly the same as simply taking the most confident one from the outset.

Acknowledgments. We would like to thank Dr. Yasser Sabtan (Al-Azhar University, Eygpt) for important suggestions and for helpful discussions. Maytham Alabbas owes his deepest gratitude to Iraqi Ministry of Higher Education and Scientific Research for financial support in his PhD study. Allan Ramsay's contribution to this work was supported in part by Qatar National Research Foundation grant NPRP 09 -046 -6 -001.


Constantinos Stylianou
email: cstylianou@cs.ucy.ac.cy

Andreas S Andreou
email: andreas.andreou@cut.ac.cy

A Multi-Objective Genetic Algorithm for Software Development Team Staffing Based on Personality Types

Keywords: Software Project Management, Team Staffing, Genetic Algorithms Personality Type Matching, Five-Factor Model

This paper proposes a multi-objective genetic algorithm for software project team staffing that focuses on optimizing human resource usage based on technical skills and personality traits of software developers. Human factors are recognized as critical aspects affecting the rate of success of software projects, as well as other properties, such as productivity, software quality, performance, and job satisfaction. However, managers often rely solely on technical criteria to staff their projects, which risks overlooking these important aspects of software development, such as the abilities and work styles of developers. The behaviour and scalability of the algorithm was validated against a series of hypothetical projects of varying size and complexity, and also through a real-world project of an SME in the local IT industry. The approach demonstrated a sufficient ability to generate both feasible and optimal staffing solutions by assigning developers most technically competent and suited personality-wise for each project task.

Introduction

For many years, researchers in the area of software engineering have argued that human factors should be taken into consideration when developing software since human resources are the most, if not only, crucial resource available for software development companies. Hence, most research targets software project managers since it is their responsibility to assign these resources to tasks and their selection directly influences the success of a software project, especially with respect to critical software development issues such as performance, productivity, quality, and job satisfaction. Assigning software developers to tasks is not a simple process as each developer has their own strengths and weaknesses, which often extend beyond the "academic" knowledge acquired from a university degree, for example, or the experience gained from using a specific tool or technology over a number of years. They also concern traits and behaviours in the form of abilities and competencies that develop from each individual's personality and psychological processes. For this reason, it is argued that software project managers need to look beyond technical skill-based and experiencebased methods when selecting developers, as it is equally important to deal with interpersonal relationships and social aspects present in software development processes and organizations [START_REF] Amrit | Coordination in Software Development: The Problem of Task Allocation[END_REF]. The goal, therefore, of the proposed approach is to support software project managers of SMEs in the selection and allocation of the most suitable developers to tasks, by attempting to optimize the assignments based on technical skills and personality traits of developers using a multi-objective genetic algorithm.

The remainder of this paper is organized as follows: Section 2 provides a brief overview of the most recent research attempts proposed regarding the inclusion of human factors in software project team staffing. Section 3 describes the method used to identify the various occupations found in the software development industry together with their characteristics and requirements in terms of technical skills and personality traits. In section 4, a description of the methodology is provided, in which various aspects of the multi-objective optimization approach are described. Next, section 5 presents the evaluation of the proposed approach, describing the projects used in the experiments carried out followed by a discussion on the results obtained. Finally, section 6 concludes the paper with a synopsis and comments on future work.

Related Work

One of the most common human factors affecting software development addressed in literature is the area of personality, with many different investigations carried out over the years. Some attempts have looked into studying the various types of personality of software development professionals. The results from such studies can provide helpful insight about the type of personalities attracted to the software development industry, and also can be used by companies looking to recruit or release personnel, or by software project managers attempting to assign developers to tasks. Some of the questions asked in these studies focus on whether software professionals share a common personality type [START_REF] Moore | Personality Characteristics of Information Systems Professionals[END_REF] and whether they differ from the rest of the general population [START_REF] Wynekoop | Revisiting the Perennial Question: Are IS People Different?[END_REF]. Some studies concentrate on specific professions, such as systems analysts [START_REF] Smith | The Personality of the Systems Analysts: An Investigation[END_REF], while others examine occupations from all development phases [START_REF] Capretz | Making Sense of Software Development and Personality Types[END_REF]. More recently, Varona, et al. carried out a survey of existing studies that attempted to profile ITrelated professions in order to identify trends and changes, and to form a better understanding of the software industry's human resources [START_REF] Varona | Evolution of Software Engineers' Personality Profile[END_REF]. Another area of research has focused on assessing the effects of personality on various properties of software development. For instance, personality has been explored with respect to team effectiveness and performance [START_REF] Peeters | Personality and Team Performance: A Meta-Analysis[END_REF][START_REF] Capretz | Why do we Need Personality Diversity in Software Engineering?[END_REF], cohesion [START_REF] Karn | A Study into the Effects of Personality Type and Methodology on Cohesion in Software Engineering Teams[END_REF], software quality [START_REF] Fernández-Sanz | Influence of Human Factors in Software Quality and Productivity[END_REF], as well as job satisfaction [START_REF] Acuña | How do Personality, Team Processes and Task Characteristics Relate to Job Satisfaction and Software Quality?[END_REF]. More recent research has also concentrated on how personality influences pair programming in agile methodologies [START_REF] Sfetsos | An Experimental Investigation of Personality Types Impact on Pair Effectiveness in Pair Programming[END_REF][START_REF] Salleh | An Empirical Study of the Effects of Conscientiousness in Pair Programming using the Five-Factor Personality Model[END_REF].

Research work relating to team staffing and formation have also been carried out, whereby attempts to build teams for software projects take into account human factors such as developers' personality types [START_REF] Rutherfoord | Using Personality Inventories to Help Form Teams for Software Engineering Class Projects[END_REF] and capabilities [START_REF] Acuña | Assigning People to Roles in Software Projects[END_REF]. Some of the team staffing approaches proposed employ computational intelligence techniques to aid with assigning developers to tasks. For instance, Martínez et al. [START_REF] Martínez | Big Five Patterns for Software Engineering Roles using an ANFIS Learning Approach with RAMSET[END_REF] proposed RAMSET as a methodology for assigning roles in software engineering teams that adopted a learning approach based on an adaptive network-based fuzzy inference system to recommend the best resource allocation possible. Also, André, Baldoquín and Acuña [START_REF] André | Formal Model for Assigning Human Resources to Teams in Software Projects[END_REF] formalized a set of rules to match the personality types of developers to fixed project roles defined by a set of generic and technical competencies. They then transformed these rules into objective functions and constraints, and applied them in various heuristic algorithms (such as, random restart hill-climbing, simulated annealing and Tabu search) to carry out optimal assignment of developers to roles.

Despite much research being conducted concerning human factors, and in particular personality, most focuses on exploring and investigating their effects in various aspects of software development. There is still, however, a great need for tools to support software project managers incorporate these factors systematically in their staffing activities. Therefore, a major contribution of this work is providing such a tool, which carries out team staffing in an automated fashion and that, in addition, employs computational intelligence through the application of multi-objective optimization to handle the balancing of personality traits and technical skills and knowledge.

Personality Traits of Software Development Occupations

One aspect of the optimization approach implemented in this paper concerns the evaluation of selected developers based on the suitability of their personality traits with respect to the type of task they have been assigned to carry out. To do this, the software professions most commonly found in SMEs of today's software development industry were first identified using the 2010 Standard Occupational Classification (SOC), which serves as a systematized taxonomy of the majority of existing professions identified by the U.S. Bureau of Labor Statistics [START_REF]Standard Occupation Classification[END_REF]. Then, detailed analysis of each profession was carried out using the Occupational Information Network (O*NET) Resource Center [19], which provides a content model and an online database defining standardized and occupation-specific descriptors of each profession using the SOC system coding. Each occupation's job-related and worker-related characteristics and requirements were retrieved containing information on: the abilities and work styles of workers, the skills required by workers, and the work activities of occupations. Once these key requirements and characteristics were identified, the most suitable personality traits required by developers to carry out activities of each profession were then were associated with corresponding personality traits. These are expressed using five basic domains of personality, which comprise the Five-Factor Model (FFM), originally identified in 1961 by Tupes and Cristal [START_REF] Tupes | Recurrent Personality Factors Based on Trait Ratings[END_REF] and later operationalized by Costa and McCrae through the NEO-Personality Inventory (NEO-PI) [START_REF] Mccrae | NEO Inventories Professional Manual[END_REF]. The Five-Factor Model has been widely adopted in many academic and application disciplines where personality measures have been required, and is a common instrument in cases involving career and personnel assessment. Specifically, the five domains are described as follows:

 Neuroticism reflects the level to which an individual is predisposed to experiencing negative emotions, such as sadness, embarrassment, fear and anger.  Extraversion refers to the level to which an individual engages with their external world through interpersonal interactions, as well as their energy and predisposition to experiencing positive emotions.  Openness to experience concerns an individual's tendencies regarding intellectual curiosity, creativity and variety in interests and experiences.  Agreeableness involves interpersonal orientation with regards issues, such as compassion, social harmony, cooperation, and trust.  Conscientiousness relates to the degree of self-discipline and control of impulses, and also ambition and organization.

Finally, the desired level of each of the domain was determined so as to ascertain whether a profession requires either a {1:low, 2:medium or 3:high} level of that particular domain. The same was applied to personality traits possessed by developers so that comparisons between the two can take place using a simple distance measure.

An important issue here is that of the validation of each profession's associated personality traits that were selected as desirable. A large number of studies were used for this purpose, such as [START_REF] Moore | Personality Characteristics of Information Systems Professionals[END_REF][START_REF] Wynekoop | Revisiting the Perennial Question: Are IS People Different?[END_REF][START_REF] Smith | The Personality of the Systems Analysts: An Investigation[END_REF][START_REF] Capretz | Making Sense of Software Development and Personality Types[END_REF][START_REF] Varona | Evolution of Software Engineers' Personality Profile[END_REF], as well as other related material, for example, career handbooks suggesting the best occupation based on personality types. However, the validation process is not currently in the scope of this paper, since the desired personality types can be easily modified and, furthermore, the study's focus is on how well the chosen encoding performs in optimizing developer assignments.

Description of Methodology

The goal of the proposed approach is to allow project managers of SMEs in the software industry to staff their projects with the most suitable teams taking into account the technical knowledge and skills of available developers as well as their personality traits and abilities. These, however, may be viewed as conflicting in some cases, where a developer may be technically capable but does not hold the appropriate traits required by activities of a task, or vice-versa. For example, if a programming task requires skills in a specific programming language, a developer possessing a high level of such skills may not necessarily possess a low level of extraversion, which is one of the desired traits of programming tasks. On this basis, a software project manager would encounter difficulties in trying to make the best selection and assignment of resources whilst trying to balance the two. Also, due to the fact that there are many different possible combinations to examine, a software project manager will be required to perform an exhaustive assessment of all possible permutations, which only becomes more difficult when the number of tasks to be performed and the number of available developers increases. Therefore, in order to decrease the search space and handle the NP-hard nature of such a problem [START_REF] Pan | A Study of Project Scheduling Optimization using Tabu Search Algorithm[END_REF], a multi-objective optimization approach was adopted. In particular, two objective functions were modelled in an implementation of the Non-dominated Sorting Genetic Algorithm II (NSGA-II), introduced by Deb in 2002 [START_REF] Deb | A Fast and Elitist Multi-Objective Genetic Algorithm: NSGA-II[END_REF]. With this technique, a set of optimal solutions will be produced suggesting a collection of possible assignments of developers to project tasks aiming to satisfy the two aforementioned considerations. Furthermore, due to the involvement of constraints influencing the feasibility of solutions, the Constrained NSGA-II algorithm was applied. The algorithm promotes the solution diversity using a crowded comparison operator during its selection procedure and population reduction process. In addition, because parent and offspring populations are combined before non-dominated sorting takes place, elitism (i.e., preservation of the best solutions) is always ensured. Further details of the NSGA-II can be found in [START_REF] Deb | A Fast and Elitist Multi-Objective Genetic Algorithm: NSGA-II[END_REF].

Encoding and Representation

Each software project consists of a number of tasks that need to be carried out. For each task, developers need to be assigned to perform the activities involved. Therefore, since it is the selection of developers that forms the basis of evaluation, each project task is denoted by a string of bits, and each bit represents one specific developer. If a bit in the string has a value of '1', then this signifies that the specific developer is assigned to work on the task, whereas a value of '0' indicates that the developer has not been selected for the task. Overall, if a software project consists of T tasks and there are E available developers, then each solution would be represented by an individual in the algorithm using ( ) bits. Since it is possible that a development company follows its own method to evaluate technical skills and knowledge, each developer is simply required to be rated based on each of the skills required by task activities in a normalized form in the range [0, 1], meaning that low possession of skill will be denoted by a value closer to zero, and high possession of a skill will be denoted by a value closer to one. Project managers can use different metrics, such as experience or IT-related aptitude tests, or can even use the experience requirements and occupation-specific information set suggested by the O*NET Content Model. As part of on-going research efforts in this area, a method to rate and match developers' skills and knowledge is currently in progress. On the other hand, regarding personality traits, developers are assessed specifically using the NEO-Five-Factor Inventory and their five domain scores are used.

Objective Functions and Constraints

A total of three objective functions were created for the evaluation of each solution along with two constraints to measure the degree of feasibility of each solution.

Technical skills objective function ( ). This maximization function is responsible

for evaluating a solution based on the assigned developers' levels of technical skills and knowledge required by the activities of each task. The objective function's value is calculated for each task by adding the maximum skill level possessed by the developers assigned to the average skill level possessed by the developers assigned, as shown in Eq. (1).

) _ _ _ _ ( ) _ _ _ _ max( 1 developers assigned of level skill avg developers assigned of level skill f    (1)
Personality traits objective function ( ). This maximization function is responsible for evaluating a solution based on developers' personality traits. Using Eq. ( 2), a distance is computed between the desired levels of the FFM domains for the profession and the selected developer's levels of the FFM domains. If more than one developer is assigned to work on a task, then the average of the distances is used.

    5 1 2 _ _ _ _ _ _ i i i domain in level developer domain in level desired f ( 2
)
Team size objective function ( ). This minimization function is responsible for evaluating a solution based on the number of developers assigned to each task. The inclusion of such a fitness measure is important in order to maximize resource utilization and avoid unnecessary assignments. For each task, the inverse of the number of team members is calculated, as shown in Eq. ( 3).

developers assigned of number f _ _ _ 1 3  (3)

Skills satisfied constraint ( ).

The purpose of this constraint function is to measure the feasibility of a solution with respect to the technical skills required by activities of a task. Using Eq. ( 4), a solution is feasible only if for all activities, there is at least one developer selected to carry it out who possesses the necessary technical skills.

required skills project of number total skills d unsatisfie of number c _ _ _ _ _ _ _ _ 1  (4)

Developer availability constraint ( ).

The second constraint implemented involves checking the feasibility of a solution regarding the availability of developers when assigned to tasks that are carried out simultaneously. An assumption made in this approach is that no developer can work on more than one task at any given point in time. Furthermore, it is also assumed that the project's schedule is fixed beforehand and, thus, each task has already been allocated a specific "time slot" to be carried out.

Eq. ( 5) determines this constraint value by calculating the number of days a developer has been assigned to more than one task throughout the duration of the project.

project the in assigned days of number total conflicts with assigned days of number c _ _ _ _ _ _ _ _ _ _ _ _ 2  (5)

Algorithm Parameters and Settings

For the execution of the multi-objective genetic algorithm, a population of 100 individuals were used. The fast non-dominated sorting procedure was applied to rank the individuals in terms of their fitness and feasibility, after which a tournament selection of size 4 was used to select which parents were to enter the mating pool. The best two Fig. 1. Real-world software project schedule (vessel policies management system).

parents were then set to produce offspring by the application of a recombination operator (one-point crossover) with a likelihood of 0.80, and a mutation operator (single bit-flip mutation) with a ( ) probability. The population evolves by repeating the steps from the selection of individuals until the termination criteria are met or the maximum number of iterations have been reached (set at 2000).

5

Results of Experiments

Design of Experiments

Two experiments were carried out to evaluate the proposed methodology. First, in order to evaluate the validity and scalability of the multi-objective optimization algorithm, two hypothetical software projects consisting of 20 and 30 tasks each were created based on the input of several project managers of SME software development companies as to the basic structure, size and complexity of the type of software projects they usually undertake. For both projects, the skill levels and personality traits of the available software developers were selected so as to represent the best-case and worst-case scenarios for the proposed team staffing approach. For the best-case scenario, all available developers possessing the highest skill levels also possessed the most suitable personality traits. On the other hand, for the worst-case scenario, all the available developers possessing the highest skill levels possessed the least suitable personality traits, and vice-versa. Through these two extreme cases, both the behaviour and correctness of the optimization approach could be observed and analyzed, and also the competitive nature of the objective functions could be investigated.

In the second experiment, a real-world software project developed by a local IT company was used that related to the implementation of a vessel policies management system for a large insurance brokers company. The project, whose schedule is shown in Fig. 1, consisted of 31 tasks involving project management, design, programming and testing activities. A total of four developers were available to carry out the project, each possessing a varied set of skills at different levels, as well as unique personality traits, which were determined by administering the NEO-FFI-3 [START_REF] Mccrae | NEO Inventories Professional Manual[END_REF].

Results and Discussion

In the first experiment, the algorithm was executed 10 times for both hypothetical projects using the best-case and worst scenarios. The results are shown in Table 1. For both hypothetical projects, the algorithm managed to provide both feasible and optimal solutions when performing team staffing in the best-case scenario. Specifically, the algorithm always managed to assign the most suitable developer with respect to both the technical skill levels and personality traits possessed, and never assigned a developer who was less suited in either aspect. Furthermore, the Pareto front consisted of individuals representing the same optimal solution. This was expected since in the "optimistic" case there would always be only one possible ideal assignment existing for each task. In the worst-case scenario, the algorithm's job was to try to balance the two objective functions, since no developers possessed both the highest skill levels and most suitable personality traits for any task. For both hypothetical projects, it was observed that this time the individuals of the Pareto front represented a number of different solutions, as seen in Fig. 2. Such behaviour again was anticipated since for each task either skill levels or personality traits could be given preference -but not both due to the nature of the characteristics of the available developers. The general behaviour of the algorithm was, thus, validated as correct.

In the second experiment, the algorithm was also executed 10 times. However, the results obtained from these executions showed that the algorithm did not actually produce any optimal solutions but, in fact, consistently generated the same infeasible developer assignments with respect to their availability (constraint c 2 ). This observation is mainly attributed to the small number of available developers in combination with a relatively high number of concurrent tasks within the project. Specifically, in cases where several tasks requiring the same skills and personality traits were set to execute simultaneously (e.g., tasks T8-T14 in Fig. 1), the algorithm would encounter difficulties in finding an optimal assignment of developers possessing these to the Fig. 2. Pareto front of hypothetical project (30 tasks) worst-case scenario (f 1 vs. f 2 ). tasks, simply because the available resources were insufficient for such concurrent scheduling of tasks.

What's interesting, however, is that whilst consulting with the project's manager to understand the method that was used to allocate human resources to tasks, it was established that the project suffered from schedule overruns due to the lack of available resources. This shows the potential of such approach as it can be used as means to pinpoint possible staffing caveats for project managers, who would then be able to seek a solution by either revising their original schedule of tasks or even consider hiring or buying the services of developers for tasks that could not be optimally allocated resources. To further investigate this, the experiment was repeated without inclusion of the developer availability constraint. The results this time showed that the algorithm was able to produce both feasible and optimal assignments, averaging around 95 unique Pareto front solutions over 10 executions. In some cases developers possessing high levels of skills required for tasks but less suitable personality traits were chosen and in other cases developers most suited with respect to personality traits were preferred even if they possessed lower levels of skills for the tasks they were assigned to. This further enhanced the initial belief that the two objectives can indeed be competing. In such a case, it is up to the project manager to decide which of the two options has higher priority and, therefore, should be followed.

Concluding Remarks

The approach described in this paper proposes the innovative use of computational intelligence as a means to help software project managers solve the problem of team staffing. In particular the approach suggests the use of a multi-objective genetic algorithm for assigning software developers to project tasks based on technical skills and personality traits. By taking into consideration these factors, it allows software project managers to view team staffing and human resource allocation from an alternative perspective, since software project success is considered to be largely influenced by the human factors present in software development. The results obtained from several experiments indicate that the algorithm is capable of generating adequate and feasible solutions, balancing the two objectives where necessary, and has the potential to constitute a decision support tool for software project team staffing. One of the major contributions of this approach is that it can allow project managers to foresee possible resource issues arising during development. With respect to either or both constraint functions, if the algorithm is unable to find feasible solutions when applied to a specific project whose schedule is fixed, this could indicate that the available resources are not sufficient or adequate enough to carry out the software project. This is very useful for project managers since it would allow them to revise their project schedule and attempt to staff their team in a slackened timeframe. Alternatively, without modifying the project's schedule, a project manage may use the results to recruit extra resources (possessing either higher skill levels or more suitable personality traits or both). In a similar way, the approach can be used to examine whether the development company has the required capacity in terms of human resources before bidding for a software project.

As part of future work, improvements can be made to the way in which developer skill levels are measured and evaluated Also, additional objective functions can be introduced, such as the minimization of a project's cost based on developers' salaries. Attempts have also been made to collect data from development companies for the purposes of further evaluation using real-world projects, in addition to comparison of the approach with other computational intelligence techniques. Another possible future enhancement can involve integrating the proposed approach with other techniques for software project management activities taking into account the solutions generated from the current optimization method. This could allow for a comparative analysis of the effectiveness of the approach when combined and used in parallel with other models. Possible applications include feeding the solutions generated by the algorithm into an intelligent scheduling mechanism or, alternatively, using the output of assigned developers to help predict the cost of a software project (per phase or as a whole).

Table 1 .

 1 Results obtain from the first experiment using hypothetical software projects.

	Experiment	Average Number of Unique Solutions	Execution Time (min.)
	Best-case scenario (20 tasks)	1	22.56
	Worst-case scenario (20 tasks)	94	22.86
	Best-case scenario (30 tasks)	1	23.90
	Worst-case scenario (30 tasks)	95	26.08


Haidar Samet
email: samet@shirazu.ac.ir

Farshid Nasrfard Jahromi

Arash Dehghani
email: dehghani_arash@yahoo.com

Afsaneh Narimani
email: afsaneh_narimani@yahoo.com

Improving Current and Voltage Transformers Accuracy Using Artificial Neural Network

Keywords: Artificial Neural Network (ANN), Capacitive Voltage Transformers (CVTs), Current Transformers (CTs), Transient

Capacitive Voltage Transformers (CVTs) and Current Transformers (CTs) are commonly used in high voltage (HV) and extra high voltage (EHV) systems to provide signals for protecting and measuring devices. Transient response of CTs and CVTs could lead to relay mal-operation. To avoid these phenomena, this paper proposes an artificial neural network (ANN) method to correct CTs and CVTs secondary waveform distortions caused by the transients. PSCAD/EMTDC software is employed to produce the required voltage and current signals which are used for the training process and finally the results show that the proposed method is accurate and reliable in estimation of the CT primary current and the CVT primary voltage.

Introduction

The operation of the protection relays is dependent on the measured signals such as current and voltage signals which are measured by CTs and CVTs. Thus an error in these signal measurements could lead to mal-operation or substantial delay in tripping of the protection relays. The CT and CVT output signals may not follow their input signals due to the transients mainly caused by current transformers saturation and discharging of the capacitive voltage transformers internal energy during faults. Several techniques on the compensation of the distorted secondary current and voltage signals have been published. An algorithm to estimate the magnetizing current at each time step by using the magnetization curve of a CT was reported in [START_REF] Kang | An algorithm for compensating secondary current of current transformers[END_REF], the other algorithm detects the saturation using the Discrete Wavelet Transform (DWT) and compensates the distorted section of a secondary current with features extracted from the healthy section using a least mean square fitting method [START_REF] Li | combined wavelet transform and regression technique for secondary current compensation of current transformer[END_REF].

Another method is to Compensate the error of CT by using hysteresis and Eddy characteristic [START_REF] Locci | hysteresis and eddy currents enompsation in CT[END_REF]. Some digital methods to correct the secondary waveforms of the CT are proposed in Ref. [START_REF] Locci | A Digital Compensation Method for Improving Current Transformer Accuracy[END_REF]. One method is to detect high Source Impedance Ratio (SIR) conditions that could lead to severe CVT transients and add a time delay to the tripping decision of distance relay [START_REF] Hou | Capacitive Voltage Transformers: Transient Overreach Concerns and Solutions for Distance Relaying[END_REF]. The method discussed in Ref. [START_REF] Kh | A compensation scheme for CVT transient effects using artificial neural network[END_REF] proposes the use of artificial neural networks (ANNs) to correct CVT secondary voltage distortions due to CVT transient. Another method is to compensate the distorted secondary waveforms of the CVT in the time domain by considering the hysteresis characteristics of the core [START_REF] Kang | Design and evaluation of a compensating algorithm for the secondary voltage of a coupling capacitor voltage transformer in the time domain[END_REF]. The method introduced in Ref. [START_REF] Izykowski | S Dynamic Compensation of Capacitive Voltage Transformers[END_REF] proposes the use of compensation algorithms based on of the CVT transfer function. Some least squares phasor estimation techniques for estimation of voltage and current phasors are presented in Refs. [START_REF] Ajaei | Minimizing the Impact of Transients of Capacitive Voltage Transformers on Distance Relay[END_REF] and [START_REF] Pajuelo | An Improved Voltage Phasor Estimation Technique to Minimize the Impact of CCVT Transients in Distance Protection[END_REF]. This paper begins with analysis of the contributing factors in CT and CVT transients. The details of the simulated power system to produce the different cases which are used in the training and testing process are presented in the later section. An ANN method for CT and CVT error compensation is presented in section 5. Results clearly show that the proposed method could estimate the primary signals accurately.

CT Transients

CT model used in simulations is shown in Figure1 [START_REF] Pan | An Efficient Compensation Algorithm for Current Transformer Saturation Effects[END_REF]. Real iron-cored CTs are not ideal and since the reluctance of the core is not infinite, the current which is obtained by dividing the primary current (I p ) to the turns ratio (N), is different from the secondary current (I s ) in amplitude and phase due to existence of exciting current (I e ).

B. Coupling capacitors

The higher the sum of the capacitances (C=C 1 +C 2 ), the lower the magnitude of the transients. High-C CVTs decrease the magnitude of the CVT transients but are more expensive. So there should be a balance between CVT performance and CVT cost [START_REF] Kasztenny | Digital Relays and Capacitive Voltage Transformers: Balancing Speed and Transient Overreach[END_REF].

C. Ferroresonance suppression circuit

There are two models of a ferroresonance suppression circuit which are shown in Figure 3. The transients of the CVT with the passive model are less severe so the output voltage is less distorted [START_REF] Kasztenny | Digital Relays and Capacitive Voltage Transformers: Balancing Speed and Transient Overreach[END_REF], [START_REF] Kh | A compensation scheme for CVT transient effects using artificial neural network[END_REF].

D. CVT burden

The energy accumulated in the CVT storage elements could be dissipated in the CVT burden. Therefore, the different CVT burden could change the shape and duration of the CVT transients. As a rule, if the CVT is fully loaded, the transients would be less severe [START_REF] Kasztenny | Digital Relays and Capacitive Voltage Transformers: Balancing Speed and Transient Overreach[END_REF].

Power System Model

A three-phase 230 kV, radial power system is used in the simulations .The one-line diagram of this system is shown in Figure 4 and its parameters are shown in Table 1.

Since ANN methods require an adequate amount of data for the training process, samples of the secondary voltage and current signals are achieved by using PSCAD/EMTDC software and then the primary voltage and current signals could be estimated by using the proposed method. The parameters of the CT and CVT are shown in tables 2 and 3, respectively. Different values between (0-360)with a step of 45 degrees Fault type Single-phase-to-ground, phase-to-phase-to-ground, threephase -to-ground Fault resistance(Ω) 0.01, 1, 100

Network Structure and Training

An ANN has the ability to learn from data. Such a situation is shown in Figure 5.

Typically many of such input/output pairs are needed to train a network. A very important feature of these networks is their adaptive nature, where "learning by example" replace "programming" in solving problems. ANNs can be used to perform different tasks in power system relaying for signal processing and decision making [START_REF] Jhr | Artificial Neural Network[END_REF]. A major problem with ANNs is that there is no exact rule to choose the number of hidden layers and neurons per hidden layer [START_REF] Saha | Artificial Intelligent Application to Power System Protection[END_REF].The most widely used learning algorithm in an ANN is the Back-propagation algorithm [START_REF] Jhr | Artificial Neural Network[END_REF], [START_REF] Saha | Artificial Intelligent Application to Power System Protection[END_REF].

Test results

The accuracy of the proposed method is tested with patterns which are different from the training patterns. Figures 6 and7 show the test results of the CT. Furthermore, Figures 8 and9 show the test results of the CVT. The left side figures show the ideal (primary side), actual (secondary side) and the output of ANN while the right side figures show the estimation errors. The results show that the ANN output could exactly follow the ideal CT and CVT outputs ("ideal output" here means the primary value of CT and CVT and the "actual output" means the secondary side value of CT and CVT).

Fig. 1 .R

 1 Fig. 1. Equivalent circuit of a CT[START_REF] Pan | An Efficient Compensation Algorithm for Current Transformer Saturation Effects[END_REF]

Fig. 2 .

 2 Fig. 2. Equivalent circuit of a CVT[START_REF] Kasztenny | Digital Relays and Capacitive Voltage Transformers: Balancing Speed and Transient Overreach[END_REF]

Fig. 3 .

 3 Fig. 3. Active and Passive Ferroresonance suppression circuit[START_REF] Locci | hysteresis and eddy currents enompsation in CT[END_REF]

Fig. 4 .

 4 Fig. 4. Structure of the power system model

Fig. 5 .

 5 Fig. 5. A learning cycle in the ANN model In this paper, for each CT and CVT a feedforward neural network with two hidden layers is used to process the training input data. The sampling rate is 40 samples per one 50-Hz cycle. The neural network includes 20 inputs, 20 outputs, two hidden layers (the first one with 4 neurons and the second one with 3 neurons). The inputs are the instantaneous values of the secondary current and voltage of the CT and CVT, respectively and the outputs are the instantaneous values of the primary sides of the CT and CVT, respectively. A log-sigmoid function as the activation function of the hidden layer neurons and a tan-sigmoid function for the output layer are used. The neural network is trained by Back-Propagation (BP) algorithms.

Fig. 6 .Fig. 7 .

 67 Fig. 6. a) CT waveforms for B-G fault with inception angle=300° and fault resistance=0. 1Ω b) Estimation error

Fig. 9 .

 9 Fig. 9. a) CVT waveforms for A-B-C fault with inception angle=240° and fault resistance=0.1Ω b) Estimation error 6 Conclusion

Table 1 .

 1 Simulated power System parameters

	Generator impedance(Ω)	1
	Line impedance(Ω/km)	0.072 + j 0.416
	Load	Active power(MW) Reactive power(MVAR)	100

Table 2 .

 2 CT circuit parameters

	Primary turns	20
	Secondary turns	200
	Secondary resistance	0.5Ω
	Secondary Inductance	0.8e3H
	Area	2.601e-3m*m
	Path length	0.6377m
	Burden resistance	0.5 Ω

Table 3 .

 3 CVT circuit parameters

	Capacitor-1	2920 pF
	Capacitor-2	134952 pF
	Compensating inductance	42 H
	VT ratio	43.48
	Primary inductance	0.47e-3 H
	Primary resistance	0.05 Ω
	Secondary inductance	0.47e-3 H
	Secondary inductance	0.18 Ω
	Eddy current loss at normal conditions	2.5 W
	Secondary operation voltage	115 V
	Operating flux density	0.8 T
	Hysteresis loss at normal conditions	5 W
	Burden resistance	301 Ω
	Burden inductance	2.4 H
	5

Proposed Neural Network Compensating Method 5.1 Training patterns To

generate training patterns, the following conditions are changed from the base case: fault inception angle, fault type and fault resistance. The training pattern data generation is shown in Table

4

.

Table 4 .

 4 Training pattern data generation

	Fault inception
	angle(°)


Věra Kůrková
email: vera@cs.cas.cz

Surrogate Modelling of Solutions of Integral Equations by Neural Networks

Keywords: surrogate modelling by neural networks, Fredholm integral equations, rates of approximation, model complexity

Surrogate modelling of solutions of integral equations by neural networks is investigated theoretically. Estimates of speed of convergence of suboptimal surrogate solutions to solutions described by Fredholm theorem are derived.

Introduction

Surrogate modelling of functional relationships by neural networks has been successfully applied to empirical functions for which experimental evaluations are too expensive or to functions, which are described by known analytic formulas but numerical calculations of such formulas are too time consuming. For construction of surrogate models, results of such experiments or calculations merely at some selected points of the domains of the functions are needed. Values obtained by such expensive or time consuming procedures are then used as training samples of data for neural networks. The networks trained on such samples are called surrogate models. For example, neural networks have been successfully used in chemistry as surrogate models of empirical functions assigning to compositions of chemicals measures of quality of catalyzers produced by reactions of these chemicals or in biology as surrogate models of empirical functions classifying structures of RNA [START_REF] Forrester | Engineering Design via Surrogate Modelling: A Practical Guide[END_REF][START_REF] Baerns | Combinatorial Development of Solid Catalytic Materials[END_REF].

In the case of empirical functions, results obtained by surrogate modelling can only be used as suggestions to be confirmed by additional experiments as no other than empirical knowledge of the functions is available. In the case of functions whose mathematical expressions are known, although complicated and difficult to calculate numerically, these expressions might enable to estimate how close to these functions their surrogate models are. In particular, theoretical analysis can provide estimates of model complexity of neural networks needed for a sufficient accuracy of approximation of the functions which are replaced by these networks.

Recently, possibilities of neural networks in surrogate modelling of solutions of Fredholm integral equations were explored. Fredholm equations play an important role in many problems in applied science and engineering. They arise in image restoration, differential problems with auxiliary boundary conditions, potential theory and elasticity, etc. (see, e.g., [START_REF] Lovitt | Linear Integral Equations[END_REF][START_REF] Lonseth | Sources and applications of integral equations[END_REF][START_REF] Lu | Integral equation models for image restoration: high accuracy methods and fast algorithms[END_REF]). Numerical calculations of values of solutions of Fredholm equations are time consuming as they involve computations of complicated expressions in terms of infinite Liouville-Neumann series with integrals coefficients. Thus various methods of approximate solutions of Fredholm equations have been developed. Traditional approach employed polynomial or wavelet interpolations of numerically calculated values at certain collocation points [START_REF] Atkinson | Theoretical Numerical Analysis: A Functional Analysis Framework[END_REF]Chapter 11]. As an alternative to the traditional approach, surrogate models formed by neural networks with perceptrons [START_REF] Effati | A neural network approach for solving Fredholm integral equations of the second kind[END_REF] and Gaussian radial-basis units [START_REF] Golbabai | Numerical solution of the second kind integral equations using radial basis function networks[END_REF] were proposed and explored experimentally. These experiments were motivated by higher flexibility of neural networks than flexibility of polynomials. Neural networks have more adjustable parameters than linear models as in addition to coefficients of linear combinations of basis functions, also inner coefficients of computational units are optimized during learning. Thus they are sometimes called variable-basis approximation schemas in contrast to traditional linear approximators which are called fixed-basis approximation schemas. In some cases, especially in the case of functions of large numbers of variables, neural networks achieve better approximation rates than linear models with much smaller model complexity [START_REF] Gnecco | Some comparisons of complexity in dictionary-based and linear computational models[END_REF][START_REF] Gnecco | Can dictionary-based computational models outperform the best linear ones?[END_REF]. Motivated by the experimental exploration [START_REF] Golbabai | Numerical solution of the second kind integral equations using radial basis function networks[END_REF][START_REF] Effati | A neural network approach for solving Fredholm integral equations of the second kind[END_REF] of surrogate models of solutions of Fredholm equations by neural networks, Gnecco et al. [START_REF] Gnecco | Accuracy of approximations of solutions to Fredholm equations by kernel methods[END_REF] initiated a theoretical analysis of this modelling. In [START_REF] Gnecco | Accuracy of approximations of solutions to Fredholm equations by kernel methods[END_REF], they estimated approximation errors in supremum norm for surrogate solutions by networks with kernel units induced by the kernels of the equations.

In this paper, we investigate surrogate solutions of Fredholm integral equation by networks with general computational units. Taking advantage of results from nonlinear approximation theory and integral representations of functions, we estimate how well surrogate solutions computable by neural networks can approximate exact solutions of Fredholm equations described by the Fredholm alternative theorem. We show that such surrogate solutions converge to the exact solutions and we give upper bounds on decrease of approximation errors measured in L 2 -norm with increasing model complexity of approximating networks.

The paper is organized as follows. In section 2, we introduce Fredholm integral equations and their surrogate solutions by neural networks. In section 3, using results from nonlinear approximation theory, we derive estimates of rates of approximation of solutions of Fredholm equations by neural networks. Section 4 is a brief discussion.

Solutions of Fredholm Equations

Solving an inhomogeneous Fredholm integral equation of the second kind on a domain

X ⊆ R d for a given λ ∈ R \ {0}, K : X × X → R, and f : X → R is a task of finding a function ϕ : X → R such that for all x ∈ X ϕ(x) -λ ∫ X ϕ(y)K(x, y) dy = f (x). ( 1
)
The function ϕ is called a solution, f data, K a kernel, and a λ parameter of the equation [START_REF] Forrester | Engineering Design via Surrogate Modelling: A Practical Guide[END_REF].

Recall that tasks of finding unknown causes of known consequences have been studied in applied science under the name inverse problems. Formally, an inverse problem is defined by a linear operator A : X → Y between two function spaces. It is a task of finding for g ∈ Y (called data) some f ∈ X (called solution) such that A(f ) = g.

Let T K denotes the integral operator with a kernel K : X × X → R defined for every ϕ in a suitable function space as

T K (ϕ)(x) := ∫ X ϕ(y) K(x, y) dy .
(

) 2
Thus solving the Fredholm equation ( 1) is an inverse problem described by the linear operator I X -λ T K , where I X is the identity operator. Existence and uniqueness of solutions of this inverse problem follows from the Fredholm alternative theorem (see, e.g., [12, p.499]).

Theorem 1. Let X be a Banach space, T : X → X be a compact operator, and I X be the identity operator. Then the operator I X + T : X → X is one-to-one if and only if it is onto.

Theorem 1 implies that when T is a compact operator and 1/λ is not its eigenvalue (i.e., there is no ϕ ∈ X for which T (ϕ) = ϕ λ ), then the operator I X -λT is invertible (one-to-one and onto) [13, p. 112], [START_REF] Atkinson | The Numerical Solution of Integral Equations of the Second Kind[END_REF]Section 1.3].

Fredholm theorem can be applied to spaces (C(X), ∥.∥ sup ) of bounded continuous functions on

X ⊆ R d with the supremum norm ∥f ∥ sup = sup x∈X |f (x)| and to spaces (L 2 (X), ∥.∥ L 2 ) of square integrable functions with the norm ∥f ∥ L 2 = (∫ X f (x) 2 dx ) 1/2 .
The following proposition is well-known and easy to check (see, e.g., [13, p. 112]).

Proposition 1. (i) If X ⊂ R d is compact and K : X × X → R is continuous, then T K : C(X) → C(X) is a compact operator. (ii) If X ⊂ R d and K ∈ L 2 (X × X), then T K : L 2 (X) → L 2 (X) is a compact operator.
So by the Fredholm theorem, when 1/λ is not an eigenvalue of T K and the assumptions of the Proposition 1 (i) or (ii) are satisfied, then for every f in C(X) or L 2 (X), resp., there exists unique solution ϕ of the equation [START_REF] Forrester | Engineering Design via Surrogate Modelling: A Practical Guide[END_REF]. Moreover,

ϕ(x) = f (x) -λ ∫ X f (y) R λ K (x, y) dy , (3)
where [START_REF] Atkinson | The Numerical Solution of Integral Equations of the Second Kind[END_REF]. Although a formula expressing the resolvent kernel is known, it is not suitable for efficient computation as it is expressed as an infinite Neumann series in powers of λ with coefficients in the form of iterated kernels [15, p.140]. Numerical calculations of values of solutions of Fredholm equations based on (3) are quite computationally demanding and so various methods of finding surrogate solutions of (1) have been explored.

R λ K : X × X → R is called a resolvent kernel
A traditional approach employed linear methods such as polynomial interpolation of approximations φ(x 1 ), . . . , φ(x m ) (computed by numerical methods) of the values ϕ(x 1 ), . . . , ϕ(x m ) of the solution ϕ at suitable points x 1 , . . . , x m from the domain X. In linear methods, approximating sets of functions are ndimensional subspaces of the form span{g 1 , . . . , g n } :=

{ n ∑ i=1 w i g i | w i ∈ R } , ( 4
)
where G = {g n | n ∈ N + } is a set of functions with a fixed linear ordering.

Nested linear subspaces of the form (4) can be generated, e.g., by sets of powers of increasing degrees or trigonometric functions with increasing frequencies.

A recent alternative to linear methods employs neural networks. Recall that one-hidden-layer networks with one linear output unit compute input-output functions from sets of the form

span n G := { n ∑ i=1 w i g i | w i ∈ R, g i ∈ G } , ( 5
)
where the set G is sometimes called a dictionary [START_REF] Gribonval | On the exponential convergence of matching pursuits in quasi-incoherent dictionaries[END_REF] and n is the number of hidden computational units. This number can be interpreted as a measure of model complexity of the network. In contrast to linear approximation, the approximation scheme formed by sets span n G is called variable-basis approximation (the set G has no fixed ordering). Often, dictionaries are parameterized families of functions modelling computational units, i.e., they are of the form

G K (X, Y ) := {K(•, y) : X → R | y ∈ Y } , ( 6
)
where K : X ×Y → R is a function of two variables, an input vector x ∈ X ⊆ R d and a parameter y ∈ Y ⊆ R s . When X = Y , we write briefly G K (X). In some contexts, K is called a kernel. However, the above-described computational scheme includes fairly general computational models, such as functions computable by perceptrons, radial or kernel units, Hermite functions, trigonometric polynomials, and splines. For example, with

K(x, y) = K(x, (v, b)) := σ(⟨v, x⟩ + b)
and σ : R → R a sigmoidal function, the computational scheme (5) describes one-hidden-layer perceptron networks. RBF units with an activation function β : R → R are modeled by kernel

K(x, y) = K(x, (v, b)) := β(v∥x -b∥).
Typical choice of β is the Gaussian function. Kernel units used in support vector machine have the form K(x, y) where K : X × X → R is a symmetric positive semidefinite function [START_REF] Steinwart | Support Vector Machines[END_REF].

Various learning algorithms optimize parameters y 1 , . . . , y n of the computational units as well as coefficients of their linear combinations w 1 , . . . , w n so that network input-output functions ∑ n i=1 w i K(., y i ) from the set span n G K fit well to training samples. In the case of surrogate solutions of Fredholm equations, training samples are the above described sets {(x 1 , φ(x 1 )), . . . , (x m , φ(x m ))} of input-output pairs formed by selected points of the domain together with numerical approximations of values of the solution ϕ at these points.

Rates of Convergence of Surrogate Solutions

Estimates of model complexity of one-hidden-layer networks approximating solutions of Fredholm equations follow from inspection of upper bounds on rates of variable-basis approximation. Such rates have been studied in mathematical theory of neurocomputing for various types of dictionaries G and norms measuring approximation errors such as Hilbert-space norms and the supremum norm (see, e.g., [START_REF] Barron | Universal approximation bounds for superpositions of a sigmoidal function[END_REF][START_REF] Girosi | Approximation error bounds that use VC-bounds[END_REF]). Typically, such bounds have the form ξ(h) √ n , where n is the number of network units and ξ(h) depends on a certain variational norm of the function h to be approximated, which is tailored to the dictionary of the network units. Variational norm is defined quite generally for any bounded nonempty subset G of a normed linear space (X , ∥.∥ X ). It is called G-variation, denoted ∥.∥ G , and defined for all f ∈ X as

∥f ∥ G,X := inf {c > 0 | f /c ∈ cl X conv (G ∪ -G)} ,
where the closure cl X is taken with respect to the topology generated by the norm ∥.∥ X and conv denotes the convex hull. So G-variation depends on the ambient space norm, but when it is clear from the context, we write merely ∥f ∥ G instead of ∥f ∥ G,X .

The concept of variational norm was introduced by Barron [19] for sets of characteristic functions. In particular, for the set of characteristic functions of half-spaces, the variational norm is tailored to the dictionary of functions computable by Heaviside perceptrons. Barron's concept was generalized in [START_REF] Kůrková | Dimension-independent rates of approximation by neural networks[END_REF][START_REF] Kůrková | High-dimensional approximation and optimization by neural networks[END_REF] to the concept of variation with respect to an arbitrary bounded set of functions and applied to various dictionaries of computational units such as Gaussian RBF or kernel units [START_REF] Kainen | Complexity of Gaussian radial-basis networks approximating smooth functions[END_REF].

The following theorem from [START_REF] Kůrková | High-dimensional approximation and optimization by neural networks[END_REF] is a reformulation of results by Maurey [START_REF] Pisier | Remarques sur un résultat non publié de B. Maurey[END_REF], Jones [START_REF] Jones | A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training[END_REF], Barron [START_REF] Barron | Universal approximation bounds for superpositions of a sigmoidal function[END_REF] in terms of G-variation. It gives an upper bound on rates of approximation by sets of the form span n G. For a normed space (X , ∥.∥ X ), g ∈ X and A ⊂ X , we denote by ∥g -

A∥ X := inf f ∈A ∥g -f ∥ X the distance of g from A.
Theorem 2. Let (X , ∥.∥ X ) be a Hilbert space, G its bounded nonempty subset, s G = sup g∈G ∥g∥ X , f ∈ X , and n be a positive integer. Then

∥h -span n G∥ 2 X ≤ s 2 G ∥h∥ 2 G -∥h∥ 2 X n .
Theorem 2 implies that for every ε > 0 and n satisfying

n ≥ ( s G ∥h∥ G ε ) 2 ,
a network with n units computing functions from the dictionary G approximates the function h within ε. So the size of the G-variation of the function h is a critical factor influencing model complexity of networks approximating h within an accuracy ε. Generally, it is not easy to estimate G-variation. However, for the special case of functions with integral representations in the form of "infinite networks", variational norms are bounded from above by the L 1 -norms of "output-weight" functions of these networks.

Theorem 3. Let X ⊆ R d , Y ⊆ R s , w ∈ L 1 (Y ), K : X × Y → R be such that G K (Y ) = {K(., y) | y ∈ Y } is a bounded subset of (L 2 (X), ∥.∥ L 2 ), and h ∈ L 2 (X) be such that for all x ∈ X, h(x) = ∫ Y w(y) K(x, y) dy. Then ∥f ∥ GK (Y ) ≤ ∥w∥ L 1 (Y ) .
Theorem 3 gives an upper bound on variation with respect to the dictionary G K induced by the kernel K of the integral operator T K which maps an "outputweight" function w of the "infinite network" ∫ Y w(y) K(., y) dy to the function h. However, to apply Theorem 2 to approximation of solutions of Fredholm equations by surrogate models formed by networks with units from a general dictionary G, we need upper bounds on G-variation for functions expressed by an integral formula with the kernel K. The next proposition describes a relationship between variations with respect to two sets, G and F . Its application to G and G K gives an upper bound on G. The proof of the next proposition follows easily from the definition of variational norm. Proposition 2. Let (X , ∥.∥ X ) be a normed linear space, F and G its bounded subsets such that c G,F := sup g∈G ∥g∥ F < ∞. Then for all h ∈ X , ∥h∥ G ≤ c G,F ∥h∥ F .

Combining Theorems 2, 3, and Proposition 2, we obtain the next theorem on rates of approximation of functions which can be expressed as h = T K (w) by networks with units from a dictionary G.

Theorem 4. Let X ⊆ R d , K : X × Y → R be a bounded kernel, and h ∈ L 2 (X) such that h = T K (w) = ∫ Y w(y)K(., y) dy for some w ∈ L 1 (Y ), where G K (X, Y ) is a bounded subset of L 2 (X). Let G be a bounded subset of L 2 (X) with s G = sup g∈G ∥g∥ L 2 such that c G,K = sup g∈G ∥g∥ GK is finite. Then for all n > 0, ∥h -span n G∥ L 2 ≤ s G c G,K ∥w∥ L 1 (Y ) √ n .
A critical factor in the upper bound from Theorem 4 is the L 1 -norm of the "output-weight function" w in the representation

h(x) = T K (w)(x) = ∫ Y w(y) K(x,

y) dy

of h as an "infinite network" with units computing K(., y).

The solution ϕ of the Fredholm equation minus the function f representing the data, ϕ -f , is the image of λ ϕ mapped by the integral operator T K , i.e.,

ϕ -f = λ ∫ X f (y) R λ K (x, y) dy = λ ∫ X ϕ(y)K(x, y) dy = T K (λ ϕ).
Thus to apply Theorem 4 to approximation of ϕ -f , we need to estimate the L 1 -norm of the solution ϕ itself as λ ϕ plays the role of the "output-weight" function in the infinite network λ ∫ X ϕ(y)K(x, y) dy. Theorem 5. Let X ⊂ R d be compact, K : X × X → R be a bounded kernel such that K ∈ L 2 (X × X), s K := sup y∈Y ∥K(., y)∥ L 2 , ρ K := ∫ X sup y∈X |K(x, y)|dx be finite, G be a bounded subset of L 2 (X) with s G = sup g∈G ∥g∥ L 2 such that c G,K = sup g∈G ∥g∥ GK is finite, and λ ̸ = 0 be such that 1 λ is not an eigenvalue of T K and |λ| ρ K < 1. Then the solution ϕ of the equation ( 1) satisfies for all n > 0,

∥ϕ -f -span n G∥ L 2 ≤ s G c G,K |λ| ∥f ∥ L 1 (1 -|λ| ρ K ) √ n .
Proof. By (1), we have for every x ∈ X,

|ϕ(x)| ≤ |λ| ∥ϕ∥ L 1 sup y∈X |K(x, y)| + |f (x)|. Hence ∥ϕ∥ L 1 ≤ |λ| ρ K ∥ϕ∥ L 1 + ∥f ∥ L 1 and so ∥ϕ∥ L 1 (1 -|λ| ρ K ) ≤ ∥f ∥ L 1 .
This inequality is non trivial only when |λ| < 1 ρK . Thus we get ∥w∥

L 1 = |λ|∥ϕ∥ L 1 ≤ |λ| ∥f ∥ L 1
1-|λ| ρK . The statement then follows from Theorem 4. Theorem 5 estimates rates of approximation of the function ϕ -f by functions computable by networks with units from a dictionary G. The values of the function λ ∫ X f (y) R λ K (x, y) dy are difficult to be compute numerically. However, ϕ -f can be replaces with its approximation computable by a neural network. With increasing number n of units, input-output functions of such networks converge to the function ϕ -f . When for a reasonable size of the network measured by the number n of units, the upper bound from Theorem 5 is sufficiently small, the network can serve as a good surrogate model of the solution of Fredholm equation.

The next corollary illustrates our results by an example of approximation of solutions of Fredholm equations with Gaussian kernels by Gaussian kernel and perceptron networks. Fredholm equations with Gaussian kernels arise, e.g., in image restoration problems [START_REF] Lu | Integral equation models for image restoration: high accuracy methods and fast algorithms[END_REF]. By µ is denoted the Lebesgue measure on R d and by P σ d (X) the dictionary of functions on X ⊆ R d computable by sigmoidal perceptrons.

Corollary 1. Let X ⊂ R d be compact, b > 0, K b (x, y) = e -b∥x-y∥ 2 , λ ̸ = 0 be such that 1 λ is not an eigenvalue of T K b and |λ| < 1. Then the solution ϕ of the equation ϕ -f = λ ∫ X ϕ(y)K(x, y) dy with f continuous satisfies for all n > 0

∥ϕ -f -span n G K b (X)∥ L 2 ≤ µ(X) |λ| ∥f ∥ L 1 (1 -|λ| µ(X)
√ n and ∥ϕ -f -

span n P σ d (X)∥ L 2 ≤ µ(X) 2d |λ| ∥f ∥ L 1 (1 -|λ| µ(X) √ n .
Proof. It was shown in [START_REF] Kainen | A Sobolev-type upper bound for rates of approximation by linear combinations of Heaviside plane waves[END_REF] that P σ d -variation (i.e., variation with respect to the dictionary formed by sigmoidal perceptrons) of the d-dimensional Gaussian is bounded from above by 2d. The statement then follows by Theorem 5, Proposition 2, equalities τ K b = 1, ρ K b = µ(X), and the estimate s K b ≤ µ(X).

Discussion

Taking advantage of results from mathematical theory of neurocomputing holding for functions representable as "infinite neural networks" we derived estimates of rates of convergence of surrogate solutions of Fredholm equations to exact solutions guaranteed by Fredholm theory. The bounds are in the form c √ n where n is the number of network units and c depends on L 1 -norm of the function f corresponding to data of the inverse problem given by the operator I -λT K , the parameter λ, the Lebesgue measure µ(X) of the domain X ⊂ R d , and a bound on ∥K(., y)∥ L 2 . The results hold under certain restrictions on the size of λ similarly as in the case of traditional linear approximation of solutions of Fredholm equations.

Acknowledgments. This work was partially supported by M ŠMT grant COST INTELLI OC10047 and RVO 67985807.