Petr Hajek
email: petr.hajek@upce.cz

Interpretable Fuzzy Rule-Based Systems for Detecting Financial Statement Fraud ⋆

Keywords: Financial statement fraud, Fuzzy rule-based systems, Interpretability, evolutionary algorithms

Systems for detecting financial statement frauds have attracted considerable interest in computational intelligence research. Diverse classification methods have been employed to perform automatic detection of fraudulent companies. However, previous research has aimed to develop highly accurate detection systems, while neglecting the interpretability of those systems. Here we propose a novel fuzzy rule-based detection system that integrates a feature selection component and rule extraction to achieve a highly interpretable system in terms of rule complexity and granularity. Specifically, we use a genetic feature selection to remove irrelevant attributes and then we perform a comparative analysis of state-of-the-art fuzzy rule-based systems, including FURIA and evolutionary fuzzy rule-based systems. Here, we show that using such systems leads not only to competitive accuracy but also to desirable interpretability. This finding has important implications for auditors and other users of the detection systems of financial statement fraud.

Introduction

Although corruption and asset misappropriation tend to occur at greater frequency, financial statement fraud is reported to be the most costly internal (occupational) fraud with a median loss of 800,000 USD at global level [START_REF]Report of the Nations -2018 Global Study on Occupational Fraud and Abuse[END_REF]. Note that this value is based on the estimate of the gross amount of the financial statement misstatement. Financial statement fraud is caused by intentional omission or misstatement of material information in a company's financial report, including net worth (net income) under-and overstatements using timing differences, improper disclosures, fictious or understated revenues, and so on. Fraud incidents have adverse impact on fraudulent companies' value, often resulting in filing for bankruptcy.

Early detection of financial statement fraud is therefore of eminent importance for companies' stakeholders. Recently, researchers have shown an increased interest in the automatic detection systems using computational intelligence methods [START_REF] West | Intelligent financial fraud detection: A comprehensive review[END_REF]. Despite their good prediction performance, the earlier models suffer from several limitations. A major problem with those intelligent systems is the lack of interpretability, which prevents their adoption in financial industry. Moreover, far too little attention has been paid to modelling uncertainty inherently present in the knowledge of financial analysts and auditors. Indeed, earlier research has shown that expert forecasts significantly outperform traditional statistical models regarding consistency and prediction accuracy [START_REF] Zhang | Half a century of research on Value Line: A comprehensive review[END_REF]. When combined with data analytics methods, expert knowledge was utilized to detect the most serious financial statement frauds [START_REF] Lin | Detecting the financial statement fraud: The analysis of the differences between data mining techniques and experts' judgments[END_REF] and improve the effectiveness of financial audits [START_REF] Tang | Financial fraud detection and big data analytics -Implications on auditors' use of fraud brainstorming session[END_REF]. The aim of this study is to propose an interpretable automatic detection system of financial statement fraud. This system incorporates two main components: [START_REF]Report of the Nations -2018 Global Study on Occupational Fraud and Abuse[END_REF] feature selection component that reduces the search space of fuzzy rules and thus the number of conditions in the antecedents of the rules, and (2) fuzzy rule-based systems optimized by evolutionary and non-evolutionary algorithms to yield a competitive prediction performance and highly interpretable rule base at the same time. This paper attempts to show that these competing, often contradictory, objectives can be met in detecting financial statement fraud.

The rest of this paper is organized in the following way. Section 2 gives a brief overview of the recent advances in the detection of financial statement fraud. Section 3 outlines the research methodology, including data description and methods used. Section 4 presents the results of experiments performed on a dataset of U.S. companies. Section 5 concludes the paper and identifies possible future directions.

Financial Statement Fraud Detection -A Literature Review

Initial research on financial statement fraud detection was concentrated on traditional computational methods, such as logistic regression and shallow neural networks, see [START_REF] West | Intelligent financial fraud detection: A comprehensive review[END_REF] for a comprehensive review. For example, Ravisankar et al. [START_REF] Ravisankar | Detection of financial statement fraud and feature selection using data mining techniques[END_REF] compared a wide range of statistical and data mining methods on a dataset of 202 Chinese companies. The results showed that probabilistic neural networks performed best, while the performance of evolutionary algorithms was significantly improved when applying feature selection first. Detecting financial statement fraud is usually approached as a two-class classification task, categorizing companies as fraudulent or non-fraudulent. However, a multi-class approach was used by [START_REF] Kim | Detecting financial misstatements with fraud intention using multi-class cost-sensitive learning[END_REF] to further discriminate between intentional and unintentional financial statement fraud.

Recently, text mining approaches have been used to increase the prediction performance of traditional financial indicators [START_REF] Throckmorton | Financial fraud detection using vocal, linguistic and financial cues[END_REF][START_REF] Chen | Enhancement of fraud detection for narratives in annual reports[END_REF][START_REF] Loughran | When is a liability not a liability? Textual analysis, dictionaries, and 10-Ks[END_REF]. It was demonstrated that companies' communication with their stakeholders may contain a higher level of uncertainty and misleading statements [START_REF] Throckmorton | Financial fraud detection using vocal, linguistic and financial cues[END_REF]. Negative sentiment present in the managerial statements was considered another important indicator of financial fraud [START_REF] Goel | Do sentiments matter in fraud detection? Estimating semantic orientation of annual reports[END_REF][START_REF] Hajek | Mining corporate annual reports for intelligent detection of financial statement fraud -A comparative study of machine learning methods[END_REF].

Fuzzy rule-based systems is a neglected area of research in this field. In a related study, a fuzzy rule-based system designed by domain experts was proposed to assist auditors in decting managerial frauds [START_REF] Deshmukh | A rule-based fuzzy reasoning system for assessing the risk of management fraud[END_REF]. However, this system was not empirically verified on real-world data and the generated rule base was derived from the subjective judgment of the authors only. To overcome this limitation, a group of experts was created to perform a multi-criteria decision making task using Analytic Hierarchy Process in [START_REF] Huang | Fraud detection using fraud triangle risk factors[END_REF]. Again, this approach was based on expert knowledge only, without utilizing available companies' data. Alden et al. [START_REF] Alden | Detection of financial statement fraud using evolutionary algorithms[END_REF] employed evolutionary algorithms to learn a rule base and demonstrated that such system is more accurate than traditional logistic regression model. Tang et al. [START_REF] Tang | Knowledge-based financial statement fraud detection system: Based on an ontology and a decision tree[END_REF] extracted crisp rules from the data using a C4.5 decision tree algorithm and incorporated them into a financial statement detection ontology. However, none of those studies have considered the interpretability of the rule-based systems as a key objective.

Research Methodology

Dataset

Our dataset comprised 622 companies, out of which 311 were identified as fraudulent by the U.S. SEC (Security and Exchange Commission) between the years 2005 and 2015. The matched sample of non-fraudulent companies was obtained based on industry classification and market capitalization. Companies' annual reports were used as the source of input attributes. Both the financial indicators were calculated and linguistic attributes were obtained using textual analysis. Note that this dataset was previously used in a comparative study of a wide range of machine learning algorithms [START_REF] Hajek | Mining corporate annual reports for intelligent detection of financial statement fraud -A comparative study of machine learning methods[END_REF], so we could easily verify the classification performance of the proposed fuzzy rule-based system. As details on the used input attributes and their desciptive statistics can be found in [START_REF] Hajek | Mining corporate annual reports for intelligent detection of financial statement fraud -A comparative study of machine learning methods[END_REF], here we provide only a briefly introduction of the attributes.

Financial indicators included 32 attributes categorized into nine subsets: (1) firm size (assets and revenues), (2) company reputation (investors' and insiders' shares), (3) profitability (net income, net margin, profitability ratios), (4) activity (growth in working capital, activity ratios), (5) asset structure (fixed capital), (6) business situation (growth in revenue), ( 7) liquidity (working capital), ( 8) leverage (debt ratio), and (9) market value (earnings per share, stock price to earnings, price to earnings, price to book value, reinvestment rate and price to revenue). This selection was based on the theoretical and empirical evidence provided in earlier studies [START_REF] West | Intelligent financial fraud detection: A comprehensive review[END_REF][START_REF] Lin | Detecting the financial statement fraud: The analysis of the differences between data mining techniques and experts' judgments[END_REF][START_REF] Abbasi | Metafraud: a meta-learning framework for detecting financial fraud[END_REF]. Poor financial performance is considered an important incentive for employee engagement in financial statement fraud. Growth in revenue and earnings have been detected as particularly important indicators of future financial statement frauds [START_REF] Hajek | Mining corporate annual reports for intelligent detection of financial statement fraud -A comparative study of machine learning methods[END_REF].

The selection of linguistic indicators was based on the theoretical assumption that fraudsters use uncertain and negative words more frequently in their communication with other company's stakeholders [START_REF] Throckmorton | Financial fraud detection using vocal, linguistic and financial cues[END_REF][START_REF] Loughran | When is a liability not a liability? Textual analysis, dictionaries, and 10-Ks[END_REF][START_REF] Goel | Do sentiments matter in fraud detection? Estimating semantic orientation of annual reports[END_REF]. Therefore, the proportion of those word categories were used as additional input attributes also in this study. More precisely, we calculated the raw frequencies of word categories developed in [START_REF] Loughran | When is a liability not a liability? Textual analysis, dictionaries, and 10-Ks[END_REF] specifically for financial domain. The management discussion section of a company's annual report was used as the source of the communication and all word counts were normalized by the length of the document.

Feature Selection

It is well known that high-dimensional datasets lead to the exponential growth of the number of fuzzy rules. Therefore, feature selection is considered an important task that reduces the search space of fuzzy rules. Moreover, previous systems for financial statement fraud detection have been equipped with feature selection component because reduction in data dimensionality may lead to increase in the accuracy of fraud detection [START_REF] West | Intelligent financial fraud detection: A comprehensive review[END_REF]. In addition, the interpretability of fuzzy rule-based system can be substantially increased by reducing the number of antecedents in the rules.

Here, we used a steady-state genetic algorithm (GA) for wrapper feature selection [START_REF] Casillas | Genetic feature selection in a fuzzy rule-based classification system learning process[END_REF]. This algorithm was designed to select the best feature subsets for fuzzy rule-based systems. The best solution is obtained by using a heuristic algortihm that maximizes the accuracy of a specific learning algorithm. To provide a computationally effective solution, the chosen approach applies the k-nearest neighbour (k-NN) algorithm because it is reportedly highly sensitive to irrelevant features and requires no learning time [START_REF] Casillas | Genetic feature selection in a fuzzy rule-based classification system learning process[END_REF]. The feature selection method is performed in two steps. First, class separability is examined to find the optimal number of features. For this purpose, the Las Vegas filter algorithm is used based on an inconsistency measure as the relevance evaluation of features. Thus, the minimum number of inconsistencies is obtained and the candidate subset is used in the next stage. Second, the subset size determines the chromosome length of GA in the process of a wrapper genetic feature selection. The integer coding is used in the GA feature selection, where each gen in the chromosome represents an attribute. The accuracy of the k-NN algorithm is employed as the fitness function. To avoid overfitting, random resampling of training data repeated five times is used and the performance is measured on the five test classification results. More diversity in the GA population is achieved by adding more than two new chromosomes in each GA generation, while maintaining the advantages of the steady-state reproduction scheme. To keep a balance between exploitation and exploration in GA, the partially complementary crossover operator and two-point crossover with repair operator is employed, respectively. Finally, more diversity in the population is introduced by the uniform mutation operator.

Fuzzy Rule-Based Systems

Fuzzy rule-based systems include a fuzzification process, an inference system and a defuzzification that converts the fuzzy sets into class associated to the input instance [START_REF] Herrera | Genetic fuzzy systems: taxonomy, current research trends and prospects[END_REF]. The inference system is comprised of data base and rule base. The data base contains the set of linguistic terms and corresponding membership functions that define the semantics of the linguistic terms. The rule base contains a set of if-then rules that can be defined as follows.

Let us assume an n-dimensional problem with m training instances classified into M classes. Then, the k-th rule R k in the fuzzy rule base can be defined as follows: In our comparative study, we used four types of fuzzy rule-bases systems: (1) evolutionary fuzzy rule-based classifiers using Michigan learning (the population size is given and each coded rule is represented by an individual in the evolutionary algorithm), ( 2) evolutionary fuzzy rule-based classifiers using iterative learning (the rule base is generated gradually), (3) evolutionary interval-valued fuzzy rule-based classifier (IVTURS), and (4) non-evolutionary fuzzy rule-based classifiers. The algorithms are briefly introduced as follows.

R k : if x 1 is A 1,k and x 2 is A 2,k and . . . and x i is A i,k and . . . and x n is A n,k then class c k with CF k , ( 1
(1) Evolutionary fuzzy rule-based classifiers using Michigan learning:

-The genetic cooperative-competitive learning (GCCL) algorithm [START_REF] Ishibuchi | Performance evaluation of fuzzy classifier systems for multidimensional pattern classification problems[END_REF] employs a GA to optimize the rule base while the data base is fixed. Thus, a computationally effective classifier with interpretable rule base can be obtained. -Genetic programming-based learning of COmpact and ACcurate fuzzy rulebased system for High-dimensional problems (COACH) [START_REF] Berlanga | GP-COACH: Genetic Programming based learning of COmpact and ACcurate fuzzy rule based classification systems for high dimensional problems[END_REF] aims to obtain a compact rule base. Rule confidence and support are combined in the fitness function and a global fitness score is used for the whole population that considers accuracy and the numbers of attributes, antecedents and rules at the same time. The mechanism of token competition is used to maintain diversity in the population of individuals.

(2) Evolutionary fuzzy rule-based classifiers using iterative learning:

-The genetics-based machine learning (GBML) algorithm [START_REF] Ishibuchi | Hybridization of fuzzy GBML approaches for pattern classification problems[END_REF] combines the Michigan learning approach with the Pittsburgh approach (each rule base is handled as an individual). First, the rules base is generated in the Pittsburgh style. This is combined with a prespecified probability given to each rule base as a single iteration of Michigan approach is performed (this is the rule generation and replacement of the worst rules in the current population).

Then, the best rule set is added to the current population to form the next population. This hybrid approach was more effective than GBML performed using the Michigan or Pittsburgh approach separately. This is due to the combination of the high search ability of the Michigan approach with the direct optimization ability of the Pittsburgh approach.

-Structural learning algorithm on vague environment (SLAVE) [START_REF] Garcia | Overview of the SLAVE learning algorithm: A review of its evolution and prospects[END_REF] uses a GA to generate rules iteratively. Each rule is then penalized by eliminating from the training data all those instances that are covered by the previous rule base. The process of generating rules ends when all the instances are eliminated and, therefore, it is not required to set the number of rules a priori. -Steady-state GA for extracting fuzzy classification rules (SGERD) [START_REF] Mansoori | SGERD: A steady-state genetic algorithm for extracting fuzzy classification rules from data[END_REF] uses a nonrandom selection strategy to retain only the best rules in the rule base. Rule confidence and support, as well as accuracy, are considered in the fitness function of the steady-state GA. -New SLAVE (NSVL) [START_REF] Gonzalez | Improving the genetic algorithm of SLAVE[END_REF] was proposed to enhance the effectivity of SLAVE.

Unlike SLAVE, NSVL obtains a complete rule (the antecedent and consequent) in each iteration, thus reducing the required learning time. In the iteration, GA selects the best antecedent for the fixed consequent (class).

(3) Evolutionary interval-valued fuzzy rule-based classifier:

-IVTURS [START_REF] Sanz | IVTURS: A linguistic fuzzy rulebased classification system based on a new Interval-Valued fuzzy reasoning method with TUning and Rule Selection[END_REF] is a generalization of fuzzy rule-based systems, which applies interval-valued fuzzy sets (defined by lower and upper bounds) instead of fuzzy sets in the antecedents of rules. Thus, additional level of uncertainty can be modelled in the rules. IVTURS is performed in three steps. First, a fuzzy association rule-based classification algorithm is used to obtain an initial population of rules. Then, the rule weights are combined with interval matching degrees between the input instances and antecedents. Finally, an evolutionary algorithm is used to tune the rule base and data base with classification accuracy as the fitness function.

(4) Non-evolutionary fuzzy rule-based classifiers:

-The weighted fuzzy (WF) classifier [START_REF] Nakashima | A weighted fuzzy classifier and its application to image processing tasks[END_REF] generates the rule base by considering both the weights and the compatibility of training instances. An incremental learning algorithm is used in the WF classifier for the generated rule base to adjust CF k in order to maximize the classification accuracy. -The rule weight (RW) classifier [START_REF] Ishibuchi | Rule weight specification in fuzzy rule-based classification systems[END_REF] generates the rule base based on an association between the feature space and the space of the classes. Membership degree between a training instance and fuzzy partitions are calculated using a conjunction operator and the matching fuzzy region is assigned to the instance. Thus, the antecedents of fuzzy rules are generated and the instance's class is used as the consequent. The maximum confidence for the antecedent is used as the rule weight. -FURIA [START_REF] Huhn | FURIA: An algorithm for unordered fuzzy rule induction[END_REF] is a fuzzy extension of the RIPPER algorithm. First, a modified RIPPER is performed by gradually adding antecedents so that the highest possible accuracy of the rule is achieved. The rule set is learned for each class using a one-vs-rest decomposition to avoid systematic bias in favor of one class. Then, the rule base is pruned to minimize its description length. Maximum support bound is used as the criterion for the fuzzification of antecedents (the antecedent with the largest rule purity is chosen). The rules are obtained by replacing crisp intervals from RIPPER with fuzzy intervals (trapezoidal membership functions).

Performance Evaluation

To evaluate the classification performance of the fuzzy rule-based systems, we used two measures, namely classification accuracy and misclassification cost (hereinafter referred to as cost). The cost is a particularly important measure in financial statement fraud detection task because false negative classification (fraudulent companies classified as non-fraudulent) is associated with higher cost compared with false positive classification (non-fraudulent companies classified as fraudulent). Here, we use the cost ratio estimate of 1:2 based on the ratio between audit fees and loss incurred by financial statement frauds [START_REF] Hajek | Mining corporate annual reports for intelligent detection of financial statement fraud -A comparative study of machine learning methods[END_REF]. In other words, the cost can be calculated as:

cost = F P R + 2 × F N R, ( 2
)
where F P R is false positive rate and F N R denotes false negative rate.

To evaluate the interpretability of the fuzzy rule-based systems, two measures at the rule base level were used, the number of conditions in the antecedents and the number of rules. Note that there is an inverse relation between the accuracy and interpretability measures and that the optimal trade-off between them depends on the needs of the user [START_REF] Hajek | Predicting corporate investment/non-investment grade by using intervalvalued fuzzy rule-based systems -A cross-region analysis[END_REF].

Experimental Results

First, a 10-fold cross-validation was applied on the dataset to avoid overfitting. For all the evaluation measures, we report average values from the 10 experiments together with standard deviations. To test the results statistically, we performed Wilcoxon signed-rank test.

As noted above, we performed the wrapper feature selection first to obtain the best feature subset for the fuzzy rule-based systems. In our experiments, we followed the recommendations in [START_REF] Casillas | Genetic feature selection in a fuzzy rule-based classification system learning process[END_REF] and set the learning paramaters of the steady-state GA for feature selection as follows: k=3 in the k-NN algorithm, GA with 100 individuals in population and 5,000 generations. The number of attributes to be selected was increased in a stepwise fashion in order to maximize k-NN accuracy. In Table 1, we present the attributes selected in at least 30% of the 10 experiments.

As expected, financial ratios prevailed in the selected feature subsets. This finding also corroborates those presented in earlier research [START_REF] West | Intelligent financial fraud detection: A comprehensive review[END_REF][START_REF] Lin | Detecting the financial statement fraud: The analysis of the differences between data mining techniques and experts' judgments[END_REF][START_REF] Hajek | Mining corporate annual reports for intelligent detection of financial statement fraud -A comparative study of machine learning methods[END_REF][START_REF] Abbasi | Metafraud: a meta-learning framework for detecting financial fraud[END_REF]. On average, 5.6±1.2 features were selected using the steady-state GA method, indicating a substantial reduction rate of 82.5% in the search space of fuzzy rules.

The feature subsets were further applied to predict financial statement fraud using the fuzzy rule-based systems. All experiments were conducted in the KEEL software environment and the settings of the algorithms are presented in Table 2. Note that we fixed the granularity of fuzzy partitions to 5 linguistic labels (membership functions) for all the algorithms to obtain a fair comparison.

Table 3 presents the prediction performance of the used fuzzy rule-based systems in terms of accuracy and cost. The obtained results clearly show that FURIA performed best regarding both performance measures, achieving average accuracy of 86.8% with low cost. This suggests that FURIA performed well on both classes, fraudulent and non-fraudulent, and that it was capable of predicting fraudulent companies correctly. GBML, IVTURS and COACH performed relatively well concerning accuracy, while SLAVE and SGERD provided a relatively low cost, indicating their good performance on the class of fraudulent companies. The Wilcoxon signed-rank test confirmed that FURIA and GBML statistically outperformed the remaining methods. Overall, the results suggest that the evolutionary fuzzy rule-based systems using the Michigan learning was not effective, whereas the gradual generation of the rule base as applied in the iterative learning is a more effective strategy. Method Setting WF learning of the rule weights was applied, learning rate=0.02, #epochs=10 RW product T -norm, penalized certainty factor for rule weights, winning rule for fuzzy reasoning FURIA two rule optimizations, three folds used for pruning GCCL population size of GA=100, #evaluations=10,000, crossover probability pc=1.0, mutation probability pm=0.1 GBML population size in the Michigan and Pittsburgh approaches=100, pc=0.9, #generations=1,000 COACH population size=100, #generations=20,000, pc=0.5, pm=0.2 SLAVE population size=100, #epochs without change=500, pc=0.1, pm=0.5 SGERD #rules per class N =min[(14×n)/(2×M ),20], where n is the number of features and M is the number of classes NSVL population size=100, #epochs without change=500, pc=0.9, pm=1.0 IVTURS minimum support=0.05, maximum confidence=0.8, #evaluations=50,000, population size=100

To compare the prediction performance with state-of-the-art models, we used the models that performed best for the same dataset in [START_REF] Hajek | Mining corporate annual reports for intelligent detection of financial statement fraud -A comparative study of machine learning methods[END_REF]. More specifically, we used Bayesian belief networks (BBN), Decision table/Naïve Bayes (DTNB) and Random forest (RF) for comparative purposes. Note that other models, such as Support vector machines and several ensemble methods, were outperformed in [START_REF] Hajek | Mining corporate annual reports for intelligent detection of financial statement fraud -A comparative study of machine learning methods[END_REF]. We obtained the accuracy of 89.8%, 87.8% and 90.4% for BBN, DTNB and RF, respectively. Only RF performed statistically better than FURIA at P <0.05. The respective cost obtained ranged between 0.340 and 0.406 for the RF and DTNB model, with statistically insignificant differences to FURIA. These results confirm that RF is considered a state-of-the-art method in this domain [START_REF] Patel | An application of ensemble random forest classifier for detecting financial statement manipulation of Indian listed companies[END_REF]. Note that these results must be compared with those in [START_REF] Hajek | Mining corporate annual reports for intelligent detection of financial statement fraud -A comparative study of machine learning methods[END_REF] with caution, given that we used a different feature selection method. Nevertheless, from these results we can deduce that the fuzzy rule-based systems performed relatively well compared with their machine learning competitors. Te number of conditions in the antecedents of the rules remained low across all methods, with GCCL as the best performer (Table 4). This can be attributed to the feature selection component performed in the first step. More complex rules were generated only by the COACH and SLAVE methods, respectively. Regarding the number of rules, SGERD provided the best interpretability at the rule base level (and statistically outperformed the other methods), with less than three rules on average. In contrast, WF failed to achieve reasonable interpretability. The remaining methods performed well concerning the number of rules (below 33 rules), suggesting that even the most accurate methods provided highly interpretable prediction models at the rule base level. For example, the FURIA rules with the highest grade of certainty for both classes were as follows:

-(InsidersShares <= 0. 232(-> Although expert forecasts were present in the set of attributes, note that domain experts were not involved in the design of the rule bases. In other words, the rules were generated automatically using membership functions distributed uniformly over the universe of discourse.

Conclusion

Returning to the hypothesis posed at the beginning of this paper, it is now possible to state that the proposed fuzzy rule-based system with evolutionary feature selection yields both competitive accuracy and desirable interpretability. One of the most significant findings to emerge from this paper is that a relatively high accurate fraud detection can be achieved using only a few fuzzy rules with several conditions in their antecedents. Therefore, the proposed system aspires to become an effective decision support system for auditors and financial analysts.

Finally, several important limitations need to be considered. First, our findings might not be transferable to other countries due to the differences in reporting and auditing. Additional empirical evidence for different countries is therefore recommended in future research. Another limitation might be the fixed granularity at the fuzzy partition level. Here we attempted to provide sufficient granularity, while retaining interpretability of fuzzy partitions at the recommended level [START_REF] Hajek | Predicting corporate investment/non-investment grade by using intervalvalued fuzzy rule-based systems -A cross-region analysis[END_REF]. However, different levels of granularity should be examined to provide empirical evidence. It is also suggested that the ensembles of fuzzy rule-based systems are investigated in future studies because ensemble methods have shown promising results for crisp rule-based systems in the related literature [START_REF] Patel | An application of ensemble random forest classifier for detecting financial statement manipulation of Indian listed companies[END_REF]. Further investigation into semantic interpretability of the fuzzy rule-based systems is also strongly recommended.

  ) where A 1,k , . . . , A 1,n are antecedent fuzzy sets, c k is the consequence class (one of the M classes), R k denotes the k-th rule, k = 1, 2, . . . , N and CF k is the grade of certainty of the k-th rule.

  0.407)) and (GrowthInEPS <= 0.485(-> 0.486)) and (ExpGrowthInRevenue >= 0.319(-> 0.314)) and (PEG >= 0.021(-> 0.019)) => FRAUD=0 (CF = 0.91); -(InsidersShares <= 0.158(-> 0.5)) and (NegativeSentiment >= 0.473(-> 0.465)) => FRAUD=1 (CF = 0.77).

Table 1 .

 1 The most frequently selected features.

	Category	Feature
	corporate reputation insiders' shares
	profitability	net margin
		return on equity
		operating margin
	activity	non-cash working capital / revenue
		cash / revenue
	asset structure	expected growth in revenue (next five years)
	market value	growth in earnings per share (last five years)

Table 2 .

 2 Settings of fuzzy rule-based systems.

Table 3 .

 3 Prediction performance of fuzzy rule-based systems.

	Method Accuracy Cost
	WF	72.9±4.3# 0.639±0.119#
	RW	70.2±4.3# 0.812±0.153#
	FURIA 86.8±4.0 0.400±0.140
	GCCL 78.4±6.9# 0.661±0.222#
	GBML 83.5±4.1 0.512±0.138
	COACH 82.3±5.4# 0.606±0.184#
	SLAVE 79.8±5.9# 0.552±0.167#
	SGERD 81.9±5.9# 0.560±0.189#
	NSVL	78.5±8.2# 0.632±0.241#
	IVTURS 82.7±4.6# 0.561±0.171#
	# significantly worse than the best performer at P <0.05

Table 4 .

 4 Interpretability measures of fuzzy rule-based systems.

	Method No. of Rules No. of Antecedents
	WF	1488.0±67.5 5.6±1.2
	RW	32.9±1.2	5.6±1.2
	FURIA 12.1±2.3	3.1±0.2
	GCCL 28.7±9.8	1.6±0.3*
	GBML 6.3±1.1	5.4±0.7
	COACH 12.2±1.7	7.1±0.8
	SLAVE 7.4±1.3	6.6±0.2
	SGERD 2.9±0.9*	2.0±0.0
	NSVL	5.7±2.5	3.6±0.6
	IVTURS 10.2±1.6	2.1±0.2
	* significantly lower at P <0.05

⋆ Supported by by the scientific research project of the Czech Sciences Foundation Grant No: 19-15498S.


Jiannan Zhao
email: jzhao@lincoln.ac.uk

Xingzao Ma

Shigang Yue
email: syue@lincoln.ac.uk

An LGMD Based Competitive Collision Avoidance Strategy for UAV

Keywords: UAV Collision Avoidance, LGMD, Bio-inspired Neural Network

Building a reliable and efficient collision avoidance system for unmanned aerial vehicles (UAVs) is still a challenging problem. This research takes inspiration from locusts, which can fly in dense swarms for hundreds of miles without collision. In the locust's brain, a visual pathway of LGMD-DCMD (lobula giant movement detector and descending contra-lateral motion detector) has been identified as collision perception system guiding fast collision avoidance for locusts, which is ideal for designing artificial vision systems. However, there is very few works investigating its potential in real-world UAV applications. In this paper, we present an LGMD based competitive collision avoidance method for UAV indoor navigation. Compared to previous works, we divided the UAV's field of view into four subfields each handled by an LGMD neuron. Therefore, four individual competitive LGMDs (C-LGMD) compete for guiding the directional collision avoidance of UAV. With more degrees of freedom compared to ground robots and vehicles, the UAV can escape from collision along four cardinal directions (e.g. the object approaching from the left-side triggers a rightward shifting of the UAV). Our proposed method has been validated by both simulations and real-time quadcopter arena experiments.

Introduction

UAV is one of the most attractive but vulnerable robot platform, which has the potential to be applied in tons of scenarios, such as geography survey, agriculture fertilization, exploration in dangerous or disaster regions, products delivery,

The first and second author contributed equally. This research is funded by the EU HORIZON 2020 project, STEP2DYNA (grant agreement No. 691154) and ULTRA-CEPT (grant agreement No. 778062) shooting photos. Safety of the UAV is always a vital property in a UAV application. Thus, researchers always seeking for better Sense and Avoidance (SAA) technics for UAVs. Classic UAVs use GPS or optic flow [START_REF] Honegger | An open source and open hardware embedded metric optical flow cmos camera for indoor and outdoor applications[END_REF][START_REF] Sabo | Bio-inspired visual navigation for a quadcopter using optic flow[END_REF] to navigate, and onboard distance sensor like ultra sonic, infrared, laser, or a cooperative system to avoid obstacles as reviewed by [START_REF] Yu | Sense and avoid technologies with applications to unmanned aircraft systems: Review and prospects[END_REF]. However, these distance sensors are largely dependent on obstacles' materials, texture and backgrounds' complexity, thus, they can only work in simple and structured environment [START_REF] Chee | Control, navigation and collision avoidance for an unmanned aerial vehicle[END_REF]. Lidar and Vision based methods is more diverse and applicable. One popular vision based method is to detect and locate obstacles in a reconstructed map, mark out the frontiers of the obstacles as banded fields in the map, and then use specified pathfinding algorithm (e.g. heuristic algorithm) to generate safe trajectories to avoid collision [START_REF] Richter | Polynomial trajectory planning for aggressive quadrotor flight in dense indoor environments[END_REF][START_REF] Bachrach | Autonomous flight in unknown indoor environments[END_REF][START_REF] Achtelik | Stereo vision and laser odometry for autonomous helicopters in gps-denied indoor environments[END_REF]. This is also named as Simultaneous Localization and Mapping(SLAM), but its high demand of computational burden prevent it from small or micro UAVs.

On the other hand, bio-inspired vision based collision detecting methods are standing out for their efficiency. For example, Optic Flow (OF) is a widely used vision based motion detecting method inspired by biological mechanism in flies and bees [START_REF] Serres | Optic flow-based collision-free strategies: From insects to robots[END_REF]. It is also introduced in collision avoidance technology, e.g. Zufferey [START_REF] Zufferey | Toward 30-gram autonomous indoor aircraft: Visionbased obstacle avoidance and altitude control[END_REF] applied 1D OF sensor onto a 30g light weight fixed wing UAV and achieved automatic obstacle avoidance in indoor(GPS denied) structured environment. And later in 2009 [START_REF] Beyeler | Vision-based control of near-obstacle flight[END_REF] their group achieved autonomous avoidance towards trees with 7 OF sensor on a fixed wing platform. Griffiths [START_REF] Griffiths | Obstacle and terrain avoidance for miniature aerial vehicles[END_REF] used optical mouse (key-point matching) converted OF sensor to fly through Canyon, besides the OF sensor, it also integrated a laser ranger for directly approaching obstacles. Serres [START_REF] Serres | A vision-based autopilot for a miniature air vehicle: joint speed control and lateral obstacle avoidance[END_REF] used a pair of EMD based OF sensor to avoid lateral obstacles for a hovercraft. Sabo [START_REF] Sabo | Bio-inspired visual navigation for a quadcopter using optic flow[END_REF] applied OF onto quadcopters and repeated some benchmark experiments to analysis the behaviours for honeybee-like flying robot, however, the algorithm was still computed off board. Stevens [START_REF] Stevens | Vision based forward sensitive reactive control for a quadrotor vtol[END_REF] achieved collision avoidance in cluttered 3D environments.

Lobula Giant Movement Detector (LGMD) is another bio-inspired neural network inspired by Locusts vision system, and especially, superior in detecting approaching obstacles and avoiding imminent collisions. Compared to Optic Flow, LGMD is more specialised for detecting directly approaching obstacles and eliminate redundant image difference caused by shifting things and backgrounds. The LGMD neuron and its presynaptic neural network has been modeled [START_REF] Rind | Neural network based on the input organization of an identified neuron signaling impending collision[END_REF] and promoted by many researchers [START_REF] Fu | Collision selective lgmds neuron models research benefits from a vision-based autonomous micro robot[END_REF][START_REF] Fu | Shaping the collision selectivity in a looming sensitive neuron model with parallel on and off pathways and spike frequency adaptation[END_REF][START_REF] Yue | Collision detection in complex dynamic scenes using an lgmdbased visual neural network with feature enhancement[END_REF]. As a collision detecting model, LGMD has been introduced to mobile robots [START_REF] Hu | A bio-inspired embedded vision system for autonomous micro-robots: the lgmd case[END_REF][4], embedded systems [START_REF] Fu | hu: Bio-inspired collision detector with enhanced selectivity for ground robotic vision system[END_REF][START_REF] Hu | Development of a bio-inspired vision system for mobile micro-robots[END_REF], hexapod walking robot [START_REF] Čížek | Neural based obstacle avoidance with cpg controlled hexapod walking robot[END_REF],blimp [START_REF] Bermudez I Badia | A fly-locust based neuronal control system applied to an unmanned aerial vehicle: the invertebrate neuronal principles for course stabilization, altitude control and collision avoidance[END_REF] and cars [START_REF] Yue | A bio-inspired visual collision detection mechanism for cars: Optimisation of a model of a locust neuron to a novel environment[END_REF][START_REF] Yue | A synthetic vision system using directionally selective motion detectors to recognize collision[END_REF].

Basic LGMD model provide the threat level of collision in the whole field of view (FoV), but it is not enough to make wise avoidance behaviour, hence, early research generated randomly turn direction in mobile robots [START_REF] Hu | A bio-inspired embedded vision system for autonomous micro-robots: the lgmd case[END_REF]. Shigang [START_REF] Yue | Reactive direction control for a mobile robot: a locust-like control of escape direction emerges when a bilateral pair of model locust visual neurons are integrated[END_REF] divided the field of view into two bilateral halves, and discussed both winnertake-all and steering-wheel network in direction control system of the mobile robot. Compared to mobile robots, UAV has more degree of freedom, and is more vulnerable during flight. In the extremely limited literature of LGMD re-search on UAV platforms, Salt [START_REF] Salt | Obstacle avoidance with lgmd neuron: towards a neuromorphic uav implementation[END_REF] implemented a neuromorphic LGMD model using recording from a UAV platform, and divided the FoV into half twice for direction information. But there is no real-time flight conducted. Our previous research has proved the applicability of LGMD on Quadcopter [START_REF] Zhao | A bio-inspired collision detector for small quadcopter[END_REF] real-time flight and collision avoidance. Previously, the quadcopter can only avoid obstacles by randomly turn left or right in horizontal plane. To acquire the information about the coming direction of imminent obstacles, this research proposed a new image partition strategy, especially for LGMD application on UAVs, and a corresponding steering method for 3D avoidance behaviour. Both video simulation and real-time flight demonstrated the performance of this method.

Model Description

LGMD Process

The LGMD process algorithm used in this paper is inherited from our previous research [START_REF] Zhao | A bio-inspired collision detector for small quadcopter[END_REF]. The LGMD process is composed of five groups of cells, which are Pcells (photoreceptor), I-cells (inhibitory), E-cells (excitatory), S-cells (summing) and G-cells (grouping), compared to previous model, we added four single competitive LGMD cells representing LGMD output of four sections: Left, Right, Up, and Down. The image is divided as shown in Fig. 1. Fig. 1: A schematic illustration of the proposed LGMD based competitive neuron network for collision detection. [#] denotes the inherited LGMD process as described in our previous research [START_REF] Zhao | A bio-inspired collision detector for small quadcopter[END_REF].

The first layer of the neuron network is composed of P cells, which are arranged in a matrix, formed by luminance change between adjacent frames. The output of a P cell is given by:

P f (x, y) = L f (x, y) -L f -1 (x, y) (1)
where P f (x, y) is the luminance change of pixel(x, y) at frame f , L f (x, y) and L f -1 (x, y) are the luminance at frame f and the previous frame.

The output of the P cells forms the input of the next layer and is processed by two different types of cells, which are I (inhibitory) cells and E (excitatory) cells. The E cells pass the excitatory flow directly to S layer so that the E cells has the same value to its counterpart in P Layer; While the I cells pass the inhibitory flow convoluted by surrounded delayed excitations.

The I layer can be described in a convolution operation:

[I] f = [P ] f ⊗ [w] I (2)
where [w] I is the convolution mask representing the local inhibiting weight distribution from the centre cell of P layer to neighbouring cells in S layer, a neighbouring cell's local weight is reciprocal to its distance from the centre cell. To adapt fast image motion during UAV flight, [w] I is set differently to it in mobile robot [START_REF] Hu | A bio-inspired embedded vision system for autonomous micro-robots: the lgmd case[END_REF], the inhibition radius is expanded to 2 pixels:

[w] I = 0.25

       1 √ 8 1 √ 5 1 2 1 √ 5 1 √ 8 1 √ 5 1 √ 2 1 1 √ 2 1 √ 5 1 2 1 0 1 1 2 1 √ 5 1 √ 2 1 1 √ 2 1 √ 5 1 √ 8 1 √ 5 1 2 1 √ 5 1 √ 8        (3)
The next layer is the Sum layer, where the excitation and inhibition from the E and I layer is combined by linear subtraction, and after summation. Next, Group layer is involved to reduce the noise caused by sporadic image change or backgrounds. Detailed equation and parameters can be found in our previous work [START_REF] Zhao | A bio-inspired collision detector for small quadcopter[END_REF].

When it comes to G layer, The unnormalized membrane potential of four C-LGMDs are Calculated respectively:

U LGM D0 = x min[Diag1,Diag2] y=0 | G f (x, y)| (4)
D LGM D0 = x max[Diag1,Diag2] y=0 | G f (x, y)| (5)
L LGM D0 = x y<=Diag2 y>=Diag1 | G f (x, y)| (6) R LGM D0 = x y<=Diag1 y=>Diag2 | G f (x, y)| (7) (8)
where Diag1, Diag2, denote the coordinates in y axis of the two diagonals, and G f (x, y) is the cells value of G layer, as illustrated in Fig. 2. For more details about the process from P f (x, y) to G f (x, y) please looks in our previous work [START_REF] Zhao | A bio-inspired collision detector for small quadcopter[END_REF].

Fig. 2: Image dividing method. The image scene is split through the diagonal.

Previously, the membrane potential of the LGMD cell K f 0 is the summation of every pixel in G layer:

K f 0 = x y | G f (x, y)| (9)
Now it also equals to the summation of the four C-LGMD neurons:

K f 0 = U LGM D + D LGM D + L LGM D + R LGM D (10)
and then K f is adjusted in range (0, 255) by a sigmoid equation:

κ f = tanh( K f 0 -n cell C 1 ) n cell C 2 × 255 (11)
where C 1 and C 2 are constants to shape the normalizing function, limiting the excitation κ f varies within [0, 255], n cell represents the total number of pixels in one frame of image. The membrane potential of the four C-LGMDs, is also limited in (0, 255) by calculating their proportion in K f 0 , instead of modified with sigmoid function again:

U LGM D = U LGM D0 K f 0 × κ f (12)
D LGM D = D LGM D0 K f 0 × κ f (13)
L LGM D = L LGM D0 K f 0 × κ f (14) R LGM D = R LGM D0 K f 0 × κ f (15) (16)
If κ f exceeds its threshold, then an LGMD spike is produced:

S spike f = 1, if κ f T s 0, otherwise. (17)
An impending collision is confirmed if successive spikes last consecutively no less than n sp frames:

C LGM D f =      1, if f f-nsp S spike f n sp 0, otherwise. (18)
And then, based on the result of the competitive C-LGMDs, DCMD will switch to the corresponding escape command, and the command is sent through USART interface to the flight control system. The process from DCMD to PID based motor control system is shown in pseudocode Algorithm 1.

System Overview

In this section, the outline of the whole system is described. A system composed of Quadcopter, embedded LGMD detector, Ground Station, Remote and auxiliary sensors is depicted in Fig. 3 . Luminance information is collected by the camera on the detector board, and then involved into the LGMD algorithm, the output command is passed through a USART port into the flight control to monitor avoiding tasks.

Quadcopter Platform

The UAV platform used in this research is a customized quadcopter with the skeleton size of 33cm between diagonally rotors. The flight control module we used is based on a STM32F407V and provides 5 USART interface for extra peripheral. Multiple sensors are applied for data collection and enhance the stability of the quadcopter, including an IMU(Inertial Measurement Unit), an ultrasonic sensor, an optic flow sensor and the LGMD detector, as illustrated in Fig. 3. The Pix4Flow optic flow module [START_REF] Honegger | An open source and open hardware embedded metric optical flow cmos camera for indoor and outdoor applications[END_REF] is occupied as a position and velocity feedback in horizontal plane. The flight control module works as the central controller to combine the other parts together. It receives source data from the embedded IMU module(MPU6050), the Pix4flow optic flow sensor, and the LGMD detector, calculates out the PWM(Pulse-Width Modulation) values as the output to the four motors and it also sends back real time data for analysis through the nRF24L01 module.

Experiments and Results

To verify the performance of the proposed algorithm, both video simulation and arena real-time flight are conducted.

Video Simulation

The algorithm is firstly implemented on matlab and tested by a series recorded video, to verify whether the algorithm can distinguish stimulus from different directions. The results in Fig. 4 indicate that the new network is able to respond differently towards coming objects from different directions.

Hovering & Features Analysis

To further analyze the performance on quadcopter platform, we transplanted the algorithm the embedded LGMD detector, and mounted the detector onto the quadcopter, stimulated the detector with test patterns while the quadcopter hovering in the air. Object is manually pushed towards the detector from four direction respectively, and each direction repeated 10 times. Fig. 5 is an example of the trial scene, in which object is pushed towards the detector from left.

According to the results in Fig. 6, the four competitive LGMD distinguished the coming direction of the object accurately. In all the four types of trials, when LGMD exceeds its threshold, the C-LGMD indicated the main direction is leading the other average values, even if the lowest performance (lower boundary of the shadow).

Arena Real-time Flight

Finally, real-time flight and obstacle avoidance experiments are conducted to test the performance and robustness of the proposed directionally obstacle avoiding method. Trials reflecting four directions of coming object are set in two types: obstacles on the left and right side or on the upside and downside on the UAV's route. The quadcopter is first challenged by a static obstacle and then challenged by a dynamic intruder. The results showed that the system is able to make smart escape behaviour based on the coming direction of the obstacle. The trajectories of these trials have been extracted and overlaid on a screenshot from the video, as shown in Fig. 7. Trajectories are detected by a python program using background subtractor [START_REF] Zivkovic | Improved adaptive gaussian mixture model for background subtraction[END_REF] and template matching [START_REF] Lewis | Fast normalized cross-correlation[END_REF] method, and then printed onto a screen shot from the recorded video.

Conclusion

To conclude, a novel competitive LGMD and corresponding UAV control algorithm is proposed to address practical problems meet in UAV's LGMD application. Both simulation and realtime flight experiments were conducted to analyze the proposed method, and the results showed high robustness. Based on the proposed competitive LGMD, quadcopter's Real-time 3D collision avoidance is achieved in indoor environment. For the future work, totally autonomous flight in a larger arena should be take to analyze the boundary of this new method.

Algorithm 1 : 3 if minDirection ≥ DLGMD then 4 minDirection ← DLGMD 5 if minDirection ≥ LLGMD then 6 minDirection ← LLGMD 7 if minDirection ≥ RLGMD then 8 minDirection ← RLGMD 9 if

 13456789 Escape direction steering algorithm Input: four competitive C-LGMDs refers to stimulus in the FoV: ULGMD, DLGMD, LLGMD, RLGMD Output: expected quadcopter speed in x,y,z axis (PID motor control input): exp sp[x], exp sp[y], exp sp[z] 1 while C LGM D f minDirection = ULGMD then 10 set exp sp[z] = speed0; 11 else if minDirection = DLGMD then 12 set exp sp[z] = -speed0; 13 else if minDirection = LLGMD then 14 set exp sp[y] = speed0; 15 else if minDirection = RLGMD then 16 set exp sp[y] = -speed0;

Fig. 3 :

 3 Fig. 3: The structure of the quadcopter platform.

Fig. 4 :

 4 Fig. 4: Simulation results with snapshot.(a),(b),(c),(d) are membrane potential in C-LGMDs, toward upside, downside, left side, and right side stimuli respectively.

Fig. 5 :

 5 Fig. 5: Hovering experiments scene

Fig. 6 :

 6 Fig. 6: Average membrane potential during hovering tests.(a),(b),(c),(d), reflected the average membrane potential of the four competitive LGMD neuron in trials. The shadow is the continuous error of the C-LGMD of the main direction.


Ying Bai
email: ybai@jcsu.edu

Dali Wang
email: dwang@cnu.edu

Using Shallow Neural Network Fitting Technique to Improve Calibration Accuracy of Modeless Robots

Keywords: Neural Networks, Modeless Robots, Robots Calibrations. ' . The

This paper describes a technique for the position error estimations and compensations of the modeless robots and manipulators calibration process based on a shallow neural network fitting function method. Unlike traditional model-based robots calibrations, the modeless robots calibrations do not need to perform any modeling and identification processes. Only two processes, measurements and compensations, are necessary for this kind of robots calibrations. By using the shallow neural network fitting technique, the accuracy of the position error compensation can be greatly improved, which is confirmed by the simulation results given in this paper. Also the comparisons among the popular traditional interpolation methods, such as bilinear and fuzzy interpolations, and this shallow neural network technique, are made via simulation studies. The simulation results show that more accurate compensation result can be achieved using the shallow neural network fitting technique compared with the bilinear and fuzzy interpolation methods.

Introduction

The prerequisite requirement of the robotic modeless calibration is the successful self-calibration of the camera [START_REF] Mooring | Fundamentals of Manipulator Calibration[END_REF][START_REF] Penna | Camera Calibration: A Quick and Easy Way to Determine the Scale Factor[END_REF] or other measurement device, such as laser tracking system [START_REF] Bai | Design and Implementation of a Control System for a Laser Tracking Measurement System[END_REF]. Both internal and external parameters of the camera need to be calibrated accurately [START_REF] Weng | Camera Calibration with Distortion Models and Accuracy Evaluation[END_REF][START_REF] Du | Self Calibration of the Intrinsic Parameters of Cameras for Active Vision Systems[END_REF]. Then the modeless robot calibration is divided into two steps [START_REF] Zhuang | Camera-Aided Robot Calibration[END_REF].

The first step is to measure the position errors for all grid points on a standard calibration board, which is installed on the robot's workspace. A calibrated camera is used to find 4 neighboring position errors. This process can be considered as a measurement process, which is shown in Fig 1 .   At each grid point, a calibrated camera is used to check the position errors of the end-effector of the robot. In Fig 1, the desired position of the grid point 0 is (x 0 , y 0 ), and the actual position of the robot end-effector is (x 0 ' , y 0 ' ). The position errors for this grid point are e x = x 0 -x 0 ' , and e y = y 0 -y 0 on these grid points will be measured and stored in the memory for future usage.

In the second step, the robot's end-effector is moved to a target position that is located in the range of the workspace. The target position error could be found by an interpolation technique using the stored 4-neighboring grid position errors around the target position, which were obtained from the first step. Finally, the target position could be compensated with the interpolation results to obtain more accurate positions. J. Triantafilis and D. Suzana et al reported approaches of using fuzzy interpolation methods to estimate the soil layer and geographical distributions for GIS database [START_REF] Triantafilis | Creation and Interpolation of Continuous Soil Layer Classes in the Lower Namoi Valley[END_REF][START_REF] Suzana | Space, time, and dynamics modeling in historical GIS databases: A Fuzzy Logic Approch[END_REF]. F. Song et al described a fuzzy logic methodology for 4dimensional (4D) systems with optimal global performance using enhanced cell state space [START_REF] Song | A Fuzzy Logic Controller Design Methodology for 4D System with Optimal Global Performance Using Enhanced Cell State Space Based Best Estimate Directed Search Method[END_REF]. The most popular interpolation techniques applied in the position compensations of the modeless robotic calibration include the bilinear interpolation and fuzzy interpolation methods; both methods can achieve satisfactory interpolation results for general calibration process [START_REF] Penna | Camera Calibration: A Quick and Easy Way to Determine the Scale Factor[END_REF][START_REF] Bai | On the Comparison of Model-Based and Modeless Robotic Calibration Based on a Fuzzy Interpolation Method[END_REF]. The bilinear interpolation technique assumes that the error of the target position is located on the surface that is constructed by the position errors of 4-neighboring grid points around the target position [START_REF] Song | A Fuzzy Logic Controller Design Methodology for 4D System with Optimal Global Performance Using Enhanced Cell State Space Based Best Estimate Directed Search Method[END_REF]. The fuzzy interpolation method assumes that the workspace can be divided into a group of smaller cells, and the target positions can be obtained by interpolating position errors on 4 neighboring grid points in cells via fuzzy inference system [START_REF] Bai | On the Comparison of Model-Based and Modeless Robotic Calibration Based on a Fuzzy Interpolation Method[END_REF]. Consequently, the target point's errors are estimated according to the equations of the error surface or fuzzy inference techniques. Since the actual position errors are randomly distributed, and it is impossible to pinpoint a position on the error surface at any given moment, the traditional interpolation technique is unable to provide an accurate estimation of the position errors. Fuzzy error interpolation technique utilizes the fuzzy inference system to estimate the position errors, which is consistent with the random distributed nature of position errors. The position errors can be considered as a fuzzy set at any given moment of the time. The fuzzification process takes into account of a range of error rather only a crisp error value. Therefore, the fuzzy error interpolation technique has the fundamentals to improve error estimation results.

However, more and more robots calibration techniques developed by using artificial neural network (ANN) are reported in recent years [START_REF] Xu | Calibrating a modular robotic joint using neural network approach[END_REF][START_REF] Wu | Hand-Eye Calibration and Inverse Kinematics of Robot Arm Using Neural Network[END_REF][START_REF] Gordonwells | Vision-based robot positioning using neural networks[END_REF][START_REF] Nguyen | A calibration method for enhancing robot accuracy through integration of an extended Kalman filter algorithm and an artificial neural network[END_REF][START_REF] Xu | Complete calibration of industrial robot with limited parameters and neural network[END_REF][START_REF] Zouaoui | 2D visual servoïng of wheeled mobile robot by neural networs[END_REF][START_REF] Tao | Calibration of industry robots with consideration of loading effects using Product-Of-Exponential (POE) and Gaussian Process (GP)[END_REF][START_REF] Luo | Low cost solution for calibration in absolute accuracy of an industrial robot for iCPS applications[END_REF].

Target Camera Robot Calibration Board (X 0 , Y 0 ) (X 1 , Y 0 ) (X ' 0 , Y ' 0 ) (X 0 , Y 1 )

Grid Points

Cell

This study describes a technique for the position error estimations and compensations of the modeless robots and manipulators calibration process based on a shallow neural network (SNN) fitting function method. A feedforward neural network or called shallow neural network fitting technique is utilized to estimate the position errors based on errors on the 4 neighboring grind points. With this method, the calibration accuracy can be significantly improved and the calibration process can also be greatly simplified. A comparison among three different error interpolation methods, bilinear, fuzzy interpolation and SNN, are performed, and the simulation results indicate that the SNN method outperforms the other methods. This paper is organized in 4 sections. After this introduction section, the principle of the fuzzy interpolation technique is provided in sections 2. A simulation is given in section to illustrate the effectiveness of the SNN technique. Section 4 presents the conclusion. Each square that is defined by 4 grid points is called a cell; and each cell is divided into 4 equal smaller cells, which are NW, NE, SW and SE, respectively (Fig 2a).

Fuzzy Error Interpolation Method

Overview of the Fuzzy Interpolation System

The position error at each grid point is defined as P 1 , P 2 , P 3 and P 4 .

For the fuzzy inference system, we apply the fuzzy error interpolation method in

Membership Functions

In this study, the distance between two neighboring grid points on the standard calibration board is 20 mm in both x and y directions, which is a standard value for a midsize calibration workspace. The calibration board includes a total of 20 by 20 cells, which is equivalent to a 400 by 400 mm space. The gaussian-bell waveforms are selected as the shape of the membership functions for both inputs (Fig 3a) in x and y directions. The ranges of inputs are between -10 mm and 10 mm (20 mm intervals). H. Zhuang and X. Wu reported a histogram method to estimate the optimal membership function distribution [START_REF] Zhuang | Membership Function Modification of Fuzzy Logic Controllers with Histogram Equalization[END_REF]. However in our case, a gaussian-bell shape is selected due to the fact that most errors in real world match this distribution. We use W and E to represent the location of inputs in x direction, N and S to represent the location of inputs in y direction. Fig 3b shows an example of the output membership functions, which are related to the simulated random errors at neighboring grid points. Each P xi and P yi correspond to the position error at the ith grid point in x and y directions, respectively. During the design stage, all output membership functions are initialized to a gaussian waveform with a mean of 0 and a range between -0.5 and 0.5 mm, which is a typical error range for this workspace in robotic calibration (Fig 3c). These output membership functions will be determined based on the errors of the neighboring grid points around the target in the workspace as mentioned above. For example, during the compensation process if the input position in the x direction is in the NW area of a cell, the associated output membership function should be modified based on the position error in the NW grid point P 1 . This modification is equivalent to shifting the P x1 Gaussian waveform (Fig 3b ) and allowing the center of that waveform to be located at x 0 = the position error value of the P 1 in the x direction. A similar modification should be performed for the position error in the y direction. It can be seen from Fig 3b that for the position compensation process, the performance loss would be significant if the default membership function is utilized, which is shown in Fig 3c.

NW

Control Rules

The control rules shown in  If e x is W and e y is N, ee x is P x1 and ee y is P y1 (NW).

 If e x is W and e y is S, ee x is P x3 and ee y is P y3 (SW).

 If e x is E and e y is N, ee x is P x2 and ee y is P y2 (NE).

 If e x is E and e y is S, ee x is P x4 and ee y is P y4 (SE).

Each Pi should be considered as a combination of two error components, P xi and P yi , which are corresponding to errors in both x and y directions. The error on NW grid point should take more weight if the target position (input) is located inside the NW area on a cell. Similar conclusion can be derived for errors on SW, NE and SE grid points.

Fuzzy Inference System

The fuzzy inference system implemented in this study is an on-line one. This means that output of the fuzzy system is not obtained from the pre-defined lookup table, but from a real time fuzzy inference calculation that utilizes the pre-defined input membership functions and the real time position errors. The input error variables can be expressed as a label set L(E), where E is a linguistic input variable:

L(E) = {NW, NE, SW, SE} (2)
Assume that u i is the membership function, U i the universe of discourse and m the number of contributions, the traditional output of the fuzzy inference system can be represented as:

u =      m i i m i i i u U u 1 1 ) ( (3)
where u is the current crisp output of the fuzzy inference system. Equation ( 3) is obtained by using the Center-Of-Gravity method (COG). In this study, both u i and U i in the output membership functions are randomly distributed variables and the actual values of these variables depend upon the position errors of four neighboring grid points around the target position. These relationships can be expressed as: u i = F i (P 1 , P 2 , P 3 , P 4 ) [START_REF] Weng | Camera Calibration with Distortion Models and Accuracy Evaluation[END_REF] U i = Q i (P 1 , P 2 , P 3 , P 4 ) [START_REF] Du | Self Calibration of the Intrinsic Parameters of Cameras for Active Vision Systems[END_REF] where F i and Q i are randomly distributed functions. Substituting (4) and ( 5

In ( 6), both F i and Q i will not be determined until the fuzzy error interpolation technique is applied in an actual compensation process, which means that this fuzzy inference system is an on-line process.

Simulation Results

Extensive simulation study has been performed to illustrate the effectiveness of the proposed SNN technique in comparison to bilinear and fuzzy interpolation methods. The uniform distributed random error is used for the simulation study due to its popularity. We estimate the valid workspace of the robot to be 400 x 400 mm. We choose the size of each cell to be 20 x 20 mm after taking into consideration the repeatability of the robot. Consequently, the workspace consists of 20 cells in each direction. In our simulation, we simulate actual position based on in this format:

P a = (x i , y i ) + A*rand(x, y) (7)
where (x i , y i ) is the nominal position, and A is the amplitude of the uniformly distributed noise in the interval (-0.5, 0.5) for the random component. We implemented MATLAB  Neural Network Toolbox  to perform these simulation studies. It can be seen that both mean errors and maximum errors of SNN are much smaller than those of fuzzy interpolation and bilinear methods. For the uniform distributed error, the mean error of the SNN method is approximately 60% to 70% smaller compared with those of bilinear and fuzzy interpolation methods. The maximum errors of the SNN technique is about 40% to 50% smaller than those of bilinear and fuzzy interpolation techniques.

The simulated results show the effectiveness of the SNN fitting method in reducing the position errors in the modeless robot compensation process.

Conclusion and Summary

A shallow neural network fitting method is presented in this paper. The compensated position errors in a modeless robot calibration can be greatly reduced by the proposed technique. Simulation results demonstrate the effectiveness of the proposed shallow neural network (SNN) fitting method. One typical error model, uniform distributed error, is utilized for comparison and simulation study purpose. This SNN technique is ideal for the modeless robot position compensation, especially the high accuracy (<10 m) robot calibration process.

Fig. 1 .

 1 Fig. 1. Setup of the Modeless Calibration.

Fig 2

 2 Fig 2 shows the definition of the fuzzy error interpolation inference system.Each square that is defined by 4 grid points is called a cell; and each cell is divided into 4 equal smaller cells, which are NW, NE, SW and SE, respectively (Fig2a).

Fig. 2 .

 2 Fig. 2. Definition of the Fuzzy Error Interpolation System two dimensions separately, so the inputs to the fuzzy inference system are e x and e y and the outputs are ee x and ee y (Fig 2b). The control rules are shown in Fig 2c, and will be discussed following the discussion of membership functions.

Fig. 3 .

 3 Input and output membership functions. The input membership functions for both x and y directions and the predefined output membership functions are shown in Fig 3. The predefined output membership functions are used as a default one, and the final output membership function will be obtained by shifting the default one by the actual error values on the grid points.

  Fig 2c can be interpreted as follows after the output

  Fig 4 shows the training, validation and testing process of this fitting SNN. The Levenberg-Marquardt algorithm is used for this training.After the first epoch training process, the mean square error (MSE) is reduced below 0.01, and the best validation result is in the first epoch (0.0096). The testing result is also good with all errors being about 0.01. Fig.5shows a comparisons in mean error, maximum error and STD values among bilinear, fuzzy interpolation and SNN technique in the histograms.

Fig. 4 .

 4 Fig. 4. Performance of the used SNN (training, validation and testing).

Fig. 5 .

 5 Fig. 5. Comparison among bilinear, fuzzy interpolation and SNN.


Qinbing Fu

Nicola Bellotto

F Claire Rind

Shigang Yue
email: syue@lincoln.ac.uk

A Visual Neural Network for Robust Collision Perception in Vehicle Driving Scenarios *

Keywords: LGMD, collision detection, adaptive inhibition mechanism, vehicle crash, complex dynamic scenes

This research addresses the challenging problem of visual collision detection in very complex and dynamic real physical scenes, specifically, the vehicle driving scenarios. This research takes inspiration from a large-field looming sensitive neuron, i.e., the lobula giant movement detector (LGMD) in the locust's visual pathways, which represents high spike frequency to rapid approaching objects. Building upon our previous models, in this paper we propose a novel inhibition mechanism that is capable of adapting to different levels of background complexity. This adaptive mechanism works effectively to mediate the local inhibition strength and tune the temporal latency of local excitation reaching the LGMD neuron. As a result, the proposed model is effective to extract colliding cues from complex dynamic visual scenes. We tested the proposed method using a range of stimuli including simulated movements in grating backgrounds and shifting of a natural panoramic scene, as well as vehicle crash video sequences. The experimental results demonstrate the proposed method is feasible for fast collision perception in real-world situations with potential applications in future autonomous vehicles.

Introduction

Autonomous vehicles, though still in early stages of development, have demonstrated huge potential for shaping our future lifestyles and benefiting a variety of human activities. Before well serving the human society, there is one critical issue to solve -trustworthy collision perception. Nowadays the number of fatalities by road crashes still remains high. To improve driving safety, the cutting-edge approaches for vehicle collision detection, such as radar, GPS-based methods and normal vision sensors, are often ineffective in terms of reliability, cost, energy consumption or size. For ground vehicle collision detection, the most effective systems comprise automated collision avoidance with emergency steering and braking assistance, as well as active lane keeping systems, e.g. [START_REF] Eichberger | Contributions to Primary, Secondary and Integrated Traffic Safety[END_REF]. The vast majority of vision-based methods implement object-and-scene segmentation, estimation or classification algorithms [START_REF] Mukhtar | Vehicle detection techniques for collision avoidance systems: A review[END_REF]. The state-of-the-art visual sensors like RGB-D and event-driven cameras can provide vehicles with more abundant visual features compared to normal cameras. However, these solutions are either computationally costly or heavily reliant on the specific sensors. A new type of reliable, low-cost, energyefficient and miniaturised collision detection techniques is demanded for future autonomous vehicles.

Nature provides a rich source of inspiration for designing artificial visual systems for collision perception and avoidance (e.g. [4, 6-8, 16, 19]). Insects have compact visual brains that deal with motion perception. For instance, locusts can fly for a long distance in very dense swarms without collision; also nocturnal insects successfully forage in the forest at night free of collision. These naturally developed visual systems are perfect sensory models for collision detection and avoidance. In the future, vehicles with or without a driver should possess similar ability to navigate as effectively as animals do.

Locusts are well known for fast collision avoidance behaviour on the basis of visual cues. A group of lobula giant movement detectors (LGMDs) has been found by biologists (e.g. [START_REF] Rind | Two identified looming detectors in the locust: ubiquitous lateral connections among their inputs contribute to selective responses to looming objects[END_REF]). The LGMD1 (namely LGMD in this paper) was firstly identified as a moving objects detector and then gradually recognised as a large-field looming sensitive neuron, which responds most vigorously to objects quickly approaching rather than other kinds of movements [START_REF] Rind | Two identified looming detectors in the locust: ubiquitous lateral connections among their inputs contribute to selective responses to looming objects[END_REF]. The neuromorphology of LGMD is illustrated in Fig. 1. Such a fascinating neuron has been computationally modelled as collision selective visual neural networks or models (e.g. [START_REF] Bermudez I Badia | Non-linear neuronal responses as an emergent property of afferent networks: a case study of the locust lobula giant movement detector[END_REF][START_REF] Keil | A neural model of the locust visual system for detection of object approaches with real-world scenes[END_REF][START_REF] Silva | Computational model of the LGMD neuron for automatic collision detection[END_REF][START_REF] Yue | Collision detection in complex dynamic scenes using a lgmd based visual neural network with feature enhancement[END_REF]), and applied in mobile machines like ground robots [START_REF] Fu | Collision selective LGMDs neuron models research benefits from a vision-based autonomous micro robot[END_REF][START_REF] Fu | Shaping the collision selectivity in a looming sensitive neuron model with parallel on and off pathways and spike frequency adaptation[END_REF][START_REF] Hu | Bio-inspired embedded vision system for autonomous micro-robots: The LGMD case[END_REF] and UAVs [START_REF] Salt | Obstacle avoidance with lgmd neuron: Towards a neuromorphic uav implementation[END_REF], and also embodied in hardware implementations like the FPGA [START_REF] Meng | A modified model for the lobula giant movement detector and its FPGA implementation[END_REF]. These works have partially reproduced the LGMD's responses and demonstrated that it features efficient neural computation for quick and reliable looming or collision sensing. However, the performance of these methods is still restricted by the background complexity, i.e., the real physical environmental noise including irrelevant motion cues greatly affect the looming detection.

For outdoor vehicle driving scenarios, there are many challenges for artificial collision detection vision systems. The more unpredictable and dynamic environments include optical flows caused by ego-motions with the movements of lane markings or rapidly approaching ground shadows, and situations like nearby vehicles approaching or surpassing. These are all significantly challenging the performance of visual collision detection systems. There have been some modelling studies showing the LGMD's efficacy of collision detection in vehicle driving scenarios. For example, a seminal work by Keil et al. demonstrated the effectiveness of ON and OFF mechanisms in an LGMD model for proximity detection in real-world scenes [START_REF] Keil | A neural model of the locust visual system for detection of object approaches with real-world scenes[END_REF]. Yue et al. introduced a genetic algorithm with an LGMD visual neural network to reduce false collision alerts rate using video sequences from a camera mounted inside a car [START_REF] Yue | A bio-inspired visual collision detection mechanism for cars: Optimisation of a model of a locust neuron to a novel environment[END_REF]. Stafford et al. proposed a method to combine the elementary motion detector (EMD) from the fly visual system with the LGMD for amplifying the colliding and translating stimuli [START_REF] Stafford | A bio-inspired visual collision detection mechanism for cars: combining insect inspired neurons to create a robust system[END_REF]. More recently, a prominent work was proposed by Harbauer for constructing an LGMD-based collision detection system for vehicles [START_REF] Hartbauer | Simplified bionic solutions: a simple bio-inspired vehicle collision detection system[END_REF]. In this study, the author specified a 'danger zone' in the centre of the vehicle's view to help exclude irrelevant optical flows in the surrounding area.

In this research, on the basis of a recent biological research [START_REF] Rind | Two identified looming detectors in the locust: ubiquitous lateral connections among their inputs contribute to selective responses to looming objects[END_REF], we looked into the signal processing within the pre-synaptic area of the LGMD giant neuron. Compared to aforementioned works and our previous model [START_REF] Fu | Shaping the collision selectivity in a looming sensitive neuron model with parallel on and off pathways and spike frequency adaptation[END_REF], we propose an adaptive inhibition mechanism, which enables the LGMD to perceive looming cues against complex dynamic background like gratings. Importantly, such a mechanism leads to further sharpen up the LGMD's selectivity by rigorously suppressing background shifting and other irrelevant motion including objects receding and translating. To verify the effectiveness of this new inhibition mechanism, we investigated its collision detection performance in complex dynamic scenes. The rest of this paper is organised as follows: Section 2 introduces the proposed method; Section 3 presents our experiments and results, with further discussion; this research is summarised in Section 4.

Model Description

In this section we will present the proposed LGMD visual neural network. The neural computation flowchart is illustrated in Fig. 2.

Spatiotemporal Neural Computation in the Pre-synaptic Area

The first layer of photoreceptors arranged in a 2-D matrix calculates the luminance change between every two successive frames at every local pixel:

P (x, y, t) = L(x, y, t) -L(x, y, t -1) + np i a i • P (x, y, t -i).
(

) 1
The The persistence of luminance change could last for a short while of several (n p ) frames. The decay coefficient a i is calculated by a i = (1 + e u•i ) -1 , and u = 1.

In addition, from the photoreceptor layer, we also compute the object size dependent feed-forward inhibition (FFI). Differently from former LGMD neural networks, e.g. [START_REF] Fu | Shaping the collision selectivity in a looming sensitive neuron model with parallel on and off pathways and spike frequency adaptation[END_REF][START_REF] Hu | Bio-inspired embedded vision system for autonomous micro-robots: The LGMD case[END_REF][START_REF] Rind | Two identified looming detectors in the locust: ubiquitous lateral connections among their inputs contribute to selective responses to looming objects[END_REF][START_REF] Yue | Collision detection in complex dynamic scenes using a lgmd based visual neural network with feature enhancement[END_REF], where the FFI can directly suppress the LGMD giant neuron if luminance changes rapidly over a large field of view, we propose a new FFI mechanism, namely the feed-forward inhibition mediation (FFI-M), which will be used to tune the local inhibitions and excitations in the ON and OFF pathways. We define the output delayed signal as F , which is computed as:

F (t) = R x C y |P (x, y, t)| • n -1 cell , d F (t) dt = 1 τ 1 (F (t) -F (t)), (2)
where C and R denote the columns and rows of the photoreceptor layer, and n cell stands for the total number of units, i.e. n cell = C × R. The output is delayed by a first-order low-pass filtering with a time constant (τ 1 = 10ms).

After that, the photoreceptors pass motion information through parallel ON and OFF pathways, depending on luminance increments (ON) or decrements (OFF):

P on (x, y, t) = [P (x, y, t)] + , P of f (x, y, t) = -[P (x, y, t)] -, (3)
where [x] + and [x] -denote max(0, x) and min(x, 0), respectively. Subsequently, the polarity signals flow into parallel pathways, each possessing multiple layers including excitation (E), inhibition (I) and summation (S).

Firstly, the ON cell leads the excitation to pass directly to the ON-E layer; meanwhile, it is fed into a low-pass filtering, which gives feedback on delayed information:

E on (x, y, t) = P on (x, y, t), d Êon (x, y, t) dt = 1 τ 2 (E on (x, y, t) -Êon (x, y, t)), ( 4
)
where Êon is the output and τ 2 is the latency varying between 60 and 180ms. The local inhibition in the ON-I layer is then convolved by delayed ON-excitations:

I on (x, y, t) = 1 i=-1 1 j=-1 Êon (x + i, y + j, t) • W (i + 1, j + 1), (5)
where W denotes a convolution kernel and fits the following matrix:

W =    1/8 1/4 1/8 1/4 1 1/4 1/8 1/4 1/8    . (6)
It is important to note that the local excitation performs also self-inhibition.

Next, in the ON-S layer, there is a competition between the local excitation and inhibition, which represents a linear calculation:

S on (x, y, t) = E on (x, y, t) -w • I on (x, y, t), (7)
where w is a local bias to the inhibition. Note that only the non-negative values can reach the forthcoming computation. In this LGMD neural network, the ON and OFF pathways share the same spatiotemporal neural computation. Therefore, for simplicity, we illustrate the processing in the ON pathway only. After the generation of local ON and OFF excitations in the ON-S and -OFF-S layers, there are interactions between both polarity channels, which represent supralinear computations. That is,

S(x, y, t) = θ 1 • S on (x, y, t) + θ 2 • S of f (x, y, t) + θ 3 • S on (x, y, t) • S of f (x, y, t), (8)
where {θ 1 , θ 2 , θ 3 } denote the combination of term coefficients for balancing the contributions of both ON and OFF pathways, all set to 1 in this model. The proposed visual neural network features also a grouping (G) layer for the purpose of reducing noise and for clustering local excitations by expanding edges. The G layer processing adopts the methods used in a former LGMD neural network [START_REF] Yue | Collision detection in complex dynamic scenes using a lgmd based visual neural network with feature enhancement[END_REF], which is omitted here. Moreover, as shown in Fig. 2, the grouped local excitation has a temporal latency before reaching the LGMD cell. The computational role is consistent with a first-order low-pass filtering, with a dynamic time τ g , updated at every frame (initially set to 10ms).

Adaptive Inhibition Mechanism

In the proposed LGMD visual neural network, the FFI-M pathway is crucial to adjust the local biases to both the ON-inhibition and the OFF-inhibition in the PNN, at every frame. That is,

w = max σ 1 , F (t) T f , τg = τ g • max σ 2 , 1 - F (t) T f , (9)
where T f = 20 is a threshold and σ 1 = 0.5. This mechanism works effectively to make the neural network adapt to different levels of background complexity.

More precisely, the model is inhibited by dramatic changes of background clutter, like background shifting and grating movements. Most importantly, the model can still detect looming objects within dynamic background clutter. In addition, the FFI-M pathway tunes the temporal latency of grouped local excitations, where the time delay at every frame is updated by a coefficient compared to a very small real number σ 2 = 0.01. This dynamic temporal tuning indicates that the time latency of local excitations reaching the LGMD will become shorter as the objects growing on the field of view, i.e., the looming case.

The LGMD Cell

Following the pre-synaptic visual processing, the LGMD giant neuron pools all the local excitations forming the neural potential, which is then exponentially transformed to the sigmoid membrane potential as follows:

k(t) = R x C y G(x, y, t), K(t) = 1 + e -k(t)•(n cell •σ3) -1 -1 , (10)
where the coefficient σ 3 = 1 shapes the neural potential within [0.5, 1). G is the grouped local excitation in the G layer (Fig. 2). After that, we apply a spike frequency adaptation mechanism, which contributes to further sharpen up the looming selectivity via weakening the LGMD's responses to translating and receding stimuli. The corresponding computation can be found in [START_REF] Fu | Shaping the collision selectivity in a looming sensitive neuron model with parallel on and off pathways and spike frequency adaptation[END_REF].

Output Spike Frequency

The membrane potential is then exponentially mapped to spikes by an integervalued function:

S spike (t) = e (σ4•(K(t)-Tsp)) , (11)
where T sp = 0.7 indicates the spiking threshold and σ 4 = 10 is a scale parameter affecting the firing rate at every frame. After that, we compute the spike frequency (spikes per second) within a range of discrete frames as the model output to indicate the recognition of collisions or not:

Coll(t) =        True, if t i=t-Nts S spike (i) • 1000/(N ts • τ i ) ≥ T sf False, otherwise , (12)
where N ts = 6 denotes the amount of frames constituting a short time window and T sf = 15 ∼ 30 spikes/s is the spike frequency threshold. τ i stands for the discrete time interval in milliseconds between successive frames.

Experimental Evaluation

All the experiments can be categorised into two types of tests: we firstly tested the proposed LGMD visual neural network against synthetic movements in dynamic background clutter, with various spatiotemporal sinusoidal grating movements and shifting of a panoramic natural scene; we then investigated its performance in very complex vehicle driving scenarios consisting of many crash and near-miss scenes.

The simulations input to the proposed model were greys-scale videos with 30Hz sampling frequency. The vehicle video sequences were adapted from dashboard camera recordings at 18Hz [START_REF]Best dash cam accidents[END_REF]. The resolutions are 380 × 334, 540 × 270, 352 × 288 for grating, panoramic and vehicle videos, respectively.

Synthetic Visual Stimuli Testing

In the first type of experiments, we aim to demonstrate that the proposed L-GMD model can detect objects approaching in complex dynamic backgrounds. Firstly, as basic trials for biologically visual systems, we tested it against grating movements with a wide range of spatial and temporal frequencies. As shown in Fig. 3, challenged by grating movements alone, the proposed LGMD is completely inhibited during each course. The results verify that the proposed adaptive inhibition mechanism works effectively to deal with grating movements, which well reconciles with the responses of a biological LGMD.

Subsequently, we simulated dark and light objects looming against grating movements in the background. In this case, the object approaches or moves away from the field of view at a constant linear speed of 10.8 cm/s; the spatial frequency varies at 40, 60, 80 units/m. The results in Fig. 4 demonstrate that the proposed LGMD can robustly recognise either dark or light object approaching within grating stimuli regardless of the background grating frequencies. The model represents dramatically increasing spike frequencies near the end of approaching when the object gets close to the retina. Notably, the light looming object generates a much higher spiking rate, which indicates the LGMD's sensitivity to the contrast between looming objects and background, i.e, the LGMD responds more strongly to looming objects with larger contrast. In the last part of the synthetic stimuli experiments, we simulated movements including looming, recession and translation embedded in the shifting of a panoramic natural scene. As illustrated in Fig. 5, the proposed LGMD responds most strongly to dark and light looming stimuli compared to receding and translating ones. The spike frequency gradually increases when looming objects grow on the field of view, and then exceeds the collision alert level remaining high for a long period. On the other hand, the LGMD shows lower spike frequency at the start of the object recession only, and little or no response to translations. The results verify the effectiveness of the proposed LGMD model for extracting only looming cues from a dynamic cluttered background.

Real-world Driving Scenes Testing

In the second type of experiments, the proposed neural network was tested in much more complex vehicle driving scenes, which comprise mixed movements to interfere the detection of real 'dangers'. Our goal was to investigate its performance for fast collision perception on vehicles. We categorised the vehicle driving video sequences adapted from [START_REF]Best dash cam accidents[END_REF] into two types of scenarios, i.e., the fatal crashes and the near-miss scenes.

In the first case, as depicted in Fig. 6, the proposed neural system works effectively in perceiving imminent collisions very quickly, either in daylight or night driving scenarios. More specifically, the spike frequency increases very quickly and goes beyond the alert threshold before the highlighted real 'crash moments'. Otherwise, the LGMD maintains a low-rate rate below the threshold. Perceiving dangers before crashes is of great importance for vehicles with or without drivers. The proposed LGMD is highly activated before crashes, so it can be a reliable assistant alert system to improve collision avoidance during navigation.

For comparison, we also challenged the proposed neural system with some near-miss situations where the vehicle avoided the collision, or faced nearby-lane approaching and translating vehicles. As illustrated in Fig. 7, compared to the crash cases, the proposed model generates few sparse spikes or remains quiet. The results demonstrate the proposed LGMD visual neural network can well discriminate urgent collisions by rapid approaching objects from other irrelevant end less dangerous situations.

Discussion

Building upon our previous work, we have shown the efficacy of an adaptive inhibition mechanism in the LGMD visual neural network dealing with looming perception in complex dynamic scenes. Notably, we showed its robust collision perception ability in vehicle driving scenarios including fatal crashes. However, there are still challenges to be solved by the proposed method. Indeed, although it can perceive impending crashes from the frontal view, we found that nearby surpassing or approaching objects may affect the collision detection performance by causing false alerts. In this case, a single neuron computation is insufficient, whereas the integration of multiple neural systems could be more effective. In addition, the alert firing threshold, which is now manually defined, should be also adaptive to varying complexity of environmental dynamics.

Concluding Remarks

This paper has introduced a bio-plausible visual neural network inspired by the locust looming sensitive giant neuron -the LGMD -for fast collision perception in complex dynamic scenes, including vehicle driving scenarios. Compared to previous studies, we focused on an adaptive inhibition mechanism capable of dealing with different levels of background complexity. The experimental results verified the feasibility and robustness of the proposed method in potential real-world applications. To improve road safety, the proposed model could be a good collision detection system embodied in miniaturised sensors for future autonomous vehicles and robots.

Fig. 1 .

 1 Fig. 1. Schematic illustration of the LGMD neuromorphology: the red 'dendrites tree' area indicates pre-synaptic visual processing; the green 'dendrite trees' field denotes a separate feed-forward inhibitory (FFI) pathway; DCMD (descending contra-lateral motion detector) is a one-to-one post-synaptic target neuron conveying the LGMD's spikes to further motion control neural systems.

Fig. 2 .

 2 Fig. 2. Schematic of the LGMD visual neural network. The input is grey-scale imagery. Pixel-wise luminance (L) is captured by n number of photoreceptors (P), each relayed into ON and OFF pathways with multi-layers inside partial neural networks (PNN). ON and OFF cells implement the functions of half-wave rectifiers. E, I, S and G stand for excitation, inhibition, summation and grouping cells; t indicates temporal latency. There are spatiotemporal convolutions in the ON/OFF channels. The output is spike frequency. A separate FFI-M pathway from the photoreceptor layer adjusts the local inhibition strength and tunes the local excitation latency at every frame.

Fig. 3 .

 3 Fig.3. Membrane potentials of the proposed LGMD model by sinusoidal gratings with a wide range of spatial (SF) and temporal (TF) frequencies: the firing threshold is set at 0.7; The potentials at 0.5 denote non-response of the LGMD model.

Fig. 4 .Fig. 5 .

 45 Fig. 4. Spike frequency of the proposed LGMD by dark and light objects looming in grating stimuli with three different SF. The alert level is set at 30 spikes/s.

Fig. 6 .

 6 Fig. 6. Spike frequency of the proposed LGMD by vehicle crash scenarios. The spikes and the frequencies are both depicted. The vertical dashed lines indicate the real 'crash moments'. A red circle marks the potential colliding object in the snapshots.

Fig. 7 .

 7 Fig. 7. Spike frequency of the proposed LGMD by vehicle near-miss scenes.

Supported by the EU Horizon 2020 projects STEP2DYNA(691154) and ULTRA-CEPT(778062).