Keywords: planning approaches, Evolutionary algorithms SIAs Swarm intelligence algorithms ABC Artificial bee colony DE Differential evolution

twelve robots . . . . . . . . . .

This research work has been realized in Ecole Centrale de Lille, in laboratory "Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL)", with the research team "System of Systems Engineering (SoftE)".

First and foremost, I would like to express my deepest gratitude to my supervisor Prof. Ahmed RAHMANI. Over the past three years, his patient guidance, understanding, and constant support have enabled me to gain a lot, including academic research experience as well as life insights. These lessons learned from him will benefit me significantly in both my professional and personal lives. Moreover, he offered his continuous advice and insight while this thesis has been written.

Words cannot express my appreciation to my thesis reviewers, Prof. Yangquan CHEN and Prof. Andreas RAUH, for their invaluable patience and feedback.

My sincere thanks also go to all the jury members for their kind acceptance to participate in my defense. Their insightful comments and expertise shone a light on my drawbacks and the research road in the future. I would also like to thank Mr. Rochdi MERZOUKI, the leader of the group SoftE in laboratory CRIStAL, for providing us great opportunities to exchange with other scholars. In addition, many thanks to Ms. Sara IFQIR, a very kind teacher in our group. Her patience and knowledge helped me a lot and gave me much encouragement.

At the same time, I would like to extend my sincere thanks to Prof. Yongguang YU for his guidance and precious suggestions. I am also very grateful to Dr. Wei HU for the great help he has given me. He has continued to assist me in many aspects throughout my Ph.D., while his research work has inspired me on the path of research.

During the past three years, I had the pleasure of working with my colleagues Yunlong ZHANG, Xinyong WANG, Yiwen CHEN, and Xiaoqin ZHAI, to name i

Contents

Acknowledgements i

Table of Contents iii

List of Figures vii

List of Tables x Abbreviations and notations xiii

Chapter 1

Background and motivation

With the accelerated development of technology, optimization problems can be found in a wide range of fields, including engineering, finance, biology, medicine, transportation, robotics, and artificial intelligence, to mention a few. At the same time, people often strive to optimize their profit while minimizing various costs in reality. For instance, the path planning in robotics or intelligent transportation

INTRODUCTION

requires creating an ideal path while taking into account various environmental conditions in order to decrease travel consumption and boost transportation efficiency; the feature selection in image analysis demands for extracting the most relevant features from a large amount of data to reduce the subsequent computational dimensions. In this context, effectively addressing these problems has significant implications for raising productivity and lowering expenses in the related tasks. Nevertheless, resources like time and money are constantly limited in practical applications. Therefore, it is crucial to optimize the utilization of these resources while complying with certain constraints.

Hence, constructing mathematical models becomes a crucial approach for handling various optimization problems effectively. At the same time, numerous optimization methods have been proposed to improve the final results. And researchers have continued to further enhance the approaches up to today due to the increasing requirements in all aspects. In addition, various simulation tools have emerged as crucial research techniques.

To resolve optimization problems, the manner of designing optimization models is also essential. Meanwhile, it is normal that a task can alternatively be accomplished through different ways of problem formulation. After properly defining the objective function, the desired solution can be found via various optimization methods. It is worth pointing out that the solution might be the exact optimal solution or a high-quality one that was produced rapidly. In this case, selecting or designing appropriate optimization algorithms for different situations is indeed a meaningful research direction.

Thus, in this context, lots of optimization methods have been recommended

which can be classified in a number of ways. According to the nature of the algorithms, they can be generally categorized into two groups, namely deterministic and stochastic algorithms [START_REF] Yang | Nature-inspired optimization algorithms[END_REF]. The former follows a rigorous searching process, which starts from an initial point, then generates the search directions and iteration steps based on specific rules. Afterwards, the algorithm iterates and updates the search coordinate until the termination condition is satisfied.

In other words, the deterministic optimization methods obtain the same result over multiple runs. This category includes many traditional approaches which have been proposed since the last century, such as Newton-like methods (e.g., Newton's and Quasi-Newton methods) [START_REF] Dennis | Quasi-Newton methods, motivation and theory[END_REF][START_REF] Fletcher | Practical methods of optimization[END_REF]; Li 2 1.1 Background and motivation et al., 2019). In fact, these methods usually reach the optimum through differentiation [START_REF] Chopra | Golden jackal optimization: A novel nature-inspired optimizer for engineering applications[END_REF]. More precisely, Hessians or gradients need to be evaluated during the searching process. In this case, the deterministic approaches may have difficulties in determining the optimal solution quickly enough when tackling non-continuous, non-convex, or large-scale optimization problems (Flor-Sánchez et al., 2022; [START_REF] Lin | A review of deterministic optimization methods in engineering and management[END_REF]. However, it is important to recognize that many real-world optimization problems are complex or have discontinuous objective functions.

The second category of approaches, in contrast, can be distinguished by the fact that randomness is a crucial component of stochastic optimization methods [START_REF] Yang | Nature-inspired optimization algorithms[END_REF]. Due to the involvement of randomness, uncertainty exists in their search process as well as the final results. Hence, the solutions obtained by this kind of method are different for each run. Heuristic and meta-heuristic are the two main types of stochastic optimization algorithms, although there is little distinction between them.

The heuristic is a way by trial and error to generate solutions, which comes from a Greek verb (ευρισκω) with the meaning "to find/discover". As mentioned above, the complexity of optimization problems keeps increasing. And solving complex problems requires extensive evaluation to determine an exact solution.

Nevertheless, the time taken to find this exact solution is often unacceptable [START_REF] Pearl | Heuristics: intelligent search strategies for computer problem solving[END_REF]. Moreover, many of the practical applications have been considered

as NP-hard problems. And their deterministic polynomial time methods hardly exist [START_REF] Li | A novel stochastic optimization algorithm[END_REF]. In this context, the heuristic techniques demonstrate their advantages for the aforementioned issues by presenting a way to minimize the number of evaluations and find a solution within a reasonable time. In fact, feasible solutions can be found to a challenging optimization problem in a reasonable time frame, but there is no guarantee that the solutions are optimal. Note that machine learning methods can also be classified into this category as they aim to improve their performance in iterative trial and error.

The so-called meta-heuristic algorithm is a further extension of the heuristics. The prefix "meta-" denotes "beyond" or "higher-level", and they usually outperform the heuristic methods [START_REF] Yang | Nature-inspired optimization algorithms[END_REF]. Meta-heuristics are not problemspecific, and they can find sufficiently good approximate solutions by exploring Based on the background above, it can be observed that meta-heuristic algorithms are very promising optimization methods with a broad variety of applications. Meanwhile, it is important to keep in mind that due to the growing need for effectiveness, every well-known meta-heuristic algorithm can be further 1.2 Overview of meta-heuristic algorithms enhanced. For instance, the convergence speed and the capability to avoid local optima are always expected to be better. And precisely handling the trade-off between exploration and exploitation for all optimization tasks is also challenging.

Therefore, this thesis focused on a class of meta-heuristic algorithms represented by the Artificial Bee Colony (ABC) algorithm and proposed a series of improved variants by analyzing the characteristics and weaknesses of the ABC algorithm.

Furthermore, problems with a higher application value are also taken into account. The proposed enhanced ABC versions have successfully solved different types of optimization problems, including a series of robot path planning problems.

Overview of meta-heuristic algorithms

The meta-heuristic algorithm has continuously attracted attention because of its outstanding performance in solving tough optimization problems. This kind of algorithm has the advantages of clear structure, high flexibility, and the ability to avoid local optima. They are usually inspired by some intelligent natural phenomena and use mathematical representations to resolve real-world problems. It is worth pointing out that problems are considered black boxes when using metaheuristic algorithms. In other words, these methods are not problem-specific, which means that they can handle a variety of problems without requiring any changes.

Such algorithms are always examined from the perspectives of exploration and exploitation (diversification and intensification). Exploration is the process of developing the entire search space by producing diverse solutions. The diversity of solutions enables the algorithms to gather more information effectively.

Actually, randomness plays a crucial role in keeping the algorithm with good exploration ability. And randomness might provide the algorithm more opportunities to escape when a local optimum is reached. On the other hand, exploitation implies concentrating on the search in promising areas by taking advantage of the current search results. The algorithms are then expected to gradually converge to the optimal solution. Nonetheless, it should be emphasized that attempting to achieve the best in both exploration and exploitation simultaneously is a big challenge. More precisely, if we concentrate on boosting the exploration ability,

INTRODUCTION

the solutions will be more dispersed. As a result, the convergence speed of the algorithm will be slowed down. And if we focus on enhancing the exploitation ability, the algorithm may converge too soon to some seemingly good solutions, which could eventually cause it to miss the actual global optimum. Therefore, for all the meta-heuristic algorithms, it is difficult but essential to find a nice balance between these two components. The trajectory-based methods reach the optimal solution with one initial solution, while the population-based algorithms have a group of randomly generated initial solutions. In general, population-based meta-heuristics are more focused on exploration, whereas trajectory-based solutions pay more attention to exploitation. Meta-heuristics can also be classified into two groups: non-nature-inspired and nature-inspired methods [START_REF] Doering | Metaheuristics for rich portfolio optimisation and risk management: Current state and future trends[END_REF]. And the distinction between these categories can be deduced from their names. The first category includes the simulated annealing (SA) algorithm, tabu search (TS), and so on. The second group can be further divided into two types: evolutionary algorithms (EAs) and swarm intelligence algorithms (SIAs). The former such as the genetic algorithm (GA) [START_REF] Holland | Adaptation in natural and artificial systems[END_REF] and the differential evolution (DE) algorithm [START_REF] Storn | Differential evolution -a simple and efficient heuristic for global optimization over continuous spaces[END_REF], obtains the best individual through a series of evolutionary operators. Meanwhile, SIAs achieve the optima by starting the search process from different positions simultaneously [START_REF] Song | A high-efficiency adaptive artificial bee colony algorithm using two strategies for continuous optimization[END_REF]. [START_REF] Karaboga | An idea based on honey bee swarm for numerical optimization[END_REF] believed that self-organization and division of labor are two necessary and sufficient conditions for swarm intelligence. For examples, particle swarm optimization (PSO) [START_REF] Kennedy | Particle swarm optimization[END_REF], artificial bee colony (ABC) [START_REF] Karaboga | An idea based on honey bee swarm for numerical optimization[END_REF], and firefly algorithm (FA) [START_REF] Yang | Firefly algorithms for multimodal optimization[END_REF]) are widespread SIAs.

Looking back at history, it is difficult to pinpoint the exact time that metaheuristics initially appeared. In the 1960s, the genetic algorithm (GA) was created by John Holland and his colleagues [START_REF] Holland | Adaptation in natural and artificial systems[END_REF][START_REF] Holland | Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence[END_REF], which was very successful and meaningful. GA was inspired by the Darwinian evolution and natural selection of biological systems. It represented the process via mathematical operators: mutation, crossover, and selection. Moreover, GA is still popular today 1.2 Overview of meta-heuristic algorithms due to its effectiveness in solving a wide range of optimization problems. Kirkpatrick et al. (1983) proposed a local search meta-heuristic based on a physical annealing process for solids, named the simulated annealing (SA) algorithm. It established a link between the thermodynamic behavior and the search for the global minimum of the discrete optimization problems [START_REF] Nikolaev | Simulated annealing[END_REF]. Then, another well-known meta-heuristic algorithm, namely ant colony optimization (ACO), was proposed by [START_REF] Dorigo | Optimization, learning and natural algorithms[END_REF]. The ACO drew inspiration from the foraging behavior of the ant colony. The ants are found to deposit pheromones in the places they pass by, which allows the ant colony to determine the quality of paths. And this mechanism was represented in ACO for solving optimization problems [START_REF] Dorigo | Ant colony optimization[END_REF]. [START_REF] Kennedy | Particle swarm optimization[END_REF] proposed the particle swarm optimization (PSO) algorithm inspired by the swarming intelligence of bird flocks and shoals. In PSO, individuals (or particles) start with some initial random positions in the search space. Hence, by communicating the current best solution and sharing the global best solution, the optimal solution can be effectively approached. It can be said that PSO is a very essential SIA, which is beneficial for many subsequent algorithms and their improved versions. Furthermore, [START_REF] Storn | Differential evolution -a simple and efficient heuristic for global optimization over continuous spaces[END_REF] proposed the differential evolution (DE) algorithm, which differs from GA in the reproduction mechanism and the solution forms. DE also contains the three major operators, namely mutation, crossover, and selection. It can effectively obtain information from the population and its parents. This algorithm has also played a crucial role in improving other algorithms. As a result, it can be observed that the two decades of the 1980s and 1990s were the most active periods for meta-heuristic algorithms.

Afterward, [START_REF] Karaboga | An idea based on honey bee swarm for numerical optimization[END_REF] proposed the ABC algorithm by simulating the foraging behavior of bee colonies. Correspondingly, three phases were constructed, including employed bees, onlooker bees, and scout bees. Later, [START_REF] Yang | Cuckoo search via Lévy flights[END_REF] proposed the cuckoo search (CS) algorithm via Lévy flights [START_REF] Lévy | Théorie de l'addition des variables aléatoires[END_REF][START_REF] Shlesinger | Levy flights: Variations on a theme[END_REF] inspired by the breeding behavior of cuckoo species. More recently, novel meta-heuristic methods have been constantly developed. For instance, stochastic fractal search (SFS) [START_REF] Salimi | Stochastic fractal search: A powerful metaheuristic algorithm[END_REF], monarch butterfly optimization (MBO) [START_REF] Wang | Monarch butterfly optimization[END_REF], slime mould algorithm (SMA) [START_REF] Li | Slime mould algorithm: A new method for stochastic optimization[END_REF], and colony predation algorithm (CPA) [START_REF] Tu | The colony predation algorithm[END_REF], etc.

INTRODUCTION

It has been demonstrated that most of these approaches are easy to acquire, simple to implement, and powerful to find out optimum as well. All of these algorithms do, however, have some flaws. Therefore, it is crucial to investigate how to improve their performances and discover more practical applications (Chen 1.3 Overview of Artificial bee colony (ABC) algorithm Among various meta-heuristic methods, the ABC algorithm [START_REF] Karaboga | An idea based on honey bee swarm for numerical optimization[END_REF] is one of the most popular algorithms. Being inspired by the intelligent foraging behaviors of bee colonies, Karaboga proposed the ABC algorithm. ABC has demonstrated its superiority like fewer parameters and excellent exploration ability compared to the other meta-heuristic algorithms. Meanwhile, its structure is clear and easy to understand. Therefore, it has been widely studied and implemented in various applications. In the followings, the search process of the basic ABC is presented in detail. Then, related literature is reviewed and summarized in subsection 1.3.2.

Remark 1.1 For simplicity of the descriptions, we assume that the concerned optimization problems can be transformed into minimization problems. Therefore, in the presentation of the algorithms, the objective function value of a candidate solution can directly reveal its quality.

The standard ABC algorithm

There are three types of bees in ABC to model the foraging behavior of honeybees: employed bees, onlooker bees, and scout bees. The object is to search for the food source with the best quality, where the food source represents the feasible solution to the optimization problem. All the candidate solutions are vectors of D dimensions for solving a D-dimensional problem.

After the initialization phase, the ABC algorithm starts to repeat the three phases until the determination condition is achieved. Firstly, the employed bees are supposed to seek nectars in the search space and share the information with 1.3 Overview of Artificial bee colony (ABC) algorithm onlooker bees by dancing. In the next step, onlooker bees are supposed to select the food sources found by employed bees according to the nectar qualities and further exploit better food sources around the areas. The scout bees are responsible for keeping the diversity of population by replacing the food sources that haven't been updated during certain cycles. The principle phases are described as follows.

Initialization phase

Firstly, SN candidate solutions are randomly generated via Eq.(1.1) in the search space. And x i = (x i,1 , x i,2 , • • • , x i,D ) denotes the i th food source (candidate solution) where D is the dimension of optimization problem.

x i,j = x l,j + rand(0, 1) × (x u,jx l,j ),

(1.1) where x i,j is the value of the j th variable of i th food source, i = 1, • • • , SN and j = 1, • • • , D. x l,j and x u,j denote the lower and upper bounds of the j th dimension, respectively. Notice that in the original ABC algorithm, the number of employed bees and onlooker bees are both SN, namely the number of food sources.

Employed bee phase

Each employed bee is associated to a candidate solution and is responsible to search for new food source positions. The search strategy is expressed as below:

v i,j = x i,j + θ i,j × (x i,j -x k,j ), (1.2)
where j is a randomly chosen variable to be updated, and k = i is randomly selected among {1, • • • , SN}. θ i,j is the scale factor which is a random real number in [-1, 1]. And v i,j is obtained considering the information of x i,j and food source

x k,j who is chosen from the swarm. Then v i is compared to the x i via the objective function f (•). The greedy selection method is shown in Eq.(1.3). If the objective function value of v i is better, v i will replace x i and the counter trial i will be reset as 0. Otherwise, x i

INTRODUCTION

remains the same and its trail i plus one.

x new i = v i if f (v i ) < f (x i ),
x i otherwise.

(1.3)

Onlooker bee phase

In this step, the onlooker bees are expected to further exploit around the promising food sources. So each onlooker bee selects one candidate solution according to the selecting probabilities via the roulette wheel selection method. The fitness values of all the food sources (f it i , i = 1, • • • , SN) are evaluated with Eq.(1.4) before calculating the selecting probabilities.

f it i =    1 1 + f i if f i 0, 1 + |f i | otherwise, (1.4)
where f i is the objective function value associated with x i .

Then the corresponding probabilities of all the candidate solutions can be calculated with Eq.(1.5) in terms of their fitness values.

p i = f it i SN m=1
f it m .

(1.5)

It can be found that, when the fitness value of a food source is large, it is considered more qualified. Hence, it is more likely to be selected and further exploited by the onlookers. Then the chosen food sources are updated via Eq.(1.2), and the greedy selection (i.e., Eq.(1.3)) is conducted. If v i wins the previous food source successfully, then the trial i will be reset to 0. Otherwise, the trail i will be added by one.

Scout bee phase

In this phase, the counter trail i , (i = 1, • • • , SN) associated with each candidate solution is compared with the limit. If a food source hasn't been improved within predetermined period then it will be abandoned by its employed bee. And a novel food source will be generated with Eq.(1.1). The corresponding counters are reset to zero at the same time. for i = 1 → SN do 5:

Generate v i with Eq.(1.2) and evaluate v i 6:

if f (v i ) < f (x i ) then 7:

Replace x i with v i 8:

trial i = 0 9: else 10:

trial i = trial i + 1 11: end if 12:
end for 13:

Calculate the fitness values of all the food sources by Eq.(1.4)

14:

Calculate the probability p i all the food sources by Eq.(1.5) % The onlooker bee phase % 15:

for t = 1 → SN do 16:

Select a food source x i based on the probability values 17:

Generate v i with Eq.(1.2) and evaluate v i 18:

if f (v i ) < f (x i ) then end for % The scout bee phase % 25:

for i = 1 → SN do 26:

if trial i > limit then 27:

Generate a new x i with Eq.(1.1) and evaluate x i 28:

trial i = 0 29: end if 30:
end for

31:

Store the best solution so far 32: until the termination condition is reached.

Related work on ABC algorithm improvement

It is widely recognized that ABC does well in diversification but is relatively poor in intensification. In this case, numerous studies have been done attempting to further enhance its performance. The existing improvement strategies can be summarized into three collections: modifying solution search equations, improving the selection mechanism, and hybridizing with other effective algorithms or auxiliary techniques. Of course, there are many improved ABC versions that used more than one category of strategies at the same time.

Modification of solution search equations

For meta-heuristic methods, their performance is highly dependent on the way of producing new feasible solutions. And the solution search equation is one of the most essential components of ABC that can considerably affect the final results. Nonetheless, the one-dimensional search equation of ABC was deemed not efficient enough. In this case, basically all the improved ABC versions modified the solution search equation. It has been observed that the search strategies based on PSO and DE algorithms are very popular, which have been incorporated into ABC in many ways.

Being inspired by the PSO algorithm, Zhu & Kwong (2010) proposed gbestguided ABC (GABC) algorithm that has been widely studied and compared until now. In GABC, information of the global best solution was added to the search equation for enhancing the search efficiency. Actually, this kind of strategy has been used repeatedly in many other related works as the global best solution can provide a useful search direction. [START_REF] Banharnsakun | The bestso-far selection in artificial bee colony algorithm[END_REF] proposed a best-sofar ABC with an enhanced solution update method. More precisely, the information of the current best solution is shared in the onlooker bee phase. Moreover, the areas of searching for new candidates were adjusted during the process. Gao et al. (2012) employed the improved search equations which adopted information of the best individual. Meanwhile, being inspired by the mutation operator of DE, the proposed search strategies (i.e., "ABC/best/1" and "ABC/best/2") considered more information about the neighbors. The chaotic maps and the opposition-based learning (OBL) method were also incorporated into the proposed algorithm to help with the convergence rate. [START_REF] Akay | A modified artificial bee colony algorithm for real-parameter optimization[END_REF] introduced a modification rate (MR) into ABC in order to control the number

INTRODUCTION

of variables that can be inherited from the previous solution, which has a similar role as the crossover rate (CR) in DE. Later Xiang & An (2013) used the bestso-far solution search equation along with another modified equation to update the solutions, and the better one was chosen greedily. [START_REF] Gao | Enhancing artificial bee colony algorithm using more information-based search equations[END_REF] proposed an enhanced ABC with two new search equations for the employed bees and onlookers, respectively. The best solution was adopted differently in these two equations. Meanwhile, a new way was utilized to determine and compare the candidate solutions with more robustness. Besides, [START_REF] Imanian | Velocity based artificial bee colony algorithm for high dimensional continuous optimization problems[END_REF] introduced more concepts from the PSO algorithm to improve ABC. In other words, the velocities of candidate solutions were updated by using the global best solution and local best position. Gao et al. (2015a) proposed a Gaussian-based search equation to produce new candidate solutions in the onlooker bee phase. As a result, the valuable information hidden in the best individual can be exploited.

In addition, the introduction of the concept of the elite is another effective improvement method that aims to better guide the swarm. [START_REF] Xiang | A particle swarm inspired multi-elitist artificial bee colony algorithm for realparameter optimization[END_REF] proposed a particle swarm inspired multi-elitist ABC algorithm. The food sources were modified using the global best solution and an elitist randomly chosen from an elitist archive. [START_REF] Xiang | An elitism based multi-objective artificial bee colony algorithm[END_REF] utilized a multi-objective ABC with an elitism strategy and fixed-size archive. An enhanced search equation was used in employed and onlooker bee phases based on the elites selected from the archive. [START_REF] Cui | A novel artificial bee colony algorithm with depth-first search framework and elite-guided search equation[END_REF] designed a depth-first search framework. The information of the current best solution and elite solutions were involved in two novel search equations. A novel ABC algorithm was devised by [START_REF] Kong | An improved artificial bee colony algorithm based on elite group guidance and combined breadth-depth search strategy[END_REF] after being impressed by the natural phenomena of following an elite group. Besides, a breadth-first search strategy was adopted in employed bee phase while a stochastic depth-first search strategy was used in onlooker bee phase. A high-efficient ABC variant was proposed [START_REF] Song | A high-efficiency adaptive artificial bee colony algorithm using two strategies for continuous optimization[END_REF]. Two novel search strategies considering the best individual were utilized and an elite group was built to generate new solutions via an adaptive selection mechanism. [START_REF] Babaoglu | Artificial bee colony algorithm with distribution-based update rule[END_REF] introduced a distribution-based solution update rule where the mean and standard deviation of two selected candidate solutions were calculated to generate a new solution. [START_REF] Kıran | A directed artificial bee colony algorithm[END_REF] regarded the single parameter as the reason for the slow convergence rate. To overcome this weakness, authors added the directional information to the algorithm. Zhang et al.

Overview of Artificial bee colony (ABC) algorithm

(2018) built a cellular structure and adopted a Gaussian-based search equation.

The new search equation was combined with a local attractor together to allow the algorithm to converge to the optimum. And a cellular automata model was introduced because it could maintain the population diversity and interaction inside neighborhoods at the same time. Moreover, in [START_REF] Wang | Improving artificial bee colony algorithm using a new neighborhood selection mechanism[END_REF], the best solution within a predefined neighborhood was selected for generating new solutions. [START_REF] Xiao | Artificial bee colony algorithm based on adaptive neighborhood search and Gaussian perturbation[END_REF] proposed an ABC with adaptive neighborhood size.

In the improved ABC algorithm, a global-best-based search equation and a new Gaussian perturbation were adopted. [START_REF] Xiang | Artificial bee colony algorithm with a pure crossover operation for binary optimization[END_REF] utilized a pure crossover operation in order to benefit information sharing. And a novel frequency of perturbation was proposed to enlarge the number of dimensions to be updated each time.

It is worth mentioning that, in many improved ABC algorithms, the search behaviors of employed bees and onlookers are designed differently, such as (Gao the random forest method with ABC so that crucial hyper-parameters could be optimized. As a result, the computational efficiency was augmented.

Application prospects

Like the other meta-heuristics, ABC and its improved versions can be used to solve almost any numerical global optimization problem. In this context, solving a group of benchmark functions using the proposed algorithm and comparing the statistical results with other well-known algorithms is a common way of validating the performance of proposed algorithm. For solving the constrained optimization problems, certain additional constraint-handling techniques are required.

INTRODUCTION

Moreover, numerous well-known optimization problems can be resolved effectively by the improved ABC algorithms, such as shop scheduling problem [START_REF] Li | Solving the large-scale hybrid flow shop scheduling problem with limited buffers by a hybrid artificial bee colony algorithm[END_REF], traveling salesman problem [START_REF] Rekaby | Introducing adaptive artificial bee colony algorithm and using it in solving traveling salesman problem[END_REF], hybrid flowshop problem [START_REF] Tao | An efficient self-adaptive artificial bee colony algorithm for the distributed resource-constrained hybrid flowshop problem[END_REF], and so on. [START_REF] Zabihi | A novel history-driven artificial bee colony algorithm for data clustering[END_REF], etc. This indicates that there will certainly be increasing practical problems that these effective ABC algorithms are able to handle in the future. Nevertheless, in order to possess high intelligence in robotic systems, a number of research issues need to be resolved. One of the major challenges for robotic systems is navigation. It could be stated that navigation and guidance are required for applying robots in any situation. Therefore, it is crucial to ensure efficient and successful navigation, especially considering the fact that robots are facing increasingly complex scenarios now.

In this context, the robots need to be aware of their positions in relation to their goals with the purpose of completing the navigation task. Moreover, in order to increase the chances of success, robots must also consider the dangers in 1.4 Overview of robot path planning (RPP) problem their surroundings and adjust the following actions [START_REF] Koubâa | Robot path planning and cooperation: foundations[END_REF]. With the purpose of achieving navigation and guidance, the process can be divided into three parts: localization, path or motion planning, and mapping. Firstly, localization indicates that a robot should continuously determine its positions.

Many types of tools like sensors and cameras have been adopted to help achieve this task. Hence, the robot should find a path according to its current location, target point, and its view of surrounding environment. In other words, path planning refers to the process of determining the pathway across the environment that allows the robot to reach its desired destination without collisions.

Subsequently, the robot is expected to act along the path planned. Under this purpose, control theory and automation techniques are necessary. Last but not the least, the robot needs a map of the environment. This map is crucial for helping the robot to realize its current location and direction. It is worth mentioning that the map can be initially stored or be constructed progressively as the robot explored the workspace. Correspondingly, the path planning can be divided into global path planning and local path planning depending on whether the environment is known or not. Figure 1.3 below summarizes the differences between them [START_REF] Koubâa | Robot path planning and cooperation: foundations[END_REF].

The objective of RPP problem is to find out an optimal collision-free path from start point to target point in an environment with obstacles [START_REF] Nazarahari | Multi-objective multi-robot path planning in continuous environment using an enhanced genetic algorithm[END_REF]. Moreover, path planning for multiple robots has gained increasing attention since these robots can work together to do certain tasks that can not be completed by an individual robot. The multi-robot path planning (MRPP) aims at searching for obstacle-free paths for a group of robots in the same environment.

It is also necessary to make sure that there is no collision between any two robots.

With the purpose of accomplishing RPP, a series of optimization problems can be formulated by considering different goals and constraints. There are three main concerns that should be taken into account: efficiency, accuracy, and safety [START_REF] Châari | smartPATH: A hybrid ACO-GA algorithm for robot path planning[END_REF]. In other words, it is essential to determine a feasible path with the least amount of energy and in the shortest amount of time. At the same time, the robot should also be able to safely avoid obstacles nearby.

Path planning approaches can be generally classified into three categories: classical, graph-based, and (meta-)heuristic approaches. Note that because the difference between the heuristics and meta-heuristics is indeed small, there is no The graph-based search methods, such as the Dijkstra [START_REF] Dijkstra | A note on two problems in connexion with graphs[END_REF] and A* [START_REF] Hart | A formal basis for the heuristic determination of minimum cost paths[END_REF][START_REF] Hawa | Light-assisted A * path planning[END_REF] algorithms, are also well-known. The feasible paths can be computed in a graph-based environment, such as a grid map.

However, the computational complexity may increase greatly in challenging environments. As a result, these traditional methods cannot remain effective when dealing with complex or dynamic environments. Hence, the demand for intelli-

Overview of robot path planning (RPP) problem

gence has increased when it comes to solving diverse path planning challenges.

In this context, heuristics and meta-heuristics have attracted increasing attention for overcoming the aforementioned limitations. Among numerous heuristic methods, meta-heuristic algorithms are outstanding in solving various optimization problems and are not problem-specific. Hence, the RPP problems can be resolved by transforming them into functional optimization problems. Then, an optimal solution to the problem can be found by adopting meta-heuristic algorithms that are effective and powerful even in complex environments. As mentioned previously, a multi-robot system possesses advantages because of the cooperation and interaction inside the team. And multi-robot collaboration has a stronger ability to resolve complex problems and has higher robustness and reliability [START_REF] Das | Multi-robot path planning using improved particle swarm optimization algorithm through novel evolutionary operators[END_REF]. In this context, multi-robot systems exist in various fields, for instance, intelligent warehouse [START_REF] Han | Effective heuristics for multi-robot path planning in warehouse environments[END_REF]

Preliminaries

From the literature above, it can be observed that some famous meta-heuristic algorithms have effective search strategies that can help to further improve other algorithms. So in the following parts, several other well-known meta-heuristics are introduced.

Differential evolution (DE) algorithm

The DE algorithm [START_REF] Storn | Differential evolution -a simple and efficient heuristic for global optimization over continuous spaces[END_REF]) is a simple yet effective EAs, which attempts to evolve a population of NP individuals to the global optimum. For 1.5 Preliminaries solving an optimization of D dimensions, the candidate solutions at g th generation can be represented as

x g i = (x g i,1 , x g i,2 , • • • , x g i,D
), where i = 1, • • • , NP . The initial population is supposed to better cover the whole search space as much as possible via uniform randomization [START_REF] Qin | Differential evolution algorithm with strategy adaptation for global numerical optimization[END_REF]. The same equation (i.e., Eq.(1.1)) is used for generating the initial solutions. Then, DE enters a loop containing three major steps until the termination condition is reached.

Mutation operation

In each generation, the individuals first generate their corresponding mutant vectors v g i via a mutation operator. Many mutation strategies have been proposed for the DE algorithm. And the most frequently used ones are presented as follows.

1) "DE/rand/1"

v g i = x g r 1 + F × (x g r 2 -x g r 3 ). (1.6)
2) "DE/best/1"

v g i = x g best + F × (x g r 1 -x g r 2 )
.

(1.7)

3) "DE/rand/2"

v g i = x g r 1 + F × (x g r 2 -x g r 3 ) + F × (x g r 4 -x g r 5 )
.

(1.8) 4) "DE/rand-to-best/1"

v g i = x g i + F × (x g best -x g i ) + F × (x g r 1 -x g r 2 )
.

(1.9)

In the above mutation operators, the neighbors

r 1 = r 2 = r 3 = r 4 = r 5 = i are randomly selected among {1, • • • , NP } while x best is the best individual. F
is the scale factor which is defined by the users.

Crossover operation

Each individual has a pair of vectors, namely the target vector x i and its mutant vector v i . Hence, for producing the next generation, individuals need to decide how much information to inherit from their parents. For each dimension, the crossover operation is used to define its value. As a result, a trial vector u g i is generated.

INTRODUCTION

u g i,j = v g i,j , if rand CR or j = j rand , x g i,j , otherwise, (1.10)
where j rand is randomly selected from {1, • • • , D} while CR is the crossover rate.

For each dimension, if the condition of (rand CR or j = j rand ) is satisfied, the new solution will get information from the mutating vector. Otherwise, it will keep the same as the original x i .

Selection operation

In the next step, the objective function value of the newly generated trial vector u g i is compared to that of x g i . And the better one will become the offspring. Then the selection operation is performed.

x g+1 i = u g i if f (u g i ) f (x g i ), x g i otherwise.
(1.11)

In addition, the pseudo-code of basic DE algorithm can be found in Algorithm for i = 1 → NP do % The mutation operation %

4:

Generate mutant vector v g i with one mutation strategy % The crossover operation % 5:

Randomly select j rand ∈ {1, • • • , D} 6: for j = 1 → D do 7:
Define the value of u g i,j with Eq.(1.10) Update the generation number g = g + 1 12: until the termination condition is reached.

Particle swarm optimization (PSO) algorithm

Preliminaries

Kennedy & Eberhart (1995) proposed the PSO algorithm by deriving inspiration from the social behaviors of bird flocks and insect swarms. Each bird in a flock is abstracted as a particle in the algorithm. The flock's flight area corresponds to the search space of concerned optimization problem. And the candidate solutions stand for the food source positions. Similar to the other meta-heuristic algorithms, a parameter of the problem that needs to be optimized is represented by each dimension of this space [START_REF] Genovesi | Particle swarm optimization for the design of frequency selective surfaces[END_REF]. The PSO algorithm is able to approach the global optimum by continuously updating the velocity and position information of the swarms.

Each particle flies in the D-dimensional search space, and the position of i th particle can be identified by a

x i = (x i,1 , x i,2 , • • • , x i,D ), i = 1, • • • , NP , where
NP is the number of particles. Meanwhile, its velocity is represented as

v i = (v i,1 , v i,2 , • • • , v i,D ).
At the beginning, each particle starts the process at an arbitrary location and moves with a random velocity that varies in amplitude as well as direction. In the next step, the PSO starts iterating until the termination condition is satisfied within the set bounds. The strategy for updating the velocity and position of each particle in the (t + 1) th iteration is as follows.

v t+1 i,j = w × v t i,j + c 1 × r 1 × (pbest t i,j -x t i,j ) + c 2 × r 2 × (gbest t j -x t i,j
), (1.12)

x t+1 i,j = x t i,j + v t+1 i,j , (1.13)
where v i,j is the velocity along the j th dimension of the i th particle. w is the inertia while c 1 and c 2 indicate the cognitive and the social rate, respectively. r 1 and r 2 are random numbers uniformly distributed in the range of

[0, 1]. pbest t i,j
is the value of the j th dimension of the best position that the i th particle has ever visited so far. And gbest represents the global best position discovered by the entire population. Hence, the location can be updated by the Eq.(1.13).

Obviously, if the newly produced position is better than the pbest i , then the current best associated to the i th particle will be replaced.

It can be found that the update equation of velocity includes three terms: the actual velocity, the cognitive part based on pbest i , and the social part based on

INTRODUCTION

gbest. The first term provides energy for the particles to fly in the search space.

The second term guides a particle towards its own best position, which is related to its independent memory. Moreover, the third term reflects the collaborative behavior among the swarm, so that each particle can move closer to the global historical optimal position. Besides, the pseudo-code of the PSO algorithm can be found in Algorithm 3. for i = 1 → NP do 7:

for j = 1 → D do 8:
Update the velocity v t i and position x t i with Eq.(1.12) and Eq.(1.13) Update the iteration number t = t + 1

19: until the termination condition is reached.

Cuckoo search (CS) algorithm

Later in the year 2009, being inspired by some cuckoo species' brood parasitism, the CS algorithm was proposed by [START_REF] Yang | Cuckoo search via Lévy flights[END_REF]. Additionally, the socalled Lévy fight [START_REF] Lévy | Théorie de l'addition des variables aléatoires[END_REF][START_REF] Shlesinger | Levy flights: Variations on a theme[END_REF]) was adopted instead of using the traditional randomization. It is worth mentioning that numerous investigations have revealed that many animal and insect flight patterns exhibit the typical characteristics of Lévy flights. This kind of search manner contains a series of straight flight pathways punctuated by a sudden change in direction.

Preliminaries

In nature, the cuckoos search for nests to lay and brood their eggs in a random or random-like manner. In order to clearly characterize the principle of the CS, the following three idealized rules are set (Yang, 2020):

• Each cuckoo lays one egg each time and lays its egg in a randomly chosen nest;

• The best nests with high-quality eggs will be carried over to the next generations;

• The number of available host nests is fixed, and the host bird has a chance of finding a cuckoo egg when it is placed. The probability is defined as

p a ∈ [0, 1].
In this situation, the host bird has two options, either get rid of the egg or just leave and build a new nest.

Note that the last assumption has been approximated by the fraction p a of the NP nests being replaced by new nests (i.e., new random solutions) for simplicity. Moreover, in the basic CS algorithm, each nest has only one egg. In this case, each nest corresponds to one egg which also indicates one cuckoo.

The initialization method Eq.(1.1), is also used in the CS algorithm to generate the initial positions of NP nests. And for an optimization problem of D dimensions, the positions are also in the form of

x i = (x i,1 , x i,2 , • • • , x i,D ).
In the main loop, each cuckoo performs a Lévy fight with the Eq.(1.14) for producing a new candidate solutions.

x t+1 i = x t i + α ⊕ Lévy(s), (1.14)
where α > 0 is the step size. The product ⊕ indicates the entry-wise multiplications. Note that the value of α is defined considering the problem scale. It is usually set as α = O(L/10), where L is the characteristic scale of the problem of interest.

In fact, the Lévy flight is a special case of random walk whose steps obey the Lévy distribution which can be defined as below

Lévy(λ) ∼ u = t -λ , (1.15)
where the stability index 1 < λ ≤ 3, while t is a random variable. And the above distribution has an infinite mean with infinite variance.

for i = 1 → NP do 5:
Generate new solution via Eq.(1.14)

6:

Evaluate its objective function value f (x t+1 i )

7:

Randomly select another nest x k 8:

if f (x t+1 i ) f (x t k ) then 9:
Replace the k th nest with x t+1 i 10:

end if

11:

end for

12:

Abandon a fraction p a of worst solutions

13:

Generate p a × NP new solutions 14:

Keep the best solutions 15:

Rank the solutions and update the current best solution

16:

Update the iteration number t = t + 1 17: until the termination condition is reached.

Contributions and outline of dissertation

Contributions and outline of dissertation

Briefly, meta-heuristic algorithms are a type of stochastic algorithm through a trade-off between randomization and local search. Such algorithms have been found to be effective and simple to understand. Thus, they have been utilized

to solve numerous optimization problems in a variety of fields. Nevertheless, there is still room for improvement, such as easy to be trapped in local optimums or slow convergence speed. And precisely handling the trade-off between exploration and exploitation for all optimization tasks is always challenging. In this context, this thesis focused on a class of meta-heuristic algorithms represented by the Artificial Bee Colony (ABC) algorithm and proposed a series of improved variants by analyzing the characteristics and weaknesses of the ABC algorithm.

Furthermore, problems with a higher application value are also taken into account. The proposed enhanced ABC versions have successfully solved different types of optimization problems, including robot path planning tasks for single and multiple robots in various environments. The main contributions and outline of dissertation are summarized as follows.

Chapter 2: In ABC, it can be found that the mission of exploration is mainly accomplished by employed bees whereas the onlookers are responsible for exploiting within certain regions. In addition to those widely mentioned improvement strategies, the impact of population composition is studied in this chapter. Actually, the invariable population composition of a bee colony cannot satisfy the needs of different search stages. In this context, improving the effectiveness of ABC by adjusting the population composition is developed. So, an ABC algorithm with dynamic population composition, namely ABCDC is proposed.

Therefore, the main contributions are as follows: firstly, the Symmetric Latin Hypercube Design (SLHD) is adopted in initialization to improve the population diversity. Secondly, a novel mechanism is proposed to adjust the colony population's composition according to the searching experiences. The number of employed bees decreases periodically while the size of onlooker bees increases to bring more energy for exploiting the global optimum in the mid-late stage of the whole process. In ABCDC, the division of labor between bees with different functions is clearer, so that global optimum can be obtained more efficiently under their cooperation. Moreover, ABCDC keeps a nice balance between diversification and intensification. And experimental studies on functional optimization

INTRODUCTION

problems are done to verify the performance of ABCDC. The comparisons show that ABCDC has better solution precision and a faster convergence rate.

Chapter 3: In fact, it is difficult to define control parameter values appropriately for all types of problems. Thus, these control parameters are usually held constant or updated with predetermined adaptation methods, such as that adopted in ABCDC. However, adaptation approaches still rely heavily on the experience of the designer. In this context, different from the existing literature, a new way of defining parameter values is proposed in this part. An ABC algorithm based on reinforcement learning (RL) is proposed, named ABC_RL. The RL method is used to vary the number of dimensions to be updated in the solution search equation. The reward value of RL is defined based on the update results. In this case, more information can be learned appropriately from the previous update experience.

The main contributions can be summarized as follows: firstly, RL is adopted to enlarge and adjust the frequency of perturbation of employed bee phase intelligently considering the immediate reward from solution update results. Secondly, two enhanced solution search equations are utilized in order to achieve a nice balance between exploration and exploitation. Thirdly, a type of heavy-tailed distribution, the Mittag-Leffler distribution, is used to generate the scale factors of search equations. Finally, the proposed ABC_RL is compared with other improved ABC algorithms on a group of benchmark functions.

Chapter 4: Since one of the most essential goals of improving such algorithms is to solve more practical problems, then its practicality and complexity must be considered. Although many modification strategies are effective in solving functional optimization problems, they do not always assist us in obtaining the optimal solution rapidly in practical applications. Therefore, it is also meaningful to improve the algorithm's performance without overcomplicating it. Therefore, in Chapter 4, enhancing the performance of ABC while avoiding it becoming too complex is investigated. In this context, a learning-based ABC (ABCL) algorithm is proposed. Hence, more energy and time can be saved when solving problems like local path planning.

The main contributions are as follows: firstly, the global best solution is adopted in the employed bee phase and scout bee phase to guide the swarm in a promising search direction. Secondly, learning phase of the TLBO algorithm 1.6 Contributions and outline of dissertation is embedded in the onlooker bee phase to improve the exploitation ability and simplify the computational complexity.

Chapter 5: In the proposed ABC variants in Chapters 2-4, the solution search equations are enhanced by enlarging the number of dimensions to be updated and increasing the amount of information that can be gained from the colony. However, this kind of improvement actually ignores some useful information about the individuals' previous search experience. It is worth pointing out that compared to the integer-order derivative, the fractional-order derivative contains entire memory of its previous events. As a result, different from existing results, the fractional-order calculus (FOC) is incorporated into the ABC algorithm considering the memory properties of FOC. In the proposed FOABC algorithm, each time generating a new candidate solution, the previous foraging behaviors stored in memory are considered.

The main contributions of this chapter are as following. The FOC is incorporated into the onlooker bee phase to make full use of the historical experiences. Meanwhile, a differential search strategy is utilized in the employed bee phase to reinforce the exploration ability. And the scale factors of these search equations are generated via Lévy distribution to increase the randomness. In order to validate the performance of FOABC, several groups of comparisons are carried out on a set of benchmark problems.

Chapter 6: After investigating different improvement strategies to improve the effectiveness of the ABC algorithm, we wanted to apply them to some more meaningful problems. Therefore, we attempted to apply these improved ABC algorithms to solve different types of path planning problems. The proposed algorithms are adopted to find better solutions in a limited time after transforming these practical tasks into optimization problems.

First, we used these methods to complete the global path planning for a single robot. Different environments with arbitrary obstacles are considered. Secondly, since multi-robot systems are demonstrating their advantages in more and more fields, we considered this meaningful problem of multi-robot path planning. For all the path planning challenges, the proposed ABC algorithms are compared to other well-known path planners in terms of path length and execution time.

Conclusions and perspectives:

The results and findings are summarized and several potential directions for our future research are discussed. The conclusion is given in the last section 2.4.

Proposed ABCDC algorithm

The proposed algorithm is introduced in details in this part. More precisely, an initialization method named Symmetric Latin Hypercube Design (SLHD) and two enhanced differential search equations are adopted. Moreover, a novel method for tuning the population composition is proposed to reinforce the intensification as well as diversification.

IMPROVED ABC ALGORITHM WITH DYNAMIC POPULATION COMPOSITION (ABCDC)

Improved initialization method

In most meta-heuristic algorithms, the initial candidate solutions are always generated randomly. It is obvious that, the higher the diversity of initial population is, the more efficient the algorithm is in searching the space. Therefore, the initialization is an essential task that can affect the quality of solutions and the convergence rate. In this context, different initialization methods have been proposed in order to enhance the diversity of population. Hence, instead of using the uniform random initialization, approaches like chaotic maps and oppositionbased learning (OBL) were utilized in some ABC variants (Gao et A comparison of the uniform random method and the SLHD in 2D space was also performed to demonstrate the advantage of SLHD. These two methods were used to produce sets of solutions, which are presented in Figure 2.1. The number of sample points are set as 200 and 400 in the two group of tests.

It can be observed that the points generated by the original method (i.e., the uniform random sampling) have some points clustered together while other places have relatively large gaps. Compared with the original method, the SLHD is able to generate more uniformly distributed points so that the space can be explored better. This is critical for the algorithms, particularly at the beginning stage when there is no prior experience.

Therefore, the SLHD is incorporated into the initialization phase of ABC algorithm. The pseudo-code of initialization phase via SLHD is presented in Algorithm 5.

Method of dynamic population composition

ABC algorithm manages to find the best solution thanks to the cooperation of the three groups of honey bees. The employed bees accomplish the mission of ex-

IMPROVED ABC ALGORITHM WITH DYNAMIC POPULATION COMPOSITION (ABCDC)

Algorithm 5 Initialization via SLHD

1: Initialize an array M of size SN × D 2: if SN is odd then M( (SN +1) 2 , j) = SN +1 2 for j = 1, • • • , D 3: end if 4: k = ⌈(SN -1)/2⌉ ⊲ ⌈•⌉ is the ceiling function. 5: for j = 1 → D do
Randomly select a permutation of 1, • • • , k and denote it by ϕ j 6: end for 7: for i = 1 → l do 8:

for j = 1 → D do 9:
if rand(0, 1) ≤ 0.5 then 10:

M(i, j) = ϕ j (i) 11: M(SN + 1 -i, j) = SN + 1 -ϕ j (i) 12:
else 13:

M(i, j) = SN + 1 -ϕ j (i) 14: M(SN + 1 -i, j) = ϕ j (i) 15:
end if

16:
end for 17: end for 18: π j = M(:, j) 19: Divide [lower j , upper j ] into SN equal subintervals and c (i) j denotes the midpoint of the ith subinterval of the jth dimension.

20: for i = 1 → SN do x SLHD (i) = (c (π 1 (i)) 1 , c (π 2 (i)) 2 , • • • , c (π D (i)) D
) 21: end for ploration while the onlooker bees focus on seeking in the promising regions. And scout bees work for preventing the stagnation of search procedure [START_REF] Babaoglu | Artificial bee colony algorithm with distribution-based update rule[END_REF]. It is noted that their sizes greatly influence the performance and convergence speed.

Based on the responsibilities of employed and onlooker bees, in the early stage of search process, we hope the size of employed bees is larger than before to search over the whole sample space as much as possible. Hence, when certain promising areas are circled, the principle mission can be shifted to exploitation which is shouldered by onlooker bees. In this way, both of the exploration and exploitation can be accomplished effectively if the composition of population is adjusted timely. Accordingly, a dynamic population composition mechanism is proposed. With this method, the number of employed bees and onlooker bees can be adjusted so 2.2 Proposed ABCDC algorithm that they can better perform their respective roles in different search periods.

The pseudo-code of the mechanism is given in Algorithm 6.

Algorithm 6 Method of dynamic population composition 1: Initialize ratio E = 0.9 2: if nb_f ail > T _f ail then ⊲ the situation needs to be adjusted

3:
Update the ratio E with Equation 2.1 4:

N employed = ⌈2 × SN × ratio E ⌉ 5:
N onlooker = 2SN -N employed ⊲ update the population composition 6:

Sort the candidate solutions according to their objective function values Reset the nb_f ail = 0 17:

F ES = F ES + N employed 18: end if
The parameter ratio E is introduced to define and adjust the proportion of employed bees in total population. According to definition in Equation 2.1, ratio E is initialized to 0.9 in the beginning. Then its value decreases gradually along with the augmentation of function evaluations F ES. Note that the max_F ES represents the maximal number of function evaluations. The coefficient a ∈ (0, 1) is important because it decides the change rate of the population composition. Once the composition is altered, a portion of the employed bees will become onlooker bees, resulting in the abandonment of several food source positions.

ratio E = 0.9 -a × F ES max_F ES . ( 2
In order to select the most hopeful candidate solutions, all the solutions are sorted according to their associated objective function values. The top N employed positions will be remained (lines 6-7 in Algorithm 6). Moreover, to avoid losing the population diversity too rapidly, the opposition-based learning (OBL) method is employed on all these remaining solutions. The opposite position of solution can be found by Equation 2.2.

x_oppo i,j = x l,j + x u,j -x i,j , (2.2)
where the x l,j and x u,j denote the lower bound and upper bound of the j th variable respectively.

Then, for each left employed bee, it will go to the fitter position by challenging its previous position with the opposite one (lines 10-14 in Algorithm 6). While the more qualified food sources are preserved, some employed bees become onlooker bees and help to exploit around those subsistent nectars. In contrast, if the counter of fails doesn't reach the sill, the bee colony will remain the same to continue searching. Furthermore, the impact of varying the value of parameter

T _f ail is discussed in subsection 2.3.2.
In addition, the method for dynamic population composition above is able to ensure that the food source positions with disappointing fitness values are removed. And new solutions can still be found by employing OBL. Therefore, ABCDC do not keep the scout bee phase because its work is accomplished by the proposed mechanism.

Two enhanced solution search equations

As mentioned previously, the original ABC algorithm works well in exploration but poorly in the exploitation. One major reason is that the solution search equa- Employed bee phase:

v i = x k1 + F employed_i × (x k2 -x k3 ), (2.3)
Onlooker bee phase:

v i,j =x i,j + F onlooker_i × (x best,j -x i,j ) + F onlooker_i × (x k1,j -x k2,j ), (2.4)
where k1 = k2 = k3 = i are randomly selected from {1,

(v i ) < f (x i ))
, then the production of new solution will be regarded as a successful search. And the utilized value of F will be recorded into a list S F _employed or S F _onlooker . The values in lists will be used when recalculating the parameters µ F _employed and µ F _onlooker via Equation 2.5 at the end of each iteration. Thus, those F employed_i and F onlooker_i who have helped to achieve successful searches will be used to lead the algorithm searching in a promising direction.

µ F = (1 -c) × µ F + c × mean L (S F ), (2.5)
where c ∈ (0, 1) is a constant. According the suggestions of JADE, c = 0.1 while µ F _employed and µ F _onlooker are initialized to be 0.5. mean L (•) is the Lehmer mean which is expressed as

mean L (S F ) = F ∈S F F 2 F ∈S F F . (2.6)
Remark 2.2 Although the parameters F employed_i and F onlooker_i are generated and updated in the same manner, they are independent from each other.

The framework of ABCDC algorithm

In this part, the complete proposed algorithm is presented. To explain the algorithm clearly, the pseudo-code is represented in Algorithm 7 and the corresponding flowchart is given in Figure 2.2.

Remark 2.3 Note that when a generated position is outside the space, it will be replaced by a new candidate solution which is randomly generated by Equation 1.1.

Proposed ABCDC algorithm

Algorithm 7 Pseudo-code of ABCDC algorithm 1: Initialize µ F _employed = 0.5; µ F _onlooker = 0.5; N employed = 2SN × 0.9; nb_f ail = 0 2: Create N employed initial food sources with Algorithm 5. S F _employed = ∅; S F _onlooker = ∅ % Enhanced employed bee phase % 6:

for i = 1 → N employed do 7:
F employed_i = randCauchy(µ F _employed , 0.1)

8:

Randomly select k1 = k2 = k3 = i from the colony 9:

Generate v i with Eq.(2.3)

10: if f (v i ) ≤ f (x i ) then 11: Replace x i with v i 12:
Add F employed_i into S F _employed 13: end if In order to do the comparisons fairly, for all the involved algorithms, the determination conditions are respect to the maximum number of function evaluations max_F ES = 5000 × D. Furthermore, all the experimental studies are based on statical results of 25 independent runs. The mean and standard deviation (Std) of errors f (X best )f (X * ) are calculated and presented in the comparison tables.

The f (X best ) is the best solution found by an algorithm and f (X * ) is the global optimum. 2.1. Among these optimization problems, f 1f 6 and f 8f 9 are uni-modal functions while f 7 is the discontinuous step function and f 10 is the noisy quartic function. Notice that f 11 is the Rosenbrock function which is uni-modal problem when D=2 and 3, however, it probably has multiple optima in the higher dimensional cases [START_REF] Kang | Rosenbrock artificial bee colony algorithm for accurate global optimization of numerical functions[END_REF]. As for the rest part, f 12f 22 are multi-modal functions and their local minima augments exponentially when the problem dimension increases.

Benchmark functions

Sensitive analysis of the parameters a and T _f ail

In the proposed algorithm, parameter a is the change rate of the Equation 2.1 which is able to adjust the population's composition. The ratio E is the proportion In other words, on the chosen optimization problems, tuning the population's composition more frequently and using a larger change rate can help to obtain better performance. As a result, the setting a = 0.8 and T _f ail = 5 will be used in the following studies.

IMPROVED ABC ALGORITHM WITH DYNAMIC POPULATION COMPOSITION (ABCDC)

f 1 (x) = D i=1 x 2 i [-100, 100] D f 2 (x) = D i=1 (10 6 ) i-1 D-1 x 2 i [-100, 100] D f 3 (x) = D i=1 ix 2 i [-10, 10] D f 4 (x) = D i=1 |x i | (i+1) [-1, 1] D f 5 (x) = D i=1 |x i | + D i=1 |x i | [-10, 10] D f 6 (x) = max i=1,...,n |x i | [-100, 100] D f 7 (x) = D i=1 (⌊x i + 0.5⌋) 2 [-100, 100] D f 8 (x) = exp(0.5 D i=1 x i ) [-10, 10] D f 9 (x) = D i=1 ix 4 i [-1.28, 1.28] D f 10 (x) = D i=1 ix 4 i + random[0, 1) [-1.28, 1.28] D f 11 (x) = D i=1 [100(x i+1 -x 2 i ) 2 + (x i -1) 2 ] [-5, 10] D f 12 (x) = D i=1 [x 2 i -10cos(2πx i ) + 10] [-5.12, 5.12] D f 13 (x) = D i=1 [y 2 i -10cos(2πy i ) + 10] y i = x i |x i | < 1 2 round(2xi) 2 |x i | ≥ 1 2 [-5.12, 5.12] D f 14 (x) = 1 + 1 4000 D i=1 x 2 i -D i=1 cos( xi √ i ) [-600, 600] D f 15 (x) = 418.98288727243380 × D -D i=1 x i sin( |x i |) [-500, 500] D f 16 (x) = -20 exp(-0.2 1 D D i=1 x 2 i ) -exp( 1 D D i=1 cos(2πx i )) + 20 + e [-50, 50] D f 17 (x) = π D {10 sin 2 (πy 1 ) + D-1 i=1 (y 1 -1) 2 [1 + 10 sin 2 (πy i+1 )] +(y D -1) 2 } + D i=1 u(x i , 10, 100 , 4)
y i = 1 + 1 4 (x i + 1), u xi,a,k,m =    k(x i -a) m x i > a 0 -a ≤ x i ≤ a k(-x i -a) m x i < -a    [-100, 100] D f 18 (x) = 1 10 {sin 2 (πx 1 ) + D-1 i=1 (x i -1) 2 [1 + sin 2 (3πx i+1 )]+ (x D -1) 2 [1 + sin 2 (sπx i+1 )]} + D i=1 u(xi, 5, 100, 4) [-100, 100] D f 19 (x) = i = 1 D |x i sin(x i ) + 0.1x i | [-10, 10] D f 20 (x) = D-1 i=1 (x i -1) 2 [1 + sin 2 (3πx i+1 )] + sin 2 (3πx 1 ) + |x D -1|[1 + sin 2 (3πx D )] [-10, 10] D f 21 (x) = D i=1 kmax k=0 [a k cos(2πb k (x i + 0.5))] -D kmax k=0 [a k cos(2πb k 0.5)] a = 0.5, b = 3, k max = 20 [-0.5, 0.5] D f 22 (x) = 1 D D i=1 (x 4 i -16x 2 i + 5x i ) [-5,

IMPROVED ABC ALGORITHM WITH DYNAMIC POPULATION COMPOSITION (ABCDC)

Comparison with ABC variants

In this part, the proposed algorithm is compared to five improved ABC variants and the standard ABC algorithm in order to evaluate its performance Table 2.4 -Table 2.6 present the comparison results in terms of the mean and standard deviation (Std) of the errors f (X best )f (X * ). And for each test function, the results of involved ABC algorithms are compared to that of ABCDC via the Wilcoxon rank sum test at 0.05 significant level. The symbols "+", "=", and "-" denote that ABCDC is better than, similar to, and worse than the compared algorithm, respectively. In addition, the Friedman test is also applied on the results and Figure 2.3 summarizes the average rankings of involved algorithms. Mean 0.00E+00 = 0.00E+00 = 0.00E+00 = 0.00E+00 = 0.00E+00 = 0.00E+00 Std 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 f 8

Experiments on function optimization problems

Mean 0.00E+00 - The statical results for 30-dimensional problems can be found in Table 2.4. In fact, the results of solving uni-modal functions (f 1 -f 9 ) can reveal the exploitation ability of concerned algorithms because each problem has only one global optimum. In this case, the proposed ABCDC algorithm has competitive exploitation capability considering its values of mean and standard deviation. For f 1f 3 and The comparison results for middle dimensional problems are shown in Table 2.5 and similar phenomena can be observed. For uni-modal functions f 4 and f 9 , ABCDC, ILTD_ABC and original ABC manage to find out the optimal solutions. For f 6 , ABCDC and ILTD_ABC attain the global optimum whereas other competitors are far from the optimal solution. And for f 1f 3 and f 5 , ABCDC Mean 0.00E+00 = 0.00E+00 = 0.00E+00 = 0.00E+00 = 0.00E+00 = 0.00E+00 Std 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 f 8

f 5 -f 6 ,
Mean 0.00E+00 - do not have significant differences on 10 functions. And these two methods each achieves better results on 6 functions.

In addition, Friedman tests are conducted on all the three comparisons as it is widely used to evaluate the overall performance of more than two algorithms. It can be seen that ABCDC obtains the best rankings in all the three cases. 52 And ILTD_ABC is the second-best algorithm followed by the original ABC algorithm. Therefore, the proposed algorithm outperforms the other ABC algorithms.

Experiments on function optimization problems

In addition to comparing with other ABC variants, we also observe and compare the results of ABCDC itself in different dimensions. It can be found that the results of D = 30, 50, and 100 calculated by ABCDC are similar. That is to say, the proposed algorithm is not sensitive to the increase of dimensions which means that it possesses superior robustness at least in solving the chosen benchmark functions.

Comparison with non-ABC algorithms

In this subsection, ABCDC is compared to four effective non-ABC meta-heuristic algorithms. The DE with mutation "DE/rand/1" is first concerned in the comparison. PSO and CS are also involved as they are also effective and famous.

Last, the firefly algorithm (FA) [START_REF] Yang | Firefly algorithms for multimodal optimization[END_REF] is also an effective method inspired by the behavior of the firefly.

The tests are carried out on those benchmarks in low, middle and high dimensions as well. The statical results of are presented in as good as the proposed algorithm. Thus, the proposed algorithm has excellent performance for low-dimensional problems considering the solution accuracy.

Similar conclusions can be derived from the comparison results of D = 50 and 100. Table 2.8 presents the results of 50-dimensional problems, for solving the uni-modal problems, ABCDC achieves the exact optimal solutions of f 3 , f 6 and f 9 whereas PSO and CS only manage to reach the optimum of f 7 . Moreover, the superiority of ABCDC is obvious on f 1 to f 6 . As for multi-modal functions, ABCDC manages to obtain the optima of five functions. Considering the results of Wilcoxon test, the mean values of ABCDC surpass all the compared algorithms on most problems. It can be concluded that ABCDC achieves the best results in the comparison with D = 50.

In Table 2.9, ABCDC attains the best results on all the benchmark functions whereas DE algorithm obtains similar solutions on f 8 and f 14 . Note that, the PSO, DE and CS algorithms have competitive results on uni-modal functions.

Nevertheless, their solution qualities are not comparable to that of ABCDC when solving multi-modal functions. Therefore, the proposed algorithm has excellent diversification and intensification abilities compared to non-ABC algorithms.

Furthermore, Figure 2.4 presents the results of Friedman tests. The advantages of ABCDC is evident in this figure and it is followed by the DE algorithm.

It can be concluded that ABCDC is outstanding in all the comparisons with other meta-heuristic algorithms. 56

Experiments on function optimization problems

Convergence behavior analysis

The convergence performances of proposed algorithm are analyzed in this subsection. Convergence graphs of compared ABC algorithms as well as the execution time of all the involved methods are presented in the follows.

In fact, convergence curves can vividly show how fast the objective function value decreases along with the increase of function evaluations. And, the convergence processes of compared ABC algorithms to solve six representative benchmark functions are plotted in Figure 2.5. f 4 , f 9 and f 10 are uni-modal while the other three problems are multi-modal. For uni-modal functions, it is obvious that ABCDC and ILTD_ABC converge much faster than other ABC algorithms, including the original ABC. In addition, the proposed algorithm achieves more accurate results than ILTD_ABC does.

For multi-modal functions, the advantages of proposed algorithm is significant which indicates its outstanding exploration ability. Therefore, the convergence curves demonstrate that proposed ABCDC enhances the convergence speed of ABC algorithm effectively.

Conclusion

With the purpose of enhancing the performance of ABC algorithm, an improved ABC with dynamic population composition (ABCDC) is proposed in this chapter.

Firstly, the SLHD is adopted in the initialization phase to ensure the diversity of initial population which can help with the convergence rate. Secondly, we divide the missions of exploration and exploitation more clearly, and distribute them to employed bees and onlooker bees respectively. Accordingly, two DE-inspired solution search strategies are utilized to reinforce the ability of employed bees and onlooker bees. Then, the balance between exploration and exploitation can be improved because the solution search equation used in employed bee phase is good at exploring while the one used in onlooker bee phase shows the strength of exploiting. In addition, a method for adjusting the population's composition is proposed. In order to help the employed bees to explore, its size is set to be very large in the beginning. And as the iteration increases, some promising regions appear, the size of onlooker bees augments gradually. According to the experimental results, ABCDC outperforms the other competitors in terms of solution accuracy.

Chapter 3

Reinforcement Learning based ABC algorithm (ABC_RL)

Introduction

As mentioned in the previous chapters, among many possible reasons that limit the performance of ABC, the ineffectiveness of its search equation has been mentioned the most frequently. Due to the fact that only one dimension can be updated at a time, it has been discovered that the solution search equation lim- ), like the one adopted in the ABCDC algorithm. However, the adaptation approaches still heavily rely on the designer's experience.

its
With the purpose of setting and adjusting the control parameters more intelligently, reinforcement learning (RL) has attracted our attention. RL is one of the most important machine learning approaches that can solve various problems by learning from the interaction between a decision-making agent and an environment [START_REF] Sutton | Reinforcement learning: An introduction[END_REF]. More details will be introduced in the next section. Some RL methods have been embedded with meta-heuristic algorithms to improve their performance. For instance, RL was utilized to select a suitable search strategy for the proposed ABC algorithm (Zhao & Zhang, 2020). To the best of our knowledge, the study of adjusting the parameters of ABC In this chapter, in order to be able to adjust certain important parameters more intelligently while improving the performance of the algorithm, the combination of RL and ABC algorithm is developed and the performance is investigated.

As a result, the improvement strategies can be summarized as follows: firstly, RL is adopted to enlarge and adjust the frequency of perturbation of employed The rest of this chapter is organized as follows. The preliminaries of RL method are explained in section 3.2. In section 3.3, the proposed ABC_RL algorithm is introduced. Experimental studies are presented in section 3.4. Finally, the conclusion is given in the section 3.5. In RL algorithms, except for the agent and environment, there are other main components: actions, states, and rewards that make up a formal framework of RL. The interaction between the agent and its environment is shown in Figure 3.1.

Preliminaries

Each time, the agent selects an action a to perform regarding its current state.

Then, after taking action, its state is updated and a reward is given by the environment. This reward value will be concerned when selecting the next action.

And the loop continues until the termination condition is reached. Suppose the agent has p possible states and q actions can take, the form of the Q table can be found in Table 3.1.

Table 3.1: Form of Q table in Q-learning method Action State a 1 a 2 • • • a q s 1 Q(s 1 , a 1 ) Q(s 1 , a 2 ) • • • Q(s 1 , a q ) s 2 Q(s 2 , a 1 ) Q(s 2 , a 2 ) • • • Q(s 2 , a q ) . . . . . . . . . . . . . . . s p Q(s p , a 1 ) Q(s p , a 2 ) • • • Q(s p , a q )
The values in this

Q new (s t , a t ) = (1 -α) × Q(s t , a t ) + α × r t+1 + γ max a Q(s t+1 , a) , (3.1)
where Q(s t , a t ) is the Q value of acting a t at current state s t , r t+1 indicates the immediate reward after executing action a t . α ∈ [0, 1] denotes the learning rate

REINFORCEMENT LEARNING BASED ABC ALGORITHM (ABC_RL)

and γ is the discount-rate parameter within range [0, 1].

Remark 3.1 γ = 0 is responsible for trading off the importance of immediate and future rewards. When γ = 0, only current rewards are taken into account.

And when γ = 1, Q-learning looks for long-term rewards.

The framework of Q-learning is described in Algorithm 8. In the initialization part, the Q table is generated as a zero-value matrix, and a state s t is arbitrarily selected. Then the learning process starts by repeating several steps (lines 4-7)

until the stopping condition is reached.

Algorithm 8 Pseudo-code of Q-learning algorithm

1: Initialize Q(s,

Proposed ABC_RL algorithm

The proposed RL-based ABC algorithm (ABC_RL for short) is introduced in this section. In ABC_RL, RL is incorporated in the employed bee phase to adjust the number of dimensions to be updated (nb up ) each time. Hence, the search strategy of onlooker bees is improved by collecting information from the global best and two neighbors. And random numbers based on heavy-tailed distribution are used as scale factors in the search strategies.

Scale factors based on heavy-tailed distribution

Randomness has a comparatively large impact on both intensification and diversification of a meta-heuristic algorithm. In basic ABC, a uniformly distributed random number in the range of [-1, 1] is utilized. This restrictive range has been found to be quite narrow, which may reduce the search efficiency. Additionally, if Based on the discussion above, in this work, the heavy-tailed distribution is adopted to help with the randomness. The Mittag-Leffler distribution, one of the most common heavy-tailed distributions, is utilized in the proposed algorithm to generate the scale factors in solution search equations. The definition of Mittag-Leffler distribution is described as follows.

A random variable is said to be subjected to Mittag-Leffler distribution if its distribution function has the following form [START_REF] Huillet | On Mittag-Leffler distributions and related stochastic processes[END_REF][START_REF] Wei | Optimal randomness in swarmbased search[END_REF]:

F β (x) = ∞ k=1 (-1) k-1 x kβ Γ(1 + kβ) , (3.2)
where Γ(•) is the Gamma function. And x > 0, β ∈ [0, 1], F β = 0 for x ≤ 0.

The Mittag-Leffler distribution is heavy-tailed when 0 < β < 1 and it is an

3. REINFORCEMENT LEARNING BASED ABC ALGORITHM (ABC_RL) exponential distribution if β = 1.
For the implementation, a convenient expression proposed by Kozubowski & Rachev (1999) is adopted: Remark 3.2 The random factor τ produced via Eq.(3.3) is adopted in both employed bee phase and onlooker bee phase. And to fit the problem scale, τ is multiplied with a coefficient. According to the different responsibility of the two phases, the corresponding coefficients are set differently. 0.07 × τ is utilized in the employed bee phase whereas onlooker bee phase uses 0.05×τ . The onlooker bee phase is supposed to exploit locally, so its coefficient value is relatively small to avoid jumping over the optimum.

τ β = -γ × ln u sin(βπ) tan(βπv) -cos(βπ) 1/β , (3.3

Employed bee phase with RL

Proposed ABC_RL algorithm

u i,j = x r 1 ,j + F × (x r 2 ,j -x r 3 ,j ), if rand CR or j = j rand x i,j , otherwise (3.4)
where r 1 = r 2 = r 3 = i are randomly selected from {1, • • • , N}, j rand is randomly chosen among all the dimensions. F ∈ [0, 1] is the scale factor while CR is crossover rate. And u i is the newly produced solution based on x i . According to the expression, each time the nb up is possible to be any integer between 1 and D.

Furthermore, a parameter M was adopted to determine the amount of variables to be updated in order to improve information sharing among the colony In this work, the search strategy of employed bees phase will be enhanced from two aspects, i.e., the amount of information learned from the swarm as well as the nb up . DE-based search equation (3.5) is utilized while the RL is introduced to tune the nb up .

v i,j = x k 1 ,j + c 1 × τ × (x k 2 ,j -x k 3 ,j ), (3.5)
where

k 1 = k 2 = k 3 = i are randomly selected from {1, • • • , SN} and j ∈ {1, • • • , D}
is the chosen dimension to be updated. τ is the heavy-tailed random scale factor produced by Eq.(3.3) and c 1 = 0.07.

Adjusting parameter nb up with Q-learning

The method of utilizing Q-learning to define the nb up during the search process is described in the following. The nb up is adjusted by altering the proportion to the total dimension (d ratio ) rather than adding or subtracting a fixed number of dimensions each time in order to tackle problems of various scales. And the range of possible values for the parameter

d ratio is {0.1, 0.2, • • • , 0.9}.
Three actions are defined to adjust the value of d ratio : stays the same, add 0.1 and subtract 0.1. As mentioned before, the reward of taking action a at state s is recorded as the value of Q(s, a) in the Q table. In the proposed algorithm, each

REINFORCEMENT LEARNING BASED ABC ALGORITHM (ABC_RL)

employed bee selects an action according to its associated Q table. After taking the action on its d ratio , the nb up is calculated as below:

nb up = ceil(d ratio × D), (3.6)
where function ceil(•) is used to compute the smallest integer that is greater than or equal to d ratio × D.

And then nb up variables are randomly selected among the D variables to be modified. Thereafter, the new candidate solution is generated via Eq.(3.5).

According to the comparison results between the new solution and the previous one, a reward is given and the state is updated. The last step of implanting Qlearning is to update the corresponding Q value via Eq.(3.1). Notice that, each bee has its corresponding Q table in order to avoid affecting each other.

State and reward set

Since the objective is to minimize the objective functions value, so two states are defined considering the result of the updates.

s 1 : f (v i ) < f (x i )
, the new generated solution is better than the previous one. And the corresponding reward value is set as 1;

s 2 : f (v i ) f (x i )
, the new solution fails to outperform the original solution, in this case the reward value is 0.

Action set

There are three possible actions that can be selected and executed by the agent:

a 1 a 2 a 3 d ratio stays same d ratio + 0.1 d ratio -0.1

Action selection strategy

In fact, always selecting the action with the highest estimated Q value is the simplest selection strategy. However, greedy selection may ignore certain actions with better potential. To get over this weakness, ǫgreedy is proposed to occasionally select the actions with smaller values, while most of the time still select the actions greedily [START_REF] Sutton | Reinforcement learning: An introduction[END_REF]. And ǫ is named the greedy rate. This mechanism allows the Q-learning method to balance exploration and

π(s t , a t ) = max a Q(s t , a), if 1 -ǫ rand, a rand , otherwise, (3.7)
where rand is a random number within [0, 1]. And a rand is randomly selected among the action set {a 1 , • • • , a q } with probability ǫ.

Then, the proposed algorithm is able to alter the nb up considering the historical updating experiences. The pseudo-code of modified employed bee phase is shown in Algorithm 9 in order to present it more clearly. For each candidate solution, an action is firstly selected based on its Q-table (line 4). Hence, those dimensions that will be updated can be determined through lines 5-6. After updating the candidate solution, the state and reward value corresponding to the action are given based on the result of greedy selection (lines 10-17). At the end, the Q-table is updated as expressed in line 18. Note that, the initialization of parameter d ratio will be discussed in subsection 3.4.2.

Algorithm 9 Pseudo-code of RL-based employed bee phase Evaluate Q new (s t , a t ) with Eq.(3.1) and update corresponding Q-table 19: end for (ABC_RL) Remark 3.3 By using RL, the proposed algorithm can not only enlarge the nb up , but also adjust parameter's value considering on-line learning rather than requiring a predefinition by users.

if f (v i ) < f (x i ) then

Improved onlooker bee phase

Considering the weakness of the basic ABC algorithm, an enhanced solution search equation is used in the onlooker bee phase of our algorithm. In the proposed search strategy Eq.(3.8), the information of the global best solution as well as two neighbors are taken into account. Moreover, the scale factor of the second term is also generated with the Mittag-Leffler distribution in order to increase the randomness. Meanwhile, the factor of the third term is set as a random number in a range of [0, 1] to help the newly produced solution learn from the global best solution stably.

v i,j = x i,j + c 2 × τ × (x k 1 ,j -x k 2 ,j ) + θ × (x best,j -x i,j ) (3.8)
where θ ∈ [0, 1] is random number drawn from uniform distribution whereas τ is generated by Eq.(3.3) and c 2 = 0.05. k 1 = k 2 = i are randomly chosen from the colony and x best is the global best solution found so far.

The framework of ABC_RL algorithm

In order to explain the entire process of the proposed algorithm, its framework and flowchart of ABC_RL are shown in Algorithm 10 and Figure 3.2, respectively.

At the beginning, the SN candidate solutions and parameters of Q-learning are initialized. Then in each iteration, the employed bee phase with RL is firstly executed to update the candidate solutions. For each candidate solution, Qlearning is adopted to define and adjust the number of dimensions being updated.

And similar to the standard ABC procedure, onlooker bee phase and scout bee phase are executed afterwards.

Proposed ABC_RL algorithm

Algorithm 10 Pseudo-code of ABC_RL algorithm

1: Generate initial population x i , i = 1, • • • , SN with Eq.(1.1) 2: Evaluate objective function values f (x i ), F ES = SN 3: Initialize Q-tables, d i ratio = 0.2 for i = 1, • • • , SN 4: while F ES ≤ max_F ES do
% RL-based employed bee phase % 5:

for i = 1 → SN do 6: Randomly select k 1 = k 2 = k 3 = i from {1, • • • , SN} 7:
Choose an action via Q- Produce v i,j via Eq.(3.5)

13:

end for

14: if f (v i ) < f (x i ) then 15:
Replace x i with v i ; trial i = 0

Randomly select k 1 = k 2 = i and j rand from {1, • • • , D} 29:
Generate the v i,j rand via Eq.(3.8)

30: if f (v i ) < f (x i ) then 31: Replace x i with v i ; trial i = 0 32: else trial i = trial i + 1 33: end if 34:
end for % Scout bee phase %

35:

for i = 1 → SN do 36:

if trial i > limit then 37:

Generate new position with Eq.(1.1), trial i = 0

Experiments on function optimization problems

In this part, the initialization method of parameter d ratio was studied at first. Hence, the experimental verification was done by comparing the proposed algorithm with five state-of-art ABC variants on CEC 2017 benchmark problems in different dimension cases.

CEC 2017 benchmark problems

In the experiments, 29 CEC 2017 benchmark problems [START_REF] Awad | Problem definitions and evaluation criteria for the CEC 2017 special session and competition on single objective real-parameter numerical optimization[END_REF] were chosen to be solved as various kinds of single-objective optimization problems were contained. There are 2 unimodal functions (f 1 , f 3 ), 7 simple multimodal functions (f 4f 10 ), 10 hybrid functions (f 11f 20 ) and 10 composition functions (f 21 -f 30 ). In fact, it is hard to distinguish the algorithms' performance on certain classical benchmarks as most of the improved ABC variants can find the optima effectively. In this case, the CEC 2017 benchmarks are selected to better test the compared methods.

According to the evaluation criteria of CEC 2017, the search space for all the benchmarks is defined as [-100, 100] D . The determination condition is set in terms of the maximum number of function evaluations and problem dimension, i.e., max_F ES = 10 4 ×D. Moreover, the official code is accessible online * . Notice that the function f 2 is not included in the following experiments because it has been deleted in the latest official code.

Effects of the initial value of parameter d ratio

The parameter d ratio is used to control the frequency of perturbation in ABC_RL.

For each employed bee, each time d ratio × D dimensions of the previous candidate solution will be updated. Note that d ratio can take value from the set {0.1, 0.2, • • • , 0.9} and is defined by Q-learning algorithm. In this section, the setting of the initial value of parameter d ratio is studied since the initialization method can impact the following search process. * https://github.com/P-N-Suganthan/CEC2017-BoundContrained Table 3.2 presents the comparison results of these ABC_RL variants on benchmark problems with D = 10. In the comparison, the mean of function error values f (X best )f (X * ) over 25 independent runs are calculated, where X best is the best solution found by algorithm and X * is the exact global optimum. For each function, the best results are marked in boldface. In order to better analyze the results, the Friedman test and Wilcoxon tests are conducted based on the average errors obtained by algorithms. The rankings in the last row are evaluated by Friedman test. The best ranking is marked in boldface. The Wilcoxon test is able to compare the difference between the two methods, so each ABC_RL variant is compared to the original ABC algorithm. The symbol "+/=/-" indicates the number of functions that ABC_RL is better, similar, or worse than ABC.

According to the comparison, the ABC_RL 0.2 perform the best and is followed by the version with random initialization ABC_RL rand . It can be found that, when the parameter d ratio is defined relatively large in the beginning, the final results are less satisfying. Likewise, the same conclusion can be drawn from Wilcoxon test results. ABC_RL 0.2 outperforms the original ABC on 19 functions while ABC_RL 0.8 surpasses ABC on 16 functions. Therefore, when the initial value of parameter d ratio is defined relatively small, the proposed algorithm is able to show better performance. And the initial value of d ratio is set as 0.2 in the following part of experiments. Furthermore, all the concerned ABC_RL versions

REINFORCEMENT LEARNING BASED ABC ALGORITHM (ABC_RL)

attained better rankings than the standard ABC algorithm did, which can verify the effectiveness of improvement strategies.

Comparison with ABC variants

To evaluate the performance of ABC_RL, four effective ABC variants and the standard ABC are involved in the following experiments. In order to compare fairly, the control parameters of the competitors are set the same as those of their original papers which are presented in Table 3. 3.4 shows the performance of the compared algorithms on 10-dimensional problems. The advantages of the proposed algorithm are significant, it obtains the best results on 18 functions (f 3 , f 5f 12 , f 17f 20 , f 22 , f 23 , f 27 , f 29 , and f 30 ).

Experiments on function optimization problems

REINFORCEMENT LEARNING BASED ABC ALGORITHM (ABC_RL)

These variants are compared with the standard ABC aiming at verifying the effectiveness of the proposed strategies. Experimental results on 29 benchmarks over 51 independent executions are shown in Table 3. For all the dimension cases, ABC_RL always gets the best ranking. Meanwhile, the rankings of ABC_RL eq and ABC_RL QL are both better than those of ABC.

Hence, the effectiveness of the proposed strategies can be proved. And it can be concluded that these strategies work better together than they do individually.

Convergence behavior analysis

The convergence curves of compared algorithms were plotted to fully compare their performance. The convergence speed of solving different types of problems with D = 10 can be observed in Figure 3 ABC_RL is able to achieve a better result in terms of solution precision.

In addition, as mentioned at the beginning of this work, it is worth studying the parameter adjusting of ABC via the RL method. As the convergence rate of ABC is acknowledged to be limited, the improvement strategies should avoid decelerating the algorithm even if the main purpose is not to enhance the convergence speed. In this context, the analysis of the influence of RL on the convergence rate of ABC is meaningful. Therefore, the convergence process of variant ABC_RL QL is compared to that of the original ABC algorithm and the final ABC_RL algorithm as presented in Figure 3.6.

According to Figure 3.6, it can be observed that the trend of convergence curves of the three involved algorithms is basically the same, especially in figures In this way, the impact of using the Q-learning method to adjust the parameter nb up on the convergence rate can be analyzed. The above observations indicate that in complex problems, the exploring efficiency of ABC may be slightly affected by RL-based strategy only in the early stage, but better solutions can be found later with the help of RL. One possible reason for this situation is that the learning experience of RL is not enough in the early stage, so the instability of RL may affect the convergence speed of external ABC algorithm. Nonetheless, from the

Conclusion

middle and late stages, RL is able to gradually provide more useful help to the algorithm.

Moreover, the convergence performance of the final ABC_RL provides strong support for utilizing the other improving strategies simultaneously. The algorithm is able to keep a nice balance between diversification and intensification via adopting the enhanced solution search equations (i.e., Eq.(3.5) and (3.8)) with RL-based parameter tuning. Thus, when all these strategies are applied together, the proposed ABC_RL can not only converge faster than basic ABC, but also reach outstanding solutions.

Conclusion

In this chapter, to improve the search efficiency of ABC, a RL-based ABC algorithm (named ABC_RL) is proposed. Different from the ABCDC in the previous chapter and many other existing ABC versions, RL is used to intelligently adjust a parameter of ABC_RL. More precisely, since the frequency of perturbations has a significant impact on the performance of algorithm, the Q-learning method is used to change the frequency by learning from the historical updating experience in employed bee phase. Moreover, the information of the global best and two neighbors is considered in the onlooker bee phase to better lead the search direction. Furthermore, random scale factors drawn from a heavy-tailed distribution are adopted in the two search strategies to enrich the algorithm's randomness.

Experiments were carried out on 29 CEC 2017 benchmark problems. ABC_RL was compared to five effective ABC algorithms in different dimension cases. The comparison results demonstrated that the proposed algorithm outperformed the other competitors in terms of solution accuracy and overall performance. This part of work also provides a novel way of incorporating RL into SIAs.

Introduction

One of the primary purposes of improving the optimization algorithms is to apply them to solve problems of practical interest with the purpose of reducing cost consumption and increasing efficiency. In this context, compared to the previous chapters, this part of work keeps in mind the situations of solving real-world problems while carrying out the improvement of the ABC algorithm. These days, there exist many on-line optimization problems, such as on-line robot path planning. And the time required for the algorithm to compute during the process has become another essential factor to be considered. That is to say, we aim at finding the improvement strategies that can ensure the solution accuracy while taking computational complexity and operation efficiency into account. The motivation for this derives from the fact that many modification strategies that are effective in resolving functional optimization problems do not always help us find the ideal answer rapidly in practical applications. Therefore, it is also meaningful to improve the algorithm's performance without overcomplicating it.

As mentioned in the previous chapters, ABC has been discovered to be excellent at diversification but not so effective in terms of intensification. This also leads to the convergence speed of ABC being relatively slow in certain problems.

In order to tackle this weakness, the solution search equation of ABC needs to be enhanced since only one neighbor of uncertain quality is considered. Moreover, the purpose of scout bee phase is to avoid stagnation. More precisely, the equation for producing new solutions is adopted to replace those food source positions that have not been updated for a long time. However, the quality of the newly generated solutions can not be guaranteed since the previous search experience is ignored. This is also one of the reasons for the low convergence speed of ABC.

In addition, researchers found that the scout bee phase demonstrates is occasionally redundant in the search process [START_REF] Anuar | A modified scout bee for artificial bee colony algorithm and its performance on optimization problems[END_REF]. In this case, the scout bee phase has also been improved in some ABC variants, and some for adjusting the population composition enables ABCDC to abandon the worst solutions timely. In this part, we would like to maintain the scout bee phase and try to strengthen it. More precisely, the successful search experience is concerned when producing new solutions, so that more valuable solutions can be transmitted to the subsequent iteration.

With the purpose of enhancing the exploitation ability and saving computation time efficiently, the search strategy of the teaching-learning based optimization (TLBO) algorithm was chosen to be incorporated into the proposed algorithm. It has been discovered that the TLBO algorithm converges quickly and is effective at exploitation. At the same, the onlooker bee phase of ABC is mainly responsible for exploiting locally. After the employed bees search the whole space, the onlookers will search locally around the solutions with higher quality. In this context, it is natural to attempt to embed the advantages of TLBO into the onlooker bee phase. Moreover, the way of choosing the regions to be further exploited can be made better. The basic ABC algorithm uses the roulette wheel selection method in onlooker bee phase for selecting the solutions to be updated. However, this process has the risk of time consumption. Consequently, the onlooker bee phase of the proposed algorithm is enhanced by the strategy of the learning phase of TLBO. And there is no longer a need to calculate probabilities and use the roulette selection method, which can help streamline the algorithm.

Therefore, in this chapter, enhancing the performance of ABC while avoiding it becoming too complex is investigated. In this context, a learning-based ABC (ABCL) algorithm is proposed for improving the exploitation ability as well as the search efficiency. So that more energy and time can be saved when solving problems like local path planning. The proposed improvement strategies include:

firstly, the global best solution is adopted in the employed bee phase and scout bee phase to guide the swarm in a promising search direction. Secondly, the learning phase of TLBO is embedded in the onlooker bee phase to improve the exploitation ability and simplify the computational complexity. Then, the performance of the proposed algorithm has been verified through numerical optimization problems.

The rest of this chapter is organized as follows. The standard TLBO algorithm is introduced in section 4.2. In section 4.3, the proposed algorithm is explained

LEARNING BASED ABC ALGORITHM (ABCL)

in details. In section 4.4, the experimental studies are presented. Finally, the conclusion is given in the section 4.5.

Remark 4.1 The ultimate objective is to solve on-line multi-robot path planning (MRPP) problems more effectively. And the implementation method and simulation tests are given in Chapter 6 along with other applications.

Preliminaries

Initialization

Same initialization method is utilized in TLBO as the Eq.(1.1) of ABC algorithm.

For solving a D-dimensional optimization problem,

x i = (x i,1 , x i,2 , • • • , x i,D
) is the i th learner and i = 1, • • • , NP where NP is the population size.

Teaching phase

In the teaching phase, the learner with the best fitness value is appointed as the T eacher. The learners are improved via the following equations.

tf = round(1 + rand(0, 1)), (4.1)

Proposed ABCL algorithm

x new i = x i + rand × (T eacher -tf × Mean), (4.2)
where x new i is the new state of the i th learner. T eacher is the learner with the best fitness value and Mean is the average state of the population. tf is a teaching factor while rand is a vector with random values in the range of [0, 1]. The better learner is then selected greedily and enters the learning phase.

Learning phase

Then in the learning phase, learners use the Eq.( 4.3) to improve their knowledge levels.

x new i = x i + rand(0, 1) × (x k -x i ) if f (x k ) < f (x i ), x i + rand(0, 1) × (x i -x k ) otherwise, (4.3)
where x new i is the new updated learner of x i while x k is randomly selected from the rest of the class. And a random vector in range of [0, 1] is generated as the scale factor of Eq.(4.3).

Then, the fitness values of the two learners f (x i ) and f (v i ) are compared together, the same selection method is adopted. After this, the search process enters into the teaching phase again and will repeat until the termination conditions are reached. The pseudo-code of the TLBO algorithm is shown in Algorithm 11.

Proposed ABCL algorithm

With the purpose of achieving problems like on-line MRPP, it is essential to further improve the efficiency of the path planner. In fact, in the case of planning paths practically, the number of iterations in the algorithm may be very limited. Hence, it is necessary to keep a nice balance between intensification and diversification during the search process. Moreover, avoiding augmentation of the algorithm's computational complexity is another critical target. Otherwise, even if a better solution can be found, it is not suitable for the concerned problem.

For meta-heuristic algorithms, exploration is related to the discovery of the whole search space while exploitation refers to search for better solutions in promising regions based on the previous experience. As for ABC, its power in exploration has been widely proved while it also exposes the shortcomings of weak exploitation ability. Therefore, in order to generate safe paths effectively,

LEARNING BASED ABC ALGORITHM (ABCL)

Algorithm 11 Pseudo-code of TLBO algorithm 1: Initialize the learners with Eq.(1.1) and evaluate the initial population 2: repeat % Teaching phase % 3:

for i = 1 → NP do 4:
Generate factor tf with Eq.(4.1) Based on the discussions above, an improved search strategy adopting both of the aforementioned ideas is proposed in Eq.(4.4) as

v i,j = x i,j + ρ i,j × (gbest j -x i,j ) + µ i,j × (x k 1 ,j -x k 2 ,j ), (4.4)
where v i,j is the updated j th element of the i th food source. And gbest j is the j th As seen from Eq.(4.4), the global best position is utilized to lead the colony in a good direction as the scale factor ρ i,j is a positive random number. Meanwhile, different from the previous ABC variants with global best solution (eq. GABC (Zhu & Kwong, 2010)), the last term µ i,j ×(x k 1 ,j -x k 2 ,j ) selects two other solutions.

In this case, more colony information can be learned. However, considering the information may not always be favorable, the coefficient µ i,j is defined within a smaller range [-0.5, 0.5].

Learning-based onlooker bee phase

In basic ABC, the onlooker bees are supposed to search around the qualified food sources which correspond to exploitation. It has been observed that ABC is weak in exploitation locally. At the same time, the TLBO algorithm has been found to be good at exploitation but relatively poor at exploration. Based on

LEARNING BASED ABC ALGORITHM (ABCL)

the complementarity of ABC and TLBO, a hybrid algorithm was proposed by Chen et al. (2018), which embedded the teaching phase of TLBO in employed bee phase and combined the learning phase with onlooker bee phase.

Similarly, in this work, the learning strategy of TLBO is incorporated in the onlooker bee phase so that individuals can learn interactively within the colony.

Different from the work of Chen et al. (2018), the proposed learning-based search strategy (i.e., Eq.(4.5)) is adopted by all the individuals without the roulette wheel selection method. In original ABC and its numerous variants, the roulette wheel selection method is adopted to select the high-quality candidate solutions in onlooker bee phase. As explained in section 1.3, the selection probability of each candidate solution is proportional to its fitness value. The greater the fitness, the more likely the individual is selected. Nonetheless, this process lessens the population diversity and may prolong the computation time. Thus, in the proposed ABCL algorithm, the calculations of fitness values, probabilities, and roulette wheel selection are dropped to avoid wasting time in path planning.

The learning-based search equation is expressed as below.

v i,j = x i,j + rand × (x k,j -x i,j ) if f (x k ) < f (x i ), x i,j + rand × (x i,j -x k,j ) otherwise, (4.5)
where k = i are randomly selected from {1,

Enhanced scout bee phase

The scout bee phase is responsible for supervising the qualities of candidate solutions. The food sources that haven't been updated over predefined limit times will be abandoned. However, the equation of generating new solutions only uses

Experiments on function optimization problems

the lower and upper bound values, which is a waste of known information. In other words, if the search experience can be considered when producing a novel solution, the newly generated one will have a higher possibility to be in a hopeful position. In this case, the enhanced scout bee phase is more likely to avoid returning to the explored regions blindly. Hence, the global best information is considered as well as the boundary values in the proposed Eq.(4.6).

x i,j = gbest j + φ i,j × (x max.jx min,j ),

j = 1, • • • , D, (4.6)
where gbest is the global best solution while x min and x max are the lower and upper bounds respectively. To avoid losing randomness, the scale factor φ is a uniform random number in range of [-1, 1].

The framework of ABCL algorithm

In order to explain the entire process of proposed algorithm, the pseudo-code of ABCL is shown in Algorithm 12 while the flowchart is represented in Fig-

LEARNING BASED ABC ALGORITHM (ABCL)

Algorithm 12 Pseudo-code of ABCL algorithm for i = 1 → SN do 6:

1: Generate initial population x i , i = 1, • • • ,
Randomly select k 1 = k 2 = i from {1, • • • , SN} 7:
Randomly select dimension j from {1, • • • , D}

8:

Produce v i via Eq.(4.4)

9: if f (v i ) < f (x i ) then 10:
Replace x i with v i ; trial i = 0 11: else 12:

trial i = trial i + 1 13: end if 14:
end for % Learning-based onlooker bee phase % 15:

for i = 1 → SN do 16:

Randomly select k = i from {1, • • • , SN} 17:
Randomly select dimension j from {1, • • • , D}

18:

Generate updated individual v i,j with Eq.(4.5)

19: if f (v i ) < f (x i ) then 20:
Replace x i with v i ; trial i = 0

Benchmark functions

The definitions of the benchmarks can be found in Table 4.1. Among these optimization problems, f 1f 2 and f 4 are uni-modal functions while f 3 is the discontinuous step function. f 5 is the Rosenbrock function which is a uni-modal problem when D = 2 and 3, but probably has multiple optima in higher dimensions [START_REF] Kang | Rosenbrock artificial bee colony algorithm for accurate global optimization of numerical functions[END_REF]. As for the rest part, f 6f 12 are multi-modal functions.

f 1 (x) = D i=1 (10 6 ) i-1 D-1 x 2 i [-100, 100] D 0 f 2 (x) = D i=1 |x i | (i+1) [-1, 1] D 0 f 3 (x) = D i=1 (⌊x i + 0.5⌋) 2 [-100, 100] D 0 f 4 (x) = exp(0.5 D i=1 x i ) [-10, 10] D 0 f 5 (x) = D i=1 [100(x i+1 -x 2 i ) 2 + (x i -1) 2 ] [-5, 10] D 0 f 6 (x) = D i=1 [x 2 i -10cos(2πx i ) + 10] [-5
.12, 5.12] D 0

f 7 (x) = D i=1 [y 2 i -10cos(2πy i ) + 10] y i = x i |x i | < 1 2 round(2xi) 2 |x i | ≥ 1 2
[-5.12, 5.12] D 0 2 [1 + 10 sin 2 (πy i+1 )] +(y D -1) 2 } + D i=1 u(x i , 10, 100, 4)

f 8 (x) = 418.98288727243380 × D -D i=1 x i sin( |x i |) [-500, 500] D 0 f 9 (x) = π D {10 sin 2 (πy 1 ) + D-1 i=1 (y 1 -1)
y i = 1 + 1 4 (x i + 1), u xi,a,k,m =    k(x i -a) m x i > a 0 -a ≤ x i ≤ a k(-x i -a) m x i < -a    [-100, 100] D 0 f 10 (x) = 1 10 {sin 2 (πx 1 ) + D-1 i=1 (x i -1) 2 [1 + sin 2 (3πx i+1 )]+ (x D -1) 2 [1 + sin 2 (sπx i+1 )]} + D i=1 u(xi, 5, 100 , 4)
[-100, 100] D 0

f 11 (x) = D-1 i=1 (x i -1) 2 [1 + sin 2 (3πx i+1 )] + sin 2 (3πx 1 ) + |x D -1|[1 + sin 2 (3πx D )] [-10, 10] D 0 f 12 (x) = D i=1 kmax k=0 [a k cos(2πb k (x i + 0.5))] -D kmax k=0 [a k cos(2πb k 0.5)] a = 0.5, b = 3, k max = 20
[-0.5, 0.5] D 0 Remark 4.2 Since this algorithm is expected to perform well in practical problems, the experimental study part does not use too complicated optimization problems. Its effectiveness in solving the MRPP problem will be given in the next Chapter 6.

Comparison with ABC variants

In the experiments, the swarm size is set as SN = 30 for all the comparative algorithms. The control parameter limit is defined as SN × D and max_F ES = times on all the problems. Then the statistical results are calculated and compared in Table 4.2 -Table 4.4. The mean and standard deviation (std) of errors f (X best )f (X * ) over the 25 runs are calculated, where X best is the best solution calculated by algorithm and X * is the exact global optimum.

In order to analyze the comparison results more comprehensively, the Wilcoxon tests are carried out to show the significance between ABCL and other concerned algorithms while the Friedman test is adopted to evaluate the overall performances of all the concerned algorithms. The test results can be found at the bottom of the comparison tables. The symbol "+/=/-" indicates the number of functions where ABCL is better than, similar to, or worse than the competitor, respectively. At the end, the average rankings are presented in Figure 4.2. 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 f 4

1.93E-22 0.00E+00 1.93E-22 0.00E+00 1.93E-22 0.00E+00 1.93E-22 0.00E+00 f 5 4.91E-02 4.50E-02 1.66E-01 1.80E-01 2.75E-01 6.16E-01 1.43E-02 1.72E-02 f 6 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 f 7 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 f 8 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 +/=/-7/5/0 4/8/0 3/8/1 -According to the comparison results with D = 10, the difference in performance of the compared algorithms is small as the low-dimensional case is simple to solve. Even so, the advantages of ABCL and GABC algorithms are noticeable, especially in terms of their high accuracy on functions such as f 1 and f 2 . Similar conclusions can be derived from the Wilcoxon test results, the proposed algorithm performs better than GABC does on 3 problems. Meanwhile, they obtain very close results on 8 out of 12 problems. Compared with the standard ABC algorithm, ABCL surpasses it on more than half of the benchmarks. Moreover, it can be seen that the performance of DABC is also remarkable. It fails to outperform ABCL on 4 out of 12 problems.

LEARNING BASED ABC ALGORITHM (ABCL)

Conclusion

In this chapter, an improved ABC algorithm named ABCL is proposed with the ultimate goal of solving the on-line MRPP problems (see Chapter 6). Hence, several improvement strategies have been adopted in order to enhance the intensification as well as the search efficiency. In the first place, the global best information is used to reinforce the algorithm. A solution search equation based on the global best and more neighbors is utilized in the employed bee phase. Secondly, the learning phase of the TLBO algorithm is introduced into the onlooker bee phase to improve the exploitation ability and search efficiency. Moreover, the scout bee phase is also enhanced with the global best solution. In this case, more useful information can be considered when generating a new candidate solution. Furthermore, the effectiveness of ABCL is proved through comparisons in solving benchmark optimization problems.

Introduction

It can be found that, in most ABC variants, the solution search equations are often improved by enlarging the number of dimensions to be updated and expanding the quantity of information that may be learned from the colony. The proposed ABC algorithms in Chapters 2-4 also improve ABC by this idea. However, these kind of improvements disregard individuals' prior search history, which could result in the loss of valuable information. In fact, from a mathematical perspective, it can be found that the majority of improved ABC variants search the optima via integer-order operations. In addition to the common improvement strategies that were reviewed in the first chapter, new idea is produced and researched in this chapter. More precisely, it is interesting to investigate whether combining fractional order calculus will help enhance the ABC algorithm.

In this context, the fractional-order calculus (FOC), a novel mathematical method is embedded into ABC algorithm to reinforce the exploitation ability and improve the solution precision. In fact, compared to the integer-order derivative, the fractional-order derivative contains entire memory of its previous events In this context, it has been observed that incorporating FOC into the ABC algorithm is very worthy to be studied. More specifically, the integration of FOC perspective into ABC enables the algorithm to access previous solutions from memory to find qualified solutions more efficiently. Meanwhile, ABC can also benefit from the acceleration in convergence as FOC has done for other metaheuristic algorithms.

Therefore, in this chapter, in order to make full use of the individuals' memories, a FO-based search strategy is proposed in the onlooker bee phase of ABC.

At the same time, a differential search strategy is adopted in the employed bee phase of the proposed FOABC algorithm to reinforce its diversification ability. The outline of this chapter is as follows. The proposed algorithm is introduced in section 5.2. Experimental studies are presented in section 5.3 with the discussions. The conclusion is given in the last section 5.4.

Proposed FOABC algorithm

In this section, the proposed FOABC will be explained in details. In the employed bee phase, a DE-based search strategy is adopted to reinforce the diversification of algorithm. Secondly, the FOC is adopted in the onlooker bee phase in order to improve the local-search ability. Moreover, random numbers drawn from the Lévy distribution are utilized in those two search equations.

Scale factors based on Lévy distribution

As one of the main components of the search equation, scale factor can affect the step size of each move. More precisely, this factor is usually a random number, depending on the range and distribution laws. In the standard ABC algorithm,

FRACTIONAL-ORDER ABC ALGORITHM (FOABC)

the scale factor φ ∈ [-1, 1] has been found to be insufficiently efficient. Additionally, when algorithm falls into a local optimum, the restricted range of φ may limit the algorithm to jump out quickly. In addition to the popular distributions like normal distribution and uniform distribution, the Lévy distribution [START_REF] Lévy | Théorie de l'addition des variables aléatoires[END_REF]) is also very important in the field of meta-heuristics. [START_REF] References Lee | Evolutionary programming using mutations based on the Lévy probability distribution[END_REF] mentioned that the Lévy probability distribution has an infinite second moment.

In this case, it is more probable to produce an offspring that is spatially far from its parent in the concerned evolutionary programming (EP) algorithm. Lévy flight is a special case of random walk whose step lengths obey the Lévy distribution which can be defined as below [START_REF] Shlesinger | Levy flights: Variations on a theme[END_REF]; Yang, 2020): where u and v are drawn from normal distributions as:

Lévy(s) ∼ |s| -1-β , ( 5
u ∼ N(0, σ 2 u ), v ∼ N(0, σ 2 v ), (5.3)
with

σ u = Γ(1 + β)sin(πβ/2) βΓ[(1 + β)/2]2 (β-1)/2 1/β , σ v = 1, (5.4)
where Γ(•) is the Gamma function and is defined as follows.

Γ(z) = ∞ 0 t z-1 e -z dt, (5.5)
when z is an integer, Γ(z + 1) = zΓ(z) = z!.

Note that the step length mentioned above plays the same role as the random factor φ in search equation (1.2). In the proposed algorithm, the scale factors l in search strategies are generated by the following Eq.(5.6).

l = α × s, (5.6)
where s is the step length generated via Eq.(5.2) and α > 0 is a factor. In the basic CS, α is used to adjust the step size for Lévy flight considering the problem scale. [START_REF] Sharma | Lévy flight artificial bee colony algorithm[END_REF] fixed this factor to 0.001 in their search strategy. And α = 0.01 in the codes of CS proposed by researchers who develop it. In fact, this factor is also important since it can probably influence the effectiveness of the search strategies. If α is defined with a large value, then the novel solution

FRACTIONAL-ORDER ABC ALGORITHM (FOABC)

may jump over the optimal position or even go outside the design domain as the step size is too large. On the other hand, if the value of α is too small, then the convergence rate will be affected. In this case, instead of using a constant throughout the search process, the logistic map is applied on the factor α in [START_REF] Liu | An artificial bee colony algorithm based on dynamic penalty and lévy flight for constrained optimization problems[END_REF]. The value of factor α changes along with the iteration and the logistic dynamic map can help the algorithm to prevent the situation of falling into local optima.

In our proposed algorithm, the two search equations in employed bee phase and onlooker bee phase both adopt the random numbers generated via Eq. (5.6).

And based on our experimental tests, α 1 = 0.7 for the Lévy-based random number in employed bee phase. The factor α 2 in onlooker bee phase should be set relatively small to avoid missing the optimal solution so α 2 = 0.5.

Remark 5.1

The possible interval for α is pretty large. And varying the values of α may also help to improve the algorithm performance when solving different kinds of problems.

Differential search strategy for employed bee phase

As mentioned previously, the search strategy of ABC is not efficient enough because it modifies only one variable of the food source positions each time. Moreover, limited information can be learned from the swarm as only one solution is randomly selected from the swarm. In most of ABC variants, the nb up (number of dimensions being updated) in the enhanced solution search equations is still one dimension each time. In order to tackle this weakness, differential search strategies were borrowed from the DE algorithm (Chen et al., 2019b; Li & Yin, 2014;

Zorarpacı & Özel, 2016) because the mutation and crossover operators enable the individuals to explore better via a sudden change. By using a predefined control parameter CR, the nb up could be greater than one at each time.

Thus, a differential search strategy is utilized in the employed bee phase.

To avoid getting over complicated, the classical but efficient DE search strategy "rand/1/bin" is adopted. The search strategy in employed bee phase of FOABC is shown as below:

v i,j = x r 1 ,j + l 1 i × (x r 2 ,j -x r 3 ,j ), if rand CR or j = j rand , x i,j , otherwise, (5.7)
where v i is the updated solution of x i . r 1 = r 2 = r 3 = i are randomly chosen from {1, • • • , SN}. And j rand is randomly selected from {1, • • • , D}. CR is the crossover rate while l 1 i is scale factor generated via Eq.(5.6) with α 1 = 0.7. For each dimension, if the condition of is satisfied, the new solution will get information from the mutating vector. Otherwise, it will keep the same as the original food source x i . In this case, by embedding the DE-based operators into FOABC, the new candidate solutions are able to learn more information from other food sources and inherit from the previous ones at the same time. Moreover, the conditions in selection process allow the individuals update more than one dimension each time.

Fractional-order search strategy for onlooker bee phase

As mentioned in the beginning, compared to an integer-order derivative, fractionalorder derivative has a memory of previous incidents [START_REF] Couceiro | Introducing the fractional-order darwinian PSO[END_REF][START_REF] Mousavi | Fractional calculus-based firefly algorithm applied to parameter estimation of chaotic systems[END_REF]Yousri et al., 2020). Considering its excellent competence of describing the historical information, the FOC is adopted in the onlooker bee phase in order to enhance the local-search ability as well as the solution precision.

Fractional-order calculus definition

Nowadays, three common definitions of fractional-order derivatives have been widely approved and used: Grunwald-Letnikov, Rieman-Liouville and Caputo definitions [START_REF] Gu | Synchronization-based parameter estimation of fractional-order neural networks[END_REF]. In this chapter, the definition of Grunwald-Letnikov (GL) derivative will be used and it can be implemented as Eq.(5.8) [START_REF] Podlubny | An introduction to fractional derivatives, fractional differential equations, to methods of their solution and some of their applications[END_REF].

D q [x(t)] = lim h→0 1 h q ∞ n=0 (-1) n q n x(t -nh), (5.8)
where q n = Γ(q + 1) Γ(n + 1)Γ(qn + 1) = q(q -1)(q -2)

• • • (q -n + 1) n! ,
(5.9)

where D q (•) denotes the GL fractional derivative of order q. Γ(•) is the Gamma function defined in (5.5).

FRACTIONAL-ORDER ABC ALGORITHM (FOABC)

The definition of GL in Eq.(5.8) can be formulated as below in the discretetime implementation.

D q [x(t)] = 1 T q r n=0
(-1) n Γ(q + 1)x(t -nT ) Γ(n + 1)Γ(qn + 1) , (5.10)

where T is the sampling period and r indicates the number of terms of the previous memory. Considering a special case that q = 1, then the definition can be expressed as follows.

D 1 [x(t)] = 1 T (x(t) -x(t -T )) = x t -x t-1 . (5.11)
It can be found that D 1 [x(t)] represents the difference between two events at adjacent moments, x t and x t-1 .

Proposed fractional-order solution search equation

As mentioned before, the FO derivative keeps memory of previous events compared to the integer-order derivative. And FOC has been verified that it is suitable to be used to describe the dynamic phenomena such as the trajectory of fireflies [START_REF] Mousavi | Fractional calculus-based firefly algorithm applied to parameter estimation of chaotic systems[END_REF]). In the exploitation stage, FOC enables the algorithm to use information from earlier solutions when looking for new ones, thus determining a more plausible solution and changing the convergence tendency [START_REF] Couceiro | Introducing the fractional-order darwinian PSO[END_REF]. Therefore, considering its excellent ability in describing historical information, FOC is employed in the onlooker bee phase to enhance the local search ability of ABC.

Firstly, in order to show the derivation process more clearly, we rewrite the Eq.(1.2) as following:

x t+1 i,j = x t i,j + l 2 i,j × (x t i,jx t k,j ), (5.12) where x t+1 i,j is the newly produced solution which is the same as v i,j and k = i is randomly selected among {1, • • • , SN}. Note that instead of generating a random number under normal distribution, the coefficient l 2 i,j is produced via Eq.(5.6) in subsection 5.2.1.

Proposed FOABC algorithm

According to the difference of two followed events in Eq.(5.11), the Eq.( 5.12) can be then reformulated as

x t+1 i,j -x t i,j = l 2 i,j × (x t i,j -x t k,j ), (5.13)
then it is easy to get the following formulas when q = 1:

D 1 [x(t + 1)] = x t+1 i,j -x t i,j = l 2 i,j × (x t i,j -x t k,j ).
(5.14)

The derivative definition above for q = 1 can be generalized as Eq.( 5.15) with any derivative order q, D q [x t+1 i,j ] = l 2 i,j × (x t i,jx t k,j ).

(5.15)

Next, we are able to formulate the Eq.( 5.15) by employing the discrete-time GL definition Eq.( 5.10) with T = 1. The novel expression are shown below:

D q [x t+1 i,j ] = x t+1 i,j + r n=1
(-1) n Γ(q + 1)x t+1-n i,j Γ(n + 1)Γ(qn + 1) = l 2 i,j × (x t i,jx t k,j ), (5.16) where r is the number of steps to record.

Based on Eq.(5.16), the general expression of the proposed fractional-order ABC solution search equation is obtained and expressed in the following:

x t+1 i,j = - r n=1 (-1) n Γ(q + 1)x t+1-n i,j Γ(n + 1)Γ(q -n + 1) + l 2 i,j × (x t i,j -x t k,j ), (5.17)
where r is the number of memory terms and q is derivative order. Γ(•) is Gamma function. And j is randomly selected dimension from {1, • • • , D}, k = i is randomly chosen from {1, • • • , SN}. The random factor l 2 i,j is generated by Eq.(5.6) with α 2 = 0.5.

For instance, r = 4, which means that the latest four terms will be stored in the memory of each food source and will be utilized to produce a new candidate solution via the Eq.(5.17). In this case, the food sources will be updated as follows.

FRACTIONAL-ORDER ABC ALGORITHM (FOABC)

x t+1 i,j = 1 1! qx t i,j + 1 2! q(1 -q)x t-1 i,j + 1 3! q(1 -q)(2 -q)x t-2 i,j + 1 4! q(1 -q)(2 -q)(3 -q)x t-3 i,j + l 2 i,j × (x t i,j -x t k,j ).
(5.18)

It is evident that the values of parameters q and r can influence the final results. However, it is relatively difficult to understand their effects, especially the fractional order q. This drove the existing FO-enhanced meta-heuristic algorithms to set these parameters basically based on experimental validation. The following analysis takes Eq.( 5.18) as an example search equation, and for the sake of clarity, the first four terms related to FOC are named as memory terms and the last term is called differential term.

Observing Eq.(5.18), when q is taken smaller in the range of [0,1], the coefficients of memory terms are also smaller, so the historical memory has less influence on the newly generated solution. In this way, the differential term may dominate the updating results. That means the historical information cannot play its role well. Put another way, the summation of the memory terms is equivalent to the base term x t i,j in the original solution search equation. That is, if q is defined too small, the base term will be scaled to be smaller. Then, the local search will be affected, because the algorithm cannot approximately locate the position of base term. Hence, the local search may not be performed effectively.

On the other hand, the memory terms play a greater role when q takes a larger value, especially around 0.5. As mentioned before, in the onlooker bee phase, the selection probability of the solutions to be updated is higher, that is, their history information is more useful. In this case, the memory terms can play an active role in the proposed search equation. And in contrast to the previous case, when q is larger, the algorithm can better perform a local search around the base term.

As for the parameter r, the memory length, is relatively easy to understand, of course. Note that r taking a value too large may cause the algorithm to take more time on computation. Based on the above analysis, since the FO-based strategy is supposed to boost the local search, taking a relatively large value of q in [0, 1] is more likely to yield the desired result. Furthermore, experiments are done to fully evaluate the impact of these two parameters on the performance of FOABC while completing the analysis of their influence.

Proposed FOABC algorithm

Implementation steps of modified onlooker bee phase

As the onlooker bee is responsible for exploiting the promising regions locally, the proposed fractional-order search strategy is utilized in the onlooker bee phase to reinforce the exploitation ability of algorithm. The principle steps of implementing the FO-based onlooker bee phase is explained as follows.

Initializing memory cells

After having generate the initial food sources, for each food source, its initial position is stored into the first term of corresponding memory cell. Suppose the number of terms r = 4, then in the first three times of updating, onlooker bees will generate new candidate solutions by using only the available memory terms.

In other words, the size of memory cells is 1 in the initialization phase, and more terms will be filled in later.

Updating the memory cells Fig. 5.2. Process of updating a food source's memory cell

In the onlooker bee phase, same roulette wheel selection method is utilized for choosing food sources to exploit. Once a food source is selected by an onlooker bee, it will be updated with Eq.(5.17). And then the new food source position will be stored into its associated memory cell. As we have introduced above, it is possible that the number of terms stored in the associated memory is smaller than r. Then in this case, the memory cell can be updated easily by adding the new position. Otherwise, when the size of memory cell is already r, a method called first in first out (FIFO) is utilized to update the memory (Yousri et al., 2020). The oldest position will be abandoned and the new position will be stored

FRACTIONAL-ORDER ABC ALGORITHM (FOABC)

into the memory cell. To demonstrate this updating mechanism more clearly, the process is shown in Figure 5.2.

The framework of FOABC algorithm

In order to explain the process of FOABC more clearly, the pseudo-code and flowchart are presented in Algorithm 13 and Figure 5.3, respectively.

Experiments on function optimization problems

In this section, several series of experiments are conducted in order to verify the performance of FOABC algorithm. Firstly, experimental studies are carried out to investigate sensitivity of FOABC to parameters r (the number of terms to memory) and q (fractional order) of FOC in subsection 5.

for i = 1 → SN do 7:
Generate l 1 via Eq.(5.6)

8:

Randomly select

j rand ∈ {1, • • • , D} and r 1 = r 2 = r 3 = i from {1, • • • , SN} 9:
Generate new food source position v i via Eq.(5.7)

10: if f (v i ) < f (x i ) then 11: Replace x i with v i , trial i = 0 12: else trial i = trial i + 1 13: end if 14:
end for

15:

Evaluate the probability values with Eq.(1.5) % FO-based onlooker bee phase % 16:

for t = 1 → SN do Produce the v i,j via Eq.(5.17)

21:

Update M i with v i based on FIFO method

22: if f (v i ) < f (x i ) then 23: Replace x i with v i , trial i = 0 24: else trial i = trial i + 1 25: end if 26:
end for 27:

F ES = F ES + 2SN % Scout bee phase % 28: for i = 1 → SN do 29:
if trial i > limit then

Experiments on function optimization problems

The best solution X best is found by the compared algorithm while X * is the exact global optimum.

Moreover, the Wilcoxon rank sum test at 0.05 significant level is employed as well. The symbol "+" indicates that FOABC is better than the compared algorithm. The symbols "-" and "=" denote that the result of FOABC is worse than and similar to the compared one, respectively. The total numbers of each symbol are counted in each comparison table. And Friedman tests are conducted to obtain the rankings of concerned methods.

Sensitive analysis of r and q

In the proposed FOABC algorithm, the control parameters of FOC can influence the final performance. So in this part, experiments are done to study the sensitivity of FOABC to the fractional order q and terms of memory r.

The FOABC variants with different values of q and r are applied on CEC 2017 benchmark functions at dimension D = 30. The population size is fixed to SN = 50. In addition, for each optimization function, each algorithm is run 30 times independently. Then the statistical results of the errors are calculated.

The terms of memory r is tested as 4, 8, and 12. And for each tested value of r, the derivative order q varies from 0.1 to 0.9 with step of 0.1. The experimental results are shown in Tables 5

.843E+02 3.873E+02 3.867E+02 3.872E+02 3.870E+02 3.870E+02 3.871E+02 3.868E+02 3.869E+02 3.873E+02 Std 7.261E-01 1.213E+00 1.833E+00 1.356E+00 1.806E+00 1.472E+00 9.534E-01 1.165E+00 1.216E+00 3.326E-01 f26 Mean 4.237E+02 1.755E+03 1.759E+03 1.704E+03 1.635E+03 1.600E+03 1.569E+03 1.532E+03 1.530E+03 1.558E+03 Std 4.844E+02 1.421E+02 1.139E+02 1.242E+02 1.151E+02 1.390E+02 1.075E+02 1.209E+02 1.056E+02 9.872E+01 f27
Mean 5.120E+02 5.026E+02 5.027E+02 5.032E+02 5.020E+02 5.025E+02 5.015E+02 5.048E+02 5.046E+02 5.037E+02 Std 4.572E+00 7.552E+00 6.264E+00 7.089E+00 5.569E+00 4.706E+00 5.470E+00 8.661E+00 5.965E+00 6.006E+00 f28 Mean 4.027E+02 3.815E+02 3.794E+02 3.750E+02 3.848E+02 3.637E+02 3.677E+02 3.882E+02

Experiments on function optimization problems

when the fractional order q is defined too small (i.e., [0.1, 0.4]), the corresponding performance is not as satisfying as the other variants.

In Table 5.1, when r = 4, the variants with q ∈ [0.5, 0.9] have close in terms of mean and Std values on most of the functions. Meanwhile, their average rankings given by Friedman test are the top five. Moreover, according to the Wilcoxon test, all the variants of FOABC perform better than the basic ABC on over half of the benchmark functions. As for the case r = 8, FOABC variants with q in the same range of [0.5, 0.9] have similar results and outstanding positions in Table 5.2.

And when fractional order q is larger than 0.5, the corresponding algorithms can obtain superior solutions than ABC algorithm on more than 19 test functions, which can be found in the line of Wilcoxon test. The results in Table 5.3 suggests a similar conclusion.Moreover, the versions with q = 0.8 obtain the best rankings in all the three comparison tables. Therefore, for the proposed algorithm, the value of fractional order q cannot be set too small.

Another comparison is made to find out the optimal configuration for FOABC.

In Table 5.4, the FOABC variants who ranked the best in the three aforementioned tables are compared together. The result indicates that F OABC r=12,q=0.8

exceeds the other two variants. Therefore, this setting will be used to compare with other competitive algorithms in the later sections.

Comparison with ABC variants

In this section, FOABC is compared with the basic ABC and five latest ABC variants on 29 CEC 2017 benchmarks with D = 10, 30, and 50. In order to make fair comparisons, the control parameters of the competitors are set the same as their original papers suggested which are presented in Table 5. Table 5.6 presents the results with D = 10, where FOABC performs outstandingly, especially in solving the uni-modal and hybrid problems. Specifically, the number of the minimum errors achieved by FOABC is 10 while the second best is 7 by iff-ABC. In solving the uni-modal functions (i.e., f 1 and f 3 ), the superiority of FOABC is significant compared to the other competitors. More precisely, the errors of FOABC on these two functions are much closer to zero while all the other algorithms have errors bigger than 10 2 . The exploitation ability of algorithms can be tested on the uni-modal problem as there is a unique optimum. In this context, the comparison result verifies the exploitation capability of FOABC. Hence, among the multi-modal functions (f 4f 10 ), NSABC obtains the best results on MGABC becomes the second-best followed by iff-ABC. Therefore, FOABC has outstanding and stable performance throughout the three cases. In this case, the effectiveness of each proposed strategy can be proved. Moreover, it can be concluded that these modifications perform better when they are used cooperatively than when they are used singly.

Comparison with non-ABC algorithms

Convergence behavior analysis

In this part, the convergence rates of involved ABC algorithms are also compared together by plotting their convergence processes of six representative benchmark functions as shown in And its convergence rate is very competitive compared to other ABC variants.

Conclusion

With the purpose of enhancing the performance of ABC Hawa, 2013). The feasible paths can be computed in a graph-based environment, such as a grid map. Nonetheless, in challenging environments, the complexity of the computation may increase significantly. Therefore, both types of approaches cannot guarantee fast and efficient path finding when dealing with complex environments. Thus, the demand for intelligence in solving different RPP challenges is increasing today.

To overcome the above limitations, the third class of methods has gained great attention. Like mentioned before, meta-heuristic algorithms are outstanding in solving various optimization problems and are not problem-specific. These features make it one of the ideal methods to accomplish different tasks. The Therefore, different types of RPP problems are concerned in this chapter.

Firstly, the proposed four ABC variants are applied to complete the global path planning for a single robot. Different environments with arbitrary obstacles are adopted. The proposed ABC algorithms are compared to ten improved ABC and the standard ABC algorithm as well as four well-known RPP approaches.

Secondly, the MRPP task is considered as well. The proposed ABCL algorithm is utilized to solve a local path planning problem for multiple robots. Simulations and comparisons are done to verify the effectiveness of ABCL.

The outline of this chapter is as follows. The single robot path planning (SRPP) problem is achieved in section 6.2. Then, the multi-robot path planning (MRPP) problem is concerned in section 6.3. Both of these sections include the problem formulations, simulations, and comparisons. The conclusion is given in section 6.4.

6.2 Single robot path planning (SRPP)

Problem formulation of SRPP

The RPP environment includes arbitrary obstacles and free space. An example is given in Figure 6.1. The S and T are the start and target points of the robot.

And O 1 , O 2 , O 3 are static obstacles that need to avoid. The goal is to determine the optimal path without any collisions with obstacles from the start point to its desired target. With this purpose, the RPP problem can be converted to an optimization problem and solved with the proposed improved ABC algorithms. Fig. 6.2. Implementation method of robot path planning

x n y n = cos α sin α -sin α cos α × X Y - x S y S , (6.1)
where

α = arcsin y T -y S |ST| , (6.2)
α denotes the rotate angle from original X-axis to line ST. (x n , y n ) is the corresponding coordinate of (X, Y ) in new coordinates system and (x S , y S ) is the coordinate of S in XOY original system.

After that, the line ST is divided into (D + 1) equal parts and the divided points can be obtained via Eq.(6.3).

δ(d) = d × |ST| D + 1 , d = 1, 2, • • • , D. (6.3)
Vertical lines are drawn on each divided point as well as points S and T.

Then a feasible path can be obtained by selecting one node on each vertical line and connecting the nodes together sequentially. Thus, the RPP problem is transformed into a D-dimensional optimization problem. And a candidate solution (i.e., a candidate path) is in form of

p i = (S, p i,1 , p i,2 , • • • , p i,D , T), i ∈ {1, • • • , SN}.
With the purpose of planning collision-free paths via the meta-heuristic optimization algorithms, two strategies are utilized: firstly, for each candidate path p i , the feasibilities of all its nodes p i,1 , p i,2 , • • • , p i,D are checked; secondly, each section of a candidate path is checked step by step to see if it has hit an obstacle.

To explain the strategies clearly, the corresponding pseudo-code is described in Algorithm 14. Each time the candidate solutions are generated or updated, this mechanism is adopted to avoid collisions. If certain nodes encounter obstacles or leave the workspace, they will be directly replaced by a newly produced feasible nodes (lines 1-7 in Algorithm 14). Hence, it is also necessary to verify if the connections of every two adjacent nodes in a candidate path collide with the obstacles. More specifically, for each line segment (i.e., Sp i,1 , p i,

1 p i,2 , • • • , p i,D T),
the inner points are checked one by one with a step increment of 1. And if there are z points encountering the obstacles, a penalty λ = z × 5000 will be imposed on the objective function value as shown in Eq.(6.4). In this way, collision-free paths can be obtained not only by ensuring the feasibility of all the nodes, but also by determining whether the node connections come across the obstacles.

The objective function is defined as

F = length(p i ) + λ, (6.4)
where length(•) is the function to calculate the total length of candidate path 6.2 Single robot path planning (SRPP) p i . λ is a penalty term, which is equal to 0 when all the nodes and segments lie in the free space. Otherwise, λ = z × 5000 when there exist z inner points located at infeasible positions. Hence, the objective function is penalized when the candidate path encounters obstacles.

Algorithm 14 Mechanism for generating collision-free path in SRPP 1: for each candidate path p i do 2:

for j = 1 : D do 3:

if node p i,j is outside the map or encounters an obstacle then 4:

Reproduce a new feasible node p new i,j

5:

Replace the infeasible p i,j with p new i,j 6:

end if 7:
end for

8:

Set λ = 0 9: for all the line segments p i,j p i,j+1 do 10:

Calculate the slope θ j and segment length l j 11:

for m = 0 : 1 : l j do ⊲ Step by increments of 1 12:

Calculate the inner point ip = p i,j + m × [sin(θ j ), cos(θ j )] T

13:

if point ip is infeasible then Evaluate the objective function value with Eq.(6.4)

19: end for Remark 6.1 Note that, like many other heuristic algorithms, the ABC algorithm is initially designed for unconstrained optimization problems, and it can be used to solve constrained optimization problems by adding penalties (Abbas & Ali, 2014). Nonetheless, there also exist constrained optimization algorithms which are established specifically for the constrained optimization problems [START_REF] Szczepanski | Comparison of constraint-handling techniques used in artificial bee colony algorithm for auto-tuning of state feedback speed controller for PMSM[END_REF].

Besides, to better describe the process of applying meta-heuristic algorithms to resolve the SRPP problem, a flowchart with ABC_RL algorithm, as an example, is given in Figure 6.3.

ROBOT PATH PLANNING VIA IMPROVED ABC ALGORITHMS

of each method is considered as well. The best values for each map are marked in boldface.

Remark 6.2 The parameter settings can be adjusted depending on the map scale, the complexity of SRPP tasks, etc. First of all, it can be observed from Table 6.1 and Table 6.2 that the SRPP problems can be effectively solved utilizing the problem formulated previously.

And all the compared algorithms completed the tasks successfully in their 30 individual runs. As a result, the adopted mechanism for searching a collision-free path for a single robot in static environments can be validated.

Among the total 15 ABC algorithms, our proposed algorithms have outstanding performance, especially our proposed FOABC and ABCL algorithms. It is obvious that FOABC achieves the minimum value on the average of the path Meanwhile, the proposed ABCL algorithm has significant advantages in average runtime thanks to its simple structure and the strategy we adopted to simplify the computational complexity.

In fact, these compared algorithms can be generally classified into three tiers based on the comparison results. Firstly, across every concerned situation, the first four algorithms (i.e., FOABC, ABCL, ILTD_ABC, and iff -ABC) perform well and remain stable. Following them, there are several algorithms whose outcomes are rather close, namely ABC_RL, GABC, ARABC, DABC, MGABC, and ABCDC algorithms. Among these algorithms, GABC, DABC, and our proposed ABC_RL outperform the others in terms of overall performance, and the difference from the top tier is also relatively small. Moreover, the standard deviation of MGABC is noticeably higher than that of the others, although their mean values are quite similar. In this context, it can be inferred that MGABC is less stable compared to competitors in the same tier. In addition, considering the average runtime, ABC_RL is more dominant in this second group.

Then, the rest of the algorithms fail to stand out in the comparisons, indicating that they are not effective at resolving this kind of SRPP problem. And the last group of algorithms needs a relatively long time to accomplish the tasks.

Furthermore, to give a more visual picture of the compared algorithms' performance, their rankings across the six tasks are plotted based on average path length and average run time in Figure 6.5 and Figure 6.6, respectively. Note that lower bars in both bar graphs denote better outcomes.

Similar conclusions can be seen in Figure 6.5 and Figure 6.6, which demonstrate that FOABC, ABCL, and ILTD_ABC algorithms surpass the other com- petitors in terms of overall performance. More precisely, FOABC obtains the first place in all tasks in terms of the average path length. And all of these three winners can successfully find paths with short lengths and fast running times within a limited number of iterations. Due to the average time consumption of iff -ABC being higher than these three algorithms, it is not included as a winner.

In addition, more discussions regarding the simulation results can be found below. Actually, the problem scenario concerned above is a uni-modal optimization problem, which means that a unique optimal solution can be found given enough time. Since the primary objective of the defined SRPP is to minimize the path length without collision, the optimal solution in this case refers to a collision-free path with the shortest path length. Therefore, it is evident from the simulations that the three winners are efficient at resolving this type of unimodal optimization problem. Meanwhile, it is possible that some improvement strategies intended for complete optimization issues do not properly assist the algorithms in finding the optimal solution to this practical problem, and even affect the convergence effect. Therefore, certain algorithms that performed well in the previous chapters were unable to maintain their superiorities in this SRPP problem. For instance, APABC, MGABC, NSABC, and the original ABC algorithm have mediocre performance in the simulations above. Nevertheless, note that these algorithms might be better suited to deal with other real-world issues.

Comparison with well-known path planners

In addition, it is also important to compare with other well-known path planning methods. Hence, comparisons are carried out between our proposed ABC algo- In the following simulations, according to the scale of environments, the number of random points to be selected in PRM is defined as 30. The step sizes of RRT and BRRT are set as 10. The parameter settings of the proposed four ABC algorithms are defined the same as above. Moreover, the comparison results of each method are calculated based on 30 independent executions as presented in

Multi-robot path planning (MRPP)

Compared to a single robot, a multi-robot system possesses advantages because of the cooperation and interaction inside the team. In this case, multi-robot collaboration has a stronger ability to resolve complex problems and has higher robustness and reliability [START_REF] Das | Multi-robot path planning using improved particle swarm optimization algorithm through novel evolutionary operators[END_REF] ). As we have mentioned, the classical and graph-based methods cannot remain effective when dealing with complex situations. These approaches become more limited when targeting MRPP tasks. Correspondingly, the type of algorithm we study is more flexible and unconstrained by the problem. Therefore, as long as an optimization problem can be properly formulated, feasible paths can be found using such meta-heuristic methods.

In this part, the objective is to solve on-line MRPP problems more effectively via the intelligent meta-heuristic algorithm. The task is to generate optimal paths without collisions for a group of robots from their starting positions to the intended targets while taking environmental constraints into account. An objective function is properly defined considering the length and safety of the paths.

To solve this established problem, the ABCL algorithm is adopted considering its simple structure and the fact that it consumes the least amount of time in the previous simulations. In fact, ABCL is designed with the ultimate goal of resolving the MRPP problem. Hence, as indicated in Chapter 4, the proposed improvement strategies shouldn't make computations more complex because we need to consider the time spent for computing during on-line planning.

Moreover, a new implementation method is adopted to determine the next positions for all the robots simultaneously. In other words, the candidate solutions of ABCL are formed with the angles to rotate and speeds of all the moving robots.

Then the optimal subsequent coordinates are computed by ABCL considering the

ROBOT PATH PLANNING VIA IMPROVED ABC ALGORITHMS

path length, distances from the obstacles, and distances between teammates. In this way, the optimal collision-free paths for multiple robots can be obtained.

Problem formulation of MRPP

Similar to the previous case, the environment where the robots move and act is composed of obstacles and free space. The start points and target points of robots are located in the free space. An example of MRPP's environment is shown in The objective of MRPP is to find collision-free paths for a group of mobile robots from their start points to their desired goals. To this end, collisions between team members should also be prevented while avoiding the obstacles. Note that, depending on the requirements of different missions, the robots can also reach the same destination. In almost all the path planning problems, there are three aspects that need to be mainly concerned with: efficiency, safety, and accuracy. That is, each robot should save as much energy as possible while finding a safe path in a short amount of time [START_REF] Koubâa | Robot path planning and cooperation: foundations[END_REF].

In the followings, we focus on the MRPP problem in 2-dimensional environments filled with static obstacles. And several assumptions are given before formulating the problem. Firstly, each robot knows its initial and target positions.

Secondly, at each moment, each robot selects its own direction and speed for the

Multi-robot path planning (MRPP)

next movement. This process continues until the robots achieve their targets or a collision occurs. Moreover, to get closer to reality, the robots are considered as squares rather than points.

With the purpose of formulating the MRPP problem, it is necessary to define a proper objective function taking into account the three aspects mentioned above.

In this context, the function should be utilized to minimize each robot's path length and arrival time while avoiding obstacles [START_REF] Das | Multi-robot path planning using improved particle swarm optimization algorithm through novel evolutionary operators[END_REF]. In each step, the proposed algorithm is used to compute the next positions from the current positions of the robots. So, the paths can be found by constantly determining the subsequent feasible positions until they reach their corresponding targets.

Notice that, when part of the robots arrive at their destinations, they will stay there and wait for the others to move toward their goals. Besides, the proposed implementation method will be explained after introducing the robot kinematic model.

Robot kinematic model

The kinematic model of each robot is demonstrated in Figure 6.9. For the robot ri, suppose its position at time t is (x present , y present ) and θ present is the angle between its forward direction and the x-axis. Fig. 6.9. Kinematic illustration of a mobile robot from present to its new position The variables α and v indicate the rotation angle and robot velocity, respectively, which need to be optimized for the next displacement. In this case, the 6. ROBOT PATH PLANNING VIA IMPROVED ABC ALGORITHMS next position (x new , y new ) that robot will be in at time (t + ∆t) can be expressed as follows. Different from the existing implementation methods, a novel way that can determine the next positions of all the robots simultaneously is designed. In other words, at each step, the algorithm is called only once to find where all the robots are moving to. In this way, the running time can be saved.

x new ri = x present ri + cos(α ri + θ present ri ) • v ri • ∆t y new ri = y present ri + sin(α ri + θ present ri ) • v ri • ∆t . ( 6
The form of candidate solutions is made up of the rotation angles and velocities of all the robots that have not reached their targets yet. The k th solution is formulated as The dimension of the candidate solution is set as D = 2 × N where N is the number of robots. When part of the robots achieve their goals, the dimension of candidate solutions will be decreased accordingly, as shown in Figure 6.10(b).

x k = (α k r1 , v k r1 , α k r2 , v k r2 , • • • , α k rN , v k rN ),
In each generation, all candidate solutions (all planning options for robots' next positions) will be evaluated by objective function F which will be described in the following. A candidate planning with a smaller function value contains more qualified subsequent positions.

By using this implementation method, the efficiency of path searching can be ensured, especially when the number of robots is large. If the robot number increases, only the dimension of candidate solutions will augment when using the proposed implementation method. However, in the case that robots call the algorithm to compute the next positions one by one, the number of times the algorithm is called will increase correspondingly, which will significantly increase the running time. Thus, the novel implementation approach can not only save time but also maximize the benefits of meta-heuristic algorithms when dealing with high-dimensional optimization problems. In addition, the outline of the proposed MRPP implementation method is shown in Algorithm 15.

Algorithm 15 Pseudo-code of the proposed MRPP method Get the L_target, L_current of robots haven't arrive their targets 6:

Calculate the optimal α ri and velocities v ri via the proposed ABCL algorithm 7:

Move to the next positions and update L_current end for 14: until all the robots reach the desired targets 6.3 Multi-robot path planning (MRPP)

Objective function of MRPP

The quality of paths is measured by the objective function. In order to find the optimal and safe paths for multiple robots, the objective function is defined in three parts. Function f 1 is used to lead all the robots toward their destinations.

And f 2 is designed to prevent collisions between teammates, while f 3 is adopted to avoid the surrounding obstacles.

First of all, the function f 1 is defined to minimize the distances between the N robots and their desired targets.

f 1 = N i=1 (x new ri -x T ri ) 2 + (y new ri -y T ri ) 2 , (6.7)
where (x new ri , y new ri ) with i = 1, • • • , N represent the next positions of all the robots generated by the algorithm, and (x T ri , y T ri ) are the corresponding target points. Then, the constraints imposed by the environment are tackled via the following objective functions. The second objective function f 2 is formulated in order to avoid collisions between teammates as shown in Eq. (6.8).

f 2 = N i=1 g ri 2 , (6.8)
where

g ri 2 =    0, if Dis(ri, rk) > Q, η × ( 1 Dis(ri, rk) - 1 Q ), otherwise, (6.9)
where η is a positive constant and Dis(ri, rk) is the distance between the i th robot and the k th robot, k = 1, • • • , N. For each robot, the distances to all the other robots are calculated with Eq.(6.9), then the security information is concerned together in Eq.(6.8). Q is the safe distance defined as Q = 2 × L_diagonal, with

L_diagonal is robot's diagonal length. The f 3 is defined to avoid obstacles in the environment, which is formulated in Eq.(6.11). The nearest obstacle is searched step by step in five specific directions, namely left, forward left diagonal, forward, forward right diagonal, and right as expressed in Eq.(6.10). In order to explain clearer, the way of measuring distances of obstacles is demonstrated in Figure 6.11. ), (6.10)

f 3 = N i=1
g ri 3 .

(6.11)

g ri 3 =    0, if minObs ri > Q, ζ × ( 1 minObs ri - 1 Q
), otherwise, (6.12)

where ζ is a positive constant and Q is the safe distance defined above. minObs ri is the minimum distance from robot ri to the obstacles calculated with Eq.(6.10).

Note that the safety of all robots is considered simultaneously.

Hence, the MRPP problem can be represented as an optimization problem with an objective function that minimizes the sum of f 1 , f 2 , and f 3 . The function f 1 is defined to attract the robots toward their targets. f 2 and f 3 are designed for avoiding collisions with obstacles and other robots, respectively. Therefore, the overall objective functions can be defined by the sum of these functions which are equally weighted as shown in Eq.(6.13)

F = f 1 + f 2 + f 3 . (6.13)
Then, each time of planning the next positions, the objective function is minimized by the proposed algorithm. For all the algorithms, the swarm number is SN = 2 × nb_robot and the determination condition is set as max_F ES = 3000×D. And the range of rotate angle α is defined as [-π/2, π/2] while the robot velocity v ∈ [10, 60]. Notice that D is equal to 2 × nb_robot according to the proposed implementation manner.

Comparative study of six-robot path planning

In this part, the simulations of six robot path planning are implemented. The initial configuration of maps is shown in Figure 6.12. When the number of robots is six, in each workspace, there are six squares color-coded differently, which are the initial positions of the robots. The other six circles drawn with dotted lines represent the target areas. The areas are centered on the predefined goal positions and the path planning is achieved once the robots enter the associated areas. Each robot has a specific color which is the color of its corresponding start point and ending area.

For each map, each compared algorithm is executed 15 independent times and the average values are calculated. In Table 6.4, the average number of steps required to reach the target and the average path length traveled by each robot are presented. The minimum values are marked in boldface. Moreover, the average running time is concerned in order to make a more meaningful comparison. The consumed time is presented in Figure 6.14.

In Table 6.4, all the algorithms manage to plan a set of collision-free paths for the six robots. Nonetheless, the effectiveness of ABCL can be proved according to the comparison results. The proposed algorithm outperforms the other competitors in terms of the average total path length in all the maps. Moreover, the steps required to reach the destinations when utilizing ABCL are also competitive. Actually, because the robot's speed is variable, having more steps does not necessarily result in a longer journey. So, in Map 1 and Map 5, certain robots move very short in some places with ABCL. Hence, in these two cases, the path length planned by ABCL is the shortest but more steps are used. And in Map 1, the GABC algorithm needs the least number of steps to get to the target points while DABC surpasses the others in Map 5. Nevertheless, ABCL can still keep a nice balance between the number of steps the robots require and the total length of paths.

Furthermore, another difficulty in MRPP is how to choose the direction when there exist obstacles in a robot's moving direction, especially in a narrow area.

A longer detour may occur if the path planner fails to determine an optimal direction. Examples are given in Figure 6.13 to demonstrate clearly the detours that may take place.

In Figure 6.13(a), the initial configuration of Map 5 is presented. It can be found that in order to reach the desired target points, all the robots will move very close to the obstacles. Particularly, for robots r1 and r3, the path lengths of bypassing the two sides are significantly different when encountering obstacles. A case where all the robots avoid obstacles successfully and do not take detours, as shown in (b). The (c)-(e) in Figure 6.13 demonstrate different detour cases. For robot r1, it is better to pass the obstacle from the left side of the map. But as shown in Figure 6.13(c), it moves through the other side from the start so that it has to travel much longer. It should be pointed out that in this case, the routes of r1, r2, and r5 are close to each other, but there is no collision because the robots take different steps to arrive there. In other words, the robots pass by at separate moments. In Figure 6.13(d), r3 takes a detour when it encounters the circular obstacle in the center of the workspace. Besides, both r2 and r3 travel a bit longer than they do in (b). In fact, if an algorithm is able to distribute the candidate solutions more comprehensively and extract the promising information throughout iterations, it is more capable of choosing the right directions for the robots. In this context, the proposed algorithm is more effective to plan reasonable paths for robots, especially in complicated cases.

In addition, the running time is an essential metric in path planning problems. Hence, the average consumed time of each algorithm over 15 executions is calculated and shown in Figure 6.14.

In Figure 6.14, it can be observed that the ABCL algorithm takes the shortest time in all designed workspaces. As mentioned before, the complexities of Maps 5 and 6 are higher than the other maps. So, in simpler cases, the execution times In Figure 6.15, the robots r1 and r6 reach their target points in three steps since their planning tasks are easier than the others. The robots that have already arrived will stay there and wait for the other teammates. Then, r5 arrives at its goal in the next step. r4 and r2 achieve their desired destinations and successfully avoid obstacles after the next two movements. Finally, the whole MRPP is accomplished in seven steps. As for Map 4, a similar process can be found in The process of planning paths for twelve robots simultaneously is demonstrated step by step in Figure 6.18. For robots r4, r5, r7, r8, r10, and r12, connecting lines from their respective start points to the end points do not cross any static obstacles, which means that the planned paths are preferred if they are closer to straight lines. Correspondingly, it can be found that the proposed MRPP method can plan almost straight paths step by step for the aforementioned robots. Meanwhile, the other robots have more difficult tasks, but they are also capable of avoiding obstacles and maintaining safe distances from them. The change in directions of robots in their midway is because the planning algorithm needs to consider the distance between all the robots and the distance from the obstacles surrounding them while minimizing the total path length. In fact, Figure 6.18 is found more complicated because of the increase in the number of robots. Due to the starting and ending points we set, it is hard to avoid that some robots' paths cross together. Actually, certain intersections of robots' connection lines between corresponding points S and T can be found from the initial configuration in Figure 6.17. Nevertheless, as mentioned before, different robots arrive at the same area at different moments. And it is worth mentioning that no robot collisions were actually detected in the simulations.

Conclusion

Conclusion

In this chapter, problems with more practical implications are concerned, namely the global path planning and local path planning (corresponding to the SRPP and MRPP problems we constructed, respectively). The main objective is to provide more effective solutions for these RPP problems by using our proposed ABC algorithms.

In the first part, after formulating the SRPP problem, two groups of comparisons are carried out. For a more comprehensive comparison, all the ABC variants involved in the previous chapters' comparisons were included in the first group. According to the comparisons, the proposed FOABC and ABCL algorithms have been found outstanding among the 15 algorithms in terms of path length and execution time. ABC_RL and ABCDC are also comparable. Then, the proposed ABC algorithms are also compared to four well-known path planners. The comparison results demonstrate the advantages of our proposed ABCs in average path length while the A* and PRM are also competitive. In addition, when the situation becomes complex, the running time of our proposed algorithms is not affected much. So this difference in running time is acceptable. Secondly, the proposed ABCL algorithm is adopted to solve on-line MRPP problems in static environments considering its superiority in runtime. Meanwhile, a new implementation method is designed. In each step, ABCL is adopted to determine the subsequent positions for all the robots. The effectiveness of ABCL is proved by achieving MRPP tasks for six robots and twelve robots. The simulation results demonstrate the efficiency of ABCL in accomplishing MRPP in terms of path length, safety, and arrival time.

As a result, implementation methods for solving different RPP problems via meta-heuristic algorithms have been proposed. And the effectiveness of our proposed ABC algorithms in practical problems is also demonstrated through simulations and comparisons.

to. The main contributions of this chapter are as follows. The FOC is incorporated into the onlooker bee phase to make full use of the historical experiences.

Meanwhile, a differential search strategy is utilized in the employed bee phase to reinforce the exploration ability. And the scale factors of these search equations are generated via Lévy distribution. Finally, according to the experimental results, FOABC outperforms the other ABC algorithms in terms of solution accuracy and robustness. Moreover, it is very competitive in comparison with the other effective meta-heuristics.

Last but not the least, in Chapter 6, after investigating different improvement strategies to improve the effectiveness of the ABC algorithm, we attempted to apply them to more meaningful applications. Therefore, the proposed algorithms are adopted to find better solutions for different types of RPP problems. Firstly, the single robot path planning problem is established. Different environments with arbitrary obstacles are considered. Secondly, since multi-robot systems are demonstrating their advantages in more and more fields, the MRPP problems are concerned. For all the path planning challenges, the proposed ABC algorithms are compared to other well-known path planners in terms of path length and execution time. As a result, implementation methods for solving different RPP problems via meta-heuristic algorithms have been proposed. And the effectiveness of our proposed algorithms in practical problems is also demonstrated through simulations and comparisons.

Perspectives

The following directions will be explored in the future • There is no doubt that the development of more efficient optimization algorithms will continue to be a popular research direction in the future.

Improving the adaptive capabilities of meta-heuristic algorithms and combining them with other powerful techniques (e.g. machine learning) are two promising approaches. Adaptation methods or machine learning methods can make meta-heuristic algorithms more intelligent and thus be capable to deal with more complex problems. Moreover, as mentioned before, the classical deterministic optimization algorithms perform well in local search. Therefore, in future research, the principles of traditional methods can be integrated with ABC to better balance its global and local search abilities. Nevertheless, it is a challenging task to effectively incorporate these methods into ABC while making sure that neither the original complexity nor its capacity to solve problems will decrease.

• It can be found that the search efficiency and accuracy of the ABC algorithm are relatively high. Therefore, applying the improved ABC algorithms effectively and extensively to more problems in various fields is another essential future task. As for the path planning problem, we can expand to more types of environments in the future, as it is tough to cover all possible situations in this thesis. Moreover, it is very meaningful to apply the proposed methods to real robots.

• Since its introduction, the ABC algorithm's structure and search equations have been modified in different ways, which have significantly increased search accuracy and convergence speed. However, the effectiveness of most meta-heuristics is verified through numerical experiments. And the theoretical study of such algorithms is still in the exploratory stage. In this context, it is one of the important tasks to theoretically prove the convergence, time complexity, and convergence speed of the ABC algorithm.

peuvent être apprises de manière appropriée à partir de l'expérience de mise à jour précédente.

Les principales contributions du chapitre 3 peuvent être résumées comme suit : Dans la dernière partie, après avoir étudié différentes stratégies d'amélioration pour accroître l'efficacité de l'algorithme ABC, nous avons appliqué ces algorithmes ABC améliorés pour résoudre différents types de problèmes de planification de trajectoire. Les algorithmes proposés sont adoptés pour trouver de meilleures solutions en un temps limité après avoir transformé ces tâches pratiques en problèmes d'optimisation. Dans le chapitre 6, premièrement, ces méthodes sont utilisées pour compléter la planification globale de trajectoire pour un seul robot. Différents environnements avec des obstacles arbitraires sont considérés. Ensuite, nous avons considéré le problème de planification de trajectoire multi-robots. La tâche consiste à générer des trajectoires optimales sans collisions pour un groupe de robots depuis leurs positions de départ jusqu'aux points cibles prévus, tout en tenant compte des contraintes de l'environnement dans lequel ils évoluent. Pour tous les défis de planification de chemin, les algorithmes ABC proposés sont comparés à d'autres planificateurs de chemin bien connus en termes de longueur de chemin et de temps d'exécution.

Titre: Algorithmes améliorés de colonies d'abeilles artificielles pour la planification de la trajectoire des robots Résumé: En tant que classe d'algorithmes stochastiques, les algorithmes méta-heuristiques sont efficaces pour résoudre des problèmes d'optimisation grâce à un compromis entre la randomisation et la recherche locale. De tels algorithmes sont avérés efficaces et simples à comprendre. Néanmoins, il y a encore des pistes d'amélioration, comme la facilité d'être piégé dans les optimums locaux ou l'accélération de la vitesse de convergence. Et la gestion précise du compromis entre l'exploration et l'exploitation pour toutes les tâches d'optimisation est toujours un défi. Dans ce contexte, cette thèse traite une classe d'algorithmes méta-heuristiques représentée par l'algorithme ABC (Artificial Bee Colony). Une série de variantes améliorées en analysant les caractéristiques et les faiblesses de l'algorithme ABC a été proposée. De plus, des problèmes ayant une valeur d'application plus élevée sont également pris en compte. Les algorithmes ABC améliorés proposés ont résolu avec succès différents types de problèmes d'optimisation pratiques, y compris des tâches de planification de trajectoire pour un seul robot et multi-robots dans divers environnements.

Mots-clés: Algorithme méta-heuristique, algorithme de colonies d'abeilles artificielles, planification de la trajectoire des robots, apprentissage par renforcement, calcul fractionnaire, optimisation globale. Title: Improved artificial bee colony algorithms for robot path planning Abstract: As a class of stochastic algorithms, meta-heuristic algorithms are effective for solving optimization problems through a trade-off between randomization and local search. Such algorithms have been found to be effective and simple to understand. Nevertheless, there is still room for improvement, such as easy to be trapped in local optimums or slow convergence speed. And precisely handling the trade-off between exploration and exploitation for all optimization tasks is always challenging. In this context, this thesis focused on a class of meta-heuristic algorithms represented by the Artificial Bee Colony (ABC) algorithm and proposed a series of improved variants by analyzing the characteristics and weaknesses of the ABC algorithm. Furthermore, problems with a higher application value are also taken into account. The proposed enhanced ABC versions have successfully solved different types of optimization problems, including robot path planning tasks for single and multiple robots in various environments.

Keywords: Meta-heuristic algorithm, artificial bee colony algorithm, robot path planning, reinforcement learning, fractional calculus, global optimization.

Contents 1. 1

 1 Background and motivation . . . . . . . . . . . . . . . 1.2 Overview of meta-heuristic algorithms . . . . . . . . . 1.3 Overview of Artificial bee colony (ABC) algorithm . 1.3.1 The standard ABC algorithm . . . . . . . . . . . . . . 1.3.2 Related work on ABC algorithm improvement . . . . . 1.3.3 Application prospects . . . . . . . . . . . . . . . . . . 1.4 Overview of robot path planning (RPP) problem . . 1.5 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . 1.5.1 Differential evolution (DE) algorithm . . . . . . . . . . 1.5.2 Particle swarm optimization (PSO) algorithm . . . . . 1.5.3 Cuckoo search (CS) algorithm . . . . . . . . . . . . . . 1.6 Contributions and outline of dissertation . . . . . . .

  the search space. Moreover, they also allow parallel implementation[START_REF] Abdel-Basset | Metaheuristic algorithms: A comprehensive review[END_REF]. Such algorithms has become increasingly popular in recent decades because they are simple to understand, gradient-free, flexible, and effective in targeting various practical problems[START_REF] Chopra | Golden jackal optimization: A novel nature-inspired optimizer for engineering applications[END_REF]. Many of the meta-heuristic algorithms are designed by modeling the intelligent behavior of biological species. More descriptions of meta-heuristic algorithms are presented in the next subsection.

Figure 1 .

 1 Figure 1.1 is drawn to clearly illustrate the hierarchy of the aforementioned optimization algorithms. As previously mentioned, there are numerous classification schemes for countless optimization algorithms, and here we simply provide one of the most popular ones.

Fig. 1. 1 .

 1 Fig. 1.1. Classification of optimization methods

  Scholars have proposed various ways to classify meta-heuristics, the most common of which are the following two: "trajectory-based & population-based" and "nature-inspired & non-nature-inspired" (Abdel-Basset et al., 2018). Gendreau & Potvin (2005) classified meta-heuristics as trajectory-based and population-based.

  et al., 2019c; Das & Jena, 2020; Das et al., 2016b; Gao et al., 2020; Hu et al., 2019; Li & Yin, 2014; Liu, 2016; Zhou et al., 2021b).

Fig. 1. 2 . 2 :

 22 Fig. 1.2. The flowchart of the standard ABC algorithm

  et al., 2014; Karaboga & Gorkemli, 2014; Kong et al., 2018; Song et al., 2017). In addition, researchers began to investigate the way to use multiple search equations with different strengths. Kiran et al. (2015) employed five solution search equations as well as a selection mechanism. Lin et al. (2018) allowed the onlookers to select one between two improved solution search equations by using an adaptive selection mechanism. They also included the best solution in the employed bee phase, which made the search behaviors directional. Chen et al. (2019b) embedded multiple differential search equations into ABC. And a selfadaptive mechanism was proposed to adjust the selection probabilities of those search strategies. Yavuz & Aydın (2019) employed a novel method to decide the expression of solution search equation. A pool of possible terms of the search equations was established. Moreover, a local search method and an increasing population size strategy were incorporated into the algorithm. Furthermore, some scholars were no longer satisfied with a single search procedure, hence they tried to divide the population into subgroups. The population of the proposed ILABC algorithm was divided into several clusters then the search processes were supposed to conduct simultaneously (Gao et al., 2015b). Two new search mechanisms were utilized to facilitate information exchange inside each group as well as between different subpopulations. Harfouchi et al. (2018) 1. INTRODUCTION proposed a cooperative learning ABC via dividing the population into three subgroups. Each of them could evolve independently with multiple search equations. 1.3.2.2 Novel selection mechanisms Gao et al. (2015a) proposed a fitness-based neighborhood mechanism in order to better exploit the hidden information of promising solutions. Cui et al. (2017a) adopted the rankings to select the food sources. In other words, candidate solutions with better rankings have more chances of being selected. Moreover, Cui et al. (2017b) improved the probability model which paid attention to the success rate as well as the objective function value. Wang et al. (2020) believed that the probability selection method of basic ABC was not effective enough, especially when the number of iterations increased. Then the proposed algorithm produced new solutions by selecting the best solution inside neighborhood radius. 1.3.2.3 Hybridization with other algorithms or techniques There are different ways of hybridization that can combine the strengths of different methods with the ABC algorithm. Li & Yin (2014) incorporated the DE algorithm into the structure of ABC algorithm. More precisely, the mutation, crossover, and selection operators were adopted in ABC for generating new food sources. Jadon et al. (2017) proposed another hybrid algorithm based on DE and ABC in order to better balance the exploration and exploitation abilities. The best individual was involved in the employed bee phase. And DE-inspired search strategy was used in the onlooker bee phase. Furthermore, Cui et al. (2020) implemented a hybrid differential ABC algorithm to solve the multi-item replenishment-distribution problem. Kıran & Gündüz (2013) proposed a recombination-based hybridization of PSO and ABC. The global best solutions and the information of neighboring food sources were used to generate novel solutions. A hybrid PS-ABC was proposed by Li et al. (2015) which combined the global search process of ABC with the local search phase in PSO. Gao et al. (2016) proposed a different structure in their hybrid algorithm. More precisely, the GABC (Zhu & Kwong, 2010) was combined with the evolutionary operators of DE to learn from the previous experiences accurately. For each individual, the probability of selecting GABC to update the position was determined based on the performances of both approaches during 1.3 Overview of Artificial bee colony (ABC) algorithm the last generation. Chen et al. (2018) combined the teaching-learning-based optimization (TLBO) algorithm with ABC for the solar PV parameter estimation problems. The teaching phase and learning phase were incorporated into the employed bee phase and onlooker bee phase, respectively. In addition, Sharma et al. (2016) employed the Lévy flights in the solution search equation to improve the local search capability of ABC. Badem et al. (2018) proposed a hybrid algorithm based on ABC and limited-memory Broyden-Fletcher-Goldfarb-Shanno algorithm to solve functions with a large number of local minima. The Boltzmann selection method was introduced to the onlooker bees in the novel ABC variants (Chen et al., 2019a). And the extremal optimization algorithm was incorporated into the search strategies. Furthermore, plenty of techniques have been applied to ABC with the aim of enhancing the optimization performance. It can be found that the chaoticbased method and OBL were widely used in the initialization phase to enhance the population diversity and convergence rate (Gao et al., 2012, 2016; Hu et al., 2015; Kuang et al., 2014). Xiang & An (2013) utilized chaotic initialization based on logistic equation as well as chaotic search in the scout bee phase. Gao et al. (2013) not only modified the solution search equation but also introduced orthogonal experimental design (OED) in an attempt to create a learning strategy for discovering more useful information. Moreover, Rosenbrock's rotational direction method was combined with ABC to improve the exploitation ability (Kang et al., 2011). Ji et al. (2019) incorporated a scale-free network into the ABC algorithm to enhance the search competence. Furthermore, Feng et al. (2022) combined

1. 4

 4 Overview of robot path planning (RPP) problem Robotics has undergone a huge revolution over the past few decades, and nowadays, related technologies are still being innovated. Numerous robotic systems have been developed and produced, and they have demonstrated their abilities to handle various missions in different situations like industrial manufacturing (Meike & Ribickis, 2011; Park et al., 2012; Xing, 2021; Yin et al., 2019), smart home environments (Khan et al., 2021; Wang et al., 2013; Wilson et al., 2019), warehouses (Jiang et al., 2020; Li et al., 2022; Plaksina et al., 2018), etc. It is obvious that intelligence is necessary for effectively accomplishing the tasks.

3 .

 3 Fig. 1.3. Comparison of global and local path planning

Fig. 1. 4 .

 4 Fig. 1.4. Classification of robot path planning approaches

  , oilfield inspection (Li et al., 2020a), etc. As a result of the aforementioned advantages 1. INTRODUCTION and broad applications, MRPP problems have been investigated by researchers for decades. The objective of MRPP is to compute collision-free and qualifying paths for a group of robots from an initial configuration to a target configuration via path planners (Han & Yu, 2020). To name a few, Xu et al. (2011) proposed a hierarchical fuzzy control algorithm for achieving MRPP. Moreover, the robots are autonomous and can communicate with each other. Jose & Pratihar (2016) proposed heuristic methods for the task allocation and collision-free path planning for multi-robot system. The GA algorithm was used for minimizing task completion time and the A* algorithm was adopted for path planning. A novel approach for planning paths for centralized and competitive multi-robot was presented (Hasan & Mosa, 2018). The hybrid method combined the pheromone trail updating of the MAX-MIN ACO algorithm with D* algorithm strategies. By locating and displaying the best path for each robot, the robots use tour construction probabilities to select the best way in the dynamic environments. Thabit & Mohades (2019) applied a multi-objective PSO to the MRPP problem considering the path shortness, safety, and smoothness. In addition, a probabilistic window was introduced in order to search for better paths. Nazarahari et al. (2019) proposed an enhanced GA for resolving multi-objective MRPP problems. A deterministic algorithm, the APF algorithm, was used to produce feasible initial paths. Then the proposed GA was supposed to optimize those paths. Sahu et al. (2022) planned paths for the twin robot via a hybrid algorithm that combined the improved Q-learning and democratic robotics PSO.

•

  Cui, Y., Hu, W., & Rahmani, A. (2022). Improved artificial bee colony algorithm with dynamic population composition for optimization problems. Nonlinear Dynamics, 107(1), 743-760.

(a)

  Fig. 2.1. Comparison between uniform random sampling and SLHD sampling in 2D.

3 :

 3 Evaluate objective function values of the population, F ES = N employed 4: while F ES ≤ max_F ES do 5:

  22 well-known functions which are widely utilized in the comparisons of optimization methods are chosen as benchmark problems (Cui et al., 2017b, 2018; Farah & Belazi, 2018; Wang et al., 2020; Zhu & Kwong, 2010). The function definitions, corresponding global minimum and the search range are presented in Table

Table 2 . 3 :

 23 . The tests are conducted on the 22 benchmark problems with D=30, 50, and 100. The reason of choosing DEABC (Li & Yin, 2014) is that DE's mutation and crossover operators are combined in this DE-inspired hybrid algorithm. APABC (Cui et al., 2017b) is an improved ABC with adaptive population size, however, different from our proposed algorithm, the total size is adjusted according to the success rate of honey bees. The ILTD_ABC (Gao et al., 2019) and NSABC (Wang et al., 2020) are proposed recently which can represent the latest direction of improving ABC. Notice that the ILTD_ABC utilizes novel solution search equations which is powerful in converging to the optimum and the its code is open to public. To start the comparison fairly, their control parameters are set the same as those of their original papers. The utilized parameter settings are listed in Table 2.3. Parameter settings of ABCDC and compared ABC algorithms Algorithm Parameter setting ABC (Karaboga, 2005) SN = 50, limit = SN × D DEABC (Li & Yin, 2014) SN = 50, limit = SN × D APABC (Cui et al., 2017b) SN = 35, SN min = 20, SN max = 35, T = 20 ILTD_ABC (Gao et al., 2019) SN = 50, limit = 100 NSABC (Wang et al., 2020) SN = 50, limit = 100, k = 10, C = 1.5 ABCDC SN = 50, µ F _employed = 0.5, µ F _onlooker = 0.5, a = 0.8, T _f ail = 5

Figure 2 .

 2 Figure 2.3 illustrates the average rankings of involved ABC algorithms based on the mean values for each dimensions case.

Fig. 2 . 5 .

 25 Fig. 2.5. The convergence performance of ABCDC and compared ABC algorithms with D=30

  Nonetheless, there exist few works on tuning the parameters of optimization algorithms via RL. Emary et al. (2017) were incorporated RL and neural networks into gray wolf optimization (GWO) to adjust the value of exploration rate intelligently. In order to enhance the performance of the simulated annealing (SA) 3.1 Introduction algorithm (Samma et al., 2020), two key control parameters were controlled by using Q-learning. A self-learning GA was proposed by Chen et al. (2020) to solve a flexible job-shop scheduling problem. SARSA and Q-learning were utilized to adjust the values of mutation probability and crossover probability. Moreover, Hu et al. (2021) used RL to adjust the scale factor of the solution search equation in DE.

3. 2 . 1

 21 Reinforcement learning (RL) Machine learning (ML) is one of the most essential components of artificial intelligence which has continued to revolutionize technologies since the last century. Countless challenges have been tackled with increasing accuracy and astonishing results in various fields like supply chain dynamics (Arora & Majumdar, 2022), healthcare (Houssein et al., 2021; Maqsood et al., 2022), etc. ML algorithms are often summarized into three categories: firstly, supervised learning trains a classifier with labeled datasets in order to classify or predict data; secondly, unsupervised learning can analyze hidden patterns in unlabeled data (Hussein et al., 2019); last but not the least, RL allows a decision-making agent to take different actions in an environment and learn good policies with an explicit goal of maximizing the cumulative reward (Chen et al., 2020; Sutton & Barto, 2018).

3. 2

 2 Fig. 3.1. The framework of reinforcement learning

3. 3

 3 Proposed ABC_RL algorithm ABC gets caught in local optima, this definition may be useless. Compared with ABC, the CS algorithm adopted a different way to produce its random parameters. Lévy flights are employed in the search equation to enhance the randomness. Lévy flight is a kind of random walk where the step lengths are drawn from a type of heavy-tailed probability distribution, namely the Lévy distribution (Yang, 2020; Yang & Deb, 2009). To tackle the lack of exploitation in ABC, Aydoğdu et al. (2016) replaced the production of new solutions in the scout bee phase by Lévy flights. Besides, Lévy flight was utilized to enhance the effectiveness of GWO, FA, and PSO algorithms (Heidari & Pahlavani, 2017; Jensi & Jiji, 2016; Kalantzis et al., 2016). For the purpose of investigating the optimal randomness in swarm-based algorithms, Wei et al. (2019) incorporated different types of heavy-tailed distributions into CS and compared them with the original CS (i.e., CS with Lévy flights). And these randomness-enhanced CS variants have been found to outperform the basic CS algorithm. In (Yousri et al., 2021), other types of heavy-tailed distributions were adopted instead of the Lévy distribution aiming at improving the performance of the proposed FOCS algorithm. As for the ABC algorithm, only Lévy distribution has been utilized in some ABC variants. To the best of our knowledge, other types of heavy-tailed distributions have not yet been employed in ABC and related studies are inadequate. Moreover, the other kinds of heavytailed distributions have demonstrated their advantages compared with the Lévy distribution in experimental studies (Wei et al., 2019).

  ) where τ β is a Mittag-Leffler random number, γ is the scale parameter, and u, v are two independent uniform random numbers. The parameters are set as γ = 4.5 and β = 0.8, based on the suggestions of Wei et al. (2019).

3. 3

 3 .2.1 Differential search strategy In fact, the way a search equation gathers useful information can greatly affect its search effectiveness. As mentioned earlier, the search equation of standard ABC was found to collect limited information at a time. In Eq.(1.2), the value in only one dimension of one neighbor is considered when producing a new feasible solution. Note that the search abilities of ABC and DE are compared by analyzing their variations of individuals in the equations (Xu et al., 2020). And DE has demonstrated that it is able to search and converge more efficiently than ABC because DE's solution search equation has more possibility to collect useful information from other members. In addition, DE has more flexibility in terms of the nb up each time. Its mutation operator updates individuals on total dimensions at first, then the crossover and selection processes allow the individuals to collect messages from both the previous generation and the latest mutation. Eq.(5.7) presents a basic version of DE search strategy, "rand/1/bin".

(

  [START_REF] Xiang | A grey artificial bee colony algorithm[END_REF]. And Xiang et al. (2021) defined a range to randomly generate the number of components to be modified. Nevertheless, most of these enhancements depend on the parameter settings, and it is nearly impossible to determine a fixed configuration that solves every issue[START_REF] Hu | Reinforcement learning-based differential evolution for parameters extraction of photovoltaic models[END_REF].

3. 3

 3 Proposed ABC_RL algorithm exploitation well (Chen et al., 2020; Shahrabi et al., 2017). The expression of ǫgreedy is as below:

11 :

 11 Replace x i with v i ; trial i = 0

  Fig. 3.2. The flowchart of ABC_RL algorithm

77 3 .

 3 REINFORCEMENT LEARNING BASED ABC ALGORITHM (ABC_RL) Hu et al. (2021) randomly initialized the parameter F that tuned by RL in their algorithm. Nevertheless, besides setting different initial values for different individuals, it is also possible to define a uniform initial value for the swarm. It is worth investigating which initialization method can provide more help to the algorithm. In this context, different methods to define the initial values of d ratio are implemented and compared with standard ABC together. Firstly, variant ABC_RL rand is constructed with the random initialization method. In other words, the d ratio of each employed bee is initialized randomly and independently. Secondly, variants that use predefined d ratio are built as well. In this uniform initialization manner, all the employed bees have the same d ratio value in the beginning stage. And three values (0.2, 0.5, and 0.8) are selected as initial values in the comparison. ABC_RL 0.2 , ABC_RL 0.5 , and ABC_RL 0.8 stand for the cases of defining d ratio with low, middle, and high value.

Table 3 . 3 :

 33 3 in order to compare fairly. Considering that the swarm size may also affect the results, SN values are consistent with the values defined in the corresponding papers. And for all the cases of dimension, SN stays the same. The six algorithms are executed to solve the 29 CEC 2017 benchmark functions with D = 10, 30, and 50. Each method is run 51 independent times on all the problems. And the comparisons are carried out by comparing the mean and standard deviation (Std) of function error values. Parameter settings of ABC_RL and compared ABC algorithms Algorithm Parameter setting ABC (Karaboga, 2005) SN = 50, limit = SN × D sdABC (Chen et al., 2019b) SN = 50, limit = SN × D, pa min = 0.2 ARABC (Cui et al., 2017a) SN = 50, limit = SN × D, ∆ = 0.01, α min = 0, α max = 5 ILTD_ABC (Gao et al., 2019) SN = 40, limit = 100 MGABC (Zhou et al., 2021a) SN = 75, limit = 100, MR = 0.5, q = 0.1, p = 0.1 ABC_RL SN = 50, limit = SN × D, α = 0.6, γ = 0.4, ǫ = 0.3 Tables 3.4 -3.6 present the comparison results in terms of the mean and Std of the errors calculated. For each problem, the Wilcoxon tests are used to display the significance between ABC_RL and other competitors. The symbol "+" means that ABC_RL outperforms the compared algorithm whereas symbol "-" indicates that compared algorithm is better than ABC_RL. And "=" denotes that the two algorithms have similar results. The total numbers of each symbol are listed in the last row of each table. Furthermore, Friedman tests are conducted to evaluate the overall performance of concerned methods. The average rankings are presented in Figure 3.3. Table

Furthermore, the Fig. 3 . 3 .

 the33 Fig. 3.3. Average rankings of ABC algorithms by Friedman tests with D = 10, 30 and 50

7 .

 7 The best results of each function are highlighted in boldface. And the last row shows the Wilcoxon test results of comparing these two variants with basic ABC algorithm. The average rankings given by Friedman tests are drawn in Figure 3.4.

Fig. 3 . 4 .

 34 Fig. 3.4. Friedman test results for effectiveness demonstration of modifications

. 5 .

 5 Fig. 3.5. The convergence performance of ABC_RL and compared ABC algorithms

  (a), (b), and (f). In (a), the final solution obtained by variant ABC_RL QL is better than that of the original ABC. And the final ABC_RL achieves the best solution accuracy. When solving multimodal and hybrid functions, i.e., (b)-(e), in the initial stage, the original ABC converges slightly faster than ABC_RL QL because ABC_RL QL encounters some stagnation points. After that, their convergence processes are nearly the same in the middle stage. It is worth pointing out that ABC_RL QL can further find more accurate solutions in the later stage.

  ABC algorithms have removed this component. For instance, Xiang & An (2013) utilized chaotic search in the scout bee phase in order to further enhance the algorithm's performance. Cui et al. (2017b) removed the scout bee phase from their proposed APABC because new solutions could be introduced via an adaptive method of population size. Moreover, our ABCDC algorithm proposed in Chapter 2 does not contain the scout bee phase either. The proposed method 4.1 Introduction

4. 2 .

 2 1 Teaching-learning based optimization (TLBO) algorithm TLBO is also a swarm-based stochastic optimization algorithm that was proposed in 2011. It was inspired by the process of teaching-learning inside classrooms (Rao et al., 2011). The search process is achieved through two phases iteratively, namely the teaching phase and the learning phase. It has been found that TLBO has advantages like fast convergence rate, simple concept, and outstanding effectiveness (Zou et al., 2019). Learners of TLBO are the individuals of the meta-heuristic algorithms, and each learner represents a candidate solution to the concerned problem. For the purpose of improving the class's knowledge level, learners learn from the teacher in the teaching phase, while they enhance their states by learning from others in the learning phase. The principle process of TLBO is explained as follows.

  element of the global best individual. k 1 = k 2 = i are selected randomly from {1, • • • , SN} and j is chosen from {1, • • • , D}. The parameter ρ i,j is a uniform random number within the range [0, 1] while µ i,j is another random number in [-0.5, 0.5].

ure 4 . 1 .

 41 Note that, the termination conditions is defined by the maximal number of function evaluations max_F ES. In other words, the iteration continues till the number of function evaluations F ES reaches the predefined max_F ES. 4.4 Experiments on function optimization problems In this section, experiments are carried out on a set of optimization problems to test the performance of ABCL. The proposed algorithm is compared with three ABC variants, namely the standard ABC, GABC (Zhu & Kwong, 2010), and DABC (Abbas & Ali, 2014). GABC is involved in the comparison because the global best solution was concerned in its effective search equation as well. The control parameter is set as the same as its best value of the original paper, i.e., C = 1.5.

Fig. 4 . 1 .

 41 Fig. 4.1. The flowchart of ABCL algorithm

Fig. 4 . 2 . 4 . 4 4 (c) f 6 (d) f 9 Fig. 4 . 3 .

 424446943 Fig. 4.2. Average rankings of ABC algorithms by Friedman test with D = 10, 30, and 50

(

  Couceiro et al., 2012; Mousavi & Alfi, 2018; Yousri et al., 2020). In recent years, the FOC has been introduced into certain optimization approaches as it can provide the historical information clearly. Solteiro Pires et al. (2010) incorporated the FOC into the velocity of individuals in PSO algorithm to better control the convergence speed. Then, the FOC was utilized to improve the convergence rate of darwinian particle swarm optimization (DPSO) which was extended from PSO (Couceiro et al., 2012). Furthermore, in fractional calculus-based firefly algorithm (FOFA) (Mousavi & Alfi, 2018), a solution search equation based on the FOC was proposed to give each firefly more historical information. The performance of FOFA has been demonstrated by testing on benchmark functions and application of image segmentation. Deshmukh & Usha Rani (2019) updated the wolf positions in the grey wolf optimizer (GWO) using the memory property of the FOC, which led to boost the convergence rate. More recently, Yousri & Mirjalili (2020) enhanced the random walk of CS algorithm by employing the FOC. And the proposed FOCS has been effectively used to identify the parameters of three different types of chaotic financial systems. Meanwhile, an improved flower pollination algorithm has adopted the FOC as well. The local-search ability of original algorithm has been enhanced by adding the memory features (Yousri 5.2 Proposed FOABC algorithm et al., 2020). Based on the above literature, the incorporation of FOC brought surprising results and was also found to make an improvement in the convergence speed of the algorithm.

  Moreover, the scale factors of these solution search equations are generated via Lévy distribution to increase the randomness of algorithm. To validate the performance of FOABC, it is compared with six effective ABC algorithms and four improved non-ABC algorithms on the Congress on Evolutionary Computation 2017 (CEC 2017) benchmark functions.

  Yang & Deb (2009) has proposed the CS algorithm with Lévy flights whose steps obey the Lévy distribution. Such search manners contain a series of straight flight pathways punctuated by a sudden change in direction. Hence, it enables the algorithm to explore the space efficiently.

  Fig. 5.1. Demonstration of Lévy flights and traditional random walk in 2D

5 . 3 2 : 3 : 4 :

 53234 3.1. Then in subsection 5.3.2, comparisons are made with five state-of-the-art ABC variants and the original ABC on 29 CEC 2017 benchmark problems with dimension D = 10, 30 and 50. Thirdly, the FOABC algorithm is compared with other improved meta-heuristic algorithms in subsection 5.3.3. And subsection 5.3.4 presents the redundancy elimination experiments to verify the effectiveness of each proposed strategy. Same as Chapter 4, the CEC 2017 benchmark problems are utilized since it contains different kinds of single-objective real-number optimization problems: uni-modal functions (f 1f 3 ), simple multi-modal functions (f 4f 10 ), hybrid functions (f 11f 20 ) and composition functions (f 21f 30 ). And the code is provided officially * . Notice that the function f 2 has been deleted in the code, so there is no results for f 2 in all the comparisons in this chapter. In all the following experiments, the evaluation criteria of CEC 2017 is respected which means the maximum functions evaluations max_F ES = 10 4 • D, the search ranges are [-100, 100] D . Each involved algorithm was run 30 times independently on all the problems. Then the mean value and standard deviation (Std) of function error values f (X best )f (X * ) are calculated for comparison. * https://github.com/P-N-Suganthan/CEC2017-BoundContrained 122 Experiments on function optimization problems Algorithm 13 Pseudo-code of FOABC algorithm 1: Set limit = SN × D, CR = 0.8, β = 1.5, σ v = 1 Generate initial population x i , i = 1, • • • , SN with Eq.(1.1) Store the initial positions into corresponding memory cell M i , i = 1, • • • , SN Evaluate objective function values of the population, F ES = SN 5: while F ES ≤ max_F ES do % DE-based employed bee phase % 6:

17 : 18 :

 1718 Select x i by roulette wheel method according prob Generate l 2 via Eq.(5.6)19:Randomly select k = i and randomly select j from {1, • • • , D} 20:

Fig. 5 . 3 .

 53 Fig. 5.3. The flowchart of FOABC algorithm

  .1 -5.3 for r = 4, 8 and 12, respectively. The basic ABC algorithm is involved in order to observe and demonstrate the effectiveness of the FOABC variants. Moreover, the Friedman test and Wilcoxon rank-sum test with significant difference 0.05 are conducted on the results. The algorithm with best ranking given by Friedman test is marked in boldface. And each FOABC version is compared against the standard ABC algorithm via Wilcoxon rank-sum tests. Based on the comparison results in Tables 5.1 -5.3, firstly, most of the FOABC variants have better results than the original ABC algorithm on the considered benchmark functions. Similar conclusion is suggested by the results of Friedman tests in each case of r. The basic ABC ranked the ninth among 10 compared methods when fractional order r = 4 and 12. Besides, the basic ABC stays in the last position when r = 8. In this case, the effectiveness of the proposed improving strategies can be demonstrated. Furthermore, for all the considered values of r, 5. FRACTIONAL-ORDER ABC ALGORITHM (FOABC)

5 .Table 5 . 5 :

 555 In addition, the swarm size is set the same SN = 50 for all the compared ABC variants. The comparison results in terms of the mean and Std of the function errors are shown 5. FRACTIONAL-ORDER ABC ALGORITHM (FOABC) in Tables 5.6-5.8. For each function, the best results are marked in boldface by comparing the mean values. Parameter settings of FOABC and compared ABC algorithms Algorithm Parameter setting ABC (Karaboga, 2005) limit = SN × D NSABC (Wang et al., 2020) limit = 100, k = 10, C = 1.5 iff -ABC (Aslan et al., 2020) limit = SN • D, pr = 80 MGABC (Zhou et al., 2021a) limit = 100, MR = 0.5, q = 0.1, p = 0.1 ILTD_ABC (Gao et al., 2019) limit = 100 sdABC (Chen et al., 2019b) limit = SN • D, pa min = 0.2 FOABC limit = SN • D, CR = 0.8, r = 12, q = 0.8

Fig. 5 . 4 .

 54 Fig. 5.4. Average rankings of ABC algorithms by Friedman test with D = 10, 30, and 50

  results than FOABC does on 15 out of 29. LIPS fails to overcome FOABC on 23 functions. Compared with other FO-based algorithms, the proposed algorithm still has a distinct superiority at D = 50. More precisely, FOABC outperforms FODPSO on 25 functions while it surpasses FOCS on 23 problems. Additionally, FOABC outperforms FOFA in terms of average values across the board.

Fig. 5 . 5 .

 55 Fig. 5.5. Average rankings of non-ABC algorithms and FOABC by Friedman test with D = 10, 30, and 50

Figure 5 . 6 .

 56 Note that, the values of log(f ) is considered since the objective function values are too large to plot in the earlier stage of searching process. In Figure 5.6, (a) and (b) present the convergence curves of ABC algorithms on two uni-modal functions. It can be found that FOABC is able to reach the best solution precision and its convergence rate is competitive to other methods. Similar conclusion can be obtained from the figure (c) of the multi-modal function f 10 . Although iff-ABC and ILTD_ABC converge faster in the early stage, they fail to determine a more precise solution. As for the hybrid and composition functions, the solutions found by most ABC algorithms are close to each other. It is obvious that ILTD_ABC and FOABC converge more rapidly than other competitors. Therefore, in terms of solution accuracy, the FOABC algorithm has outstanding performance, especially in solving uni-modal and hybrid functions.

Fig. 5 . 6 .

 56 Fig. 5.6. The convergence performance of FOABC and compared ABC algorithms 144

6 .

 6 ROBOT PATH PLANNING VIA IMPROVED ABC ALGORITHMS Pareto dominance relationship. Each path point was positioned by optimizing the angle since the robot step length was constant. Multiple feasible paths were generated through the proposed ABC.

6. 2

 2 Fig. 6.1. Example of RPP environment

6 Fig. 6 . 4 .

 664 Fig. 6.4. The initial configurations of six SRPP workspaces

6 .Fig. 6 . 5 .

 665 Fig. 6.5. Rankings of compared ABC algorithms based on average path length over 30 independent executions

  rithms and four well-known global path planners: A* algorithm, Probabilistic Road Map (PRM), Rapidly-exploring Random Trees (RRT), and Bidirectional Rapidly-exploring Random Trees (BRRT), which are widely used to find the shortest path. Note that the code implementation made reference to the resources (Kala, 2014a,b,c,d).

6 Fig. 6 . 7 .

 667 Fig. 6.7. The best paths of FOABC algorithm and four well-known path planners in six workspaces

Figure 6 . 8 ,

 68 Figure 6.8, where O 1 , O 2 , O 3 are the static obstacles. S ri and T ri are the start and target points of the i th robot ri (i = 1, 2, 3). Note that, for clarity, the start positions and goals of different robots are colored differently.

Fig. 6 . 8 .

 68 Fig. 6.8. Environment of multi-robot path planning

. 5 ) 2

 52 Set ∆t = 1 and θ new ri = α ri + θ present ri , then the equation can be rewritten as Implementation method of MRPP There exist various manners of employing this kind of algorithm to achieve robot navigation. One common implementation approach is to generate a group of feasible paths as initial solutions, then optimize the paths with the optimization algorithms (Agarwal & Bharti, 2021; Nazarahari et al., 2019; Oleiwi et al., 2014).Besides, some classical path planning methods are used to generate the initial paths before adopting the heuristic algorithms. In the case of global path planning, a complete path from the robot's start point to its target is determined before the robot starts to move. So the environment should be completely known[START_REF] Sedighi | Autonomous local path planning for a mobile robot using a genetic algorithm[END_REF]. On the other hand, local path planning methods are able to navigate the robots in dynamic or incomplete workspaces. And the path is generated while the robot is moving towards the destination[START_REF] Koubâa | Robot path planning and cooperation: foundations[END_REF].Accordingly, some improved meta-heuristic algorithms have been used for local path planning. For instance,Das et al. (2016a) proposed a hybrid IPSO-IGSA algorithm for MRPP. The algorithm was utilized to determine the next positions from the current positions of every robot. And for each robot, the algorithm was called after each move by taking its current position and velocity as parameters.

  Figure 6.10.

1: 2 : 3 :

 23 Input Map; Number of robots N; Start points S ri and Target points T ri (i = 1, • • • , N); Robot size; Search ranges of rotate angles α and velocities v Initialize parameters and set list of current positions L_current = [S r1 ; S r2 ; • • • ; S rN ] List of targets of moving robots L_target = [T r1 ; T r2 ; • • • ; T rN ] 4: repeat 5:

8 :

 8 for each moving robot do 9:if robot ri arrives its target T ri then 10:Remove position of ri from L_current

Fig. 6 .

 6 Fig. 6.11. Illustration of measuring distances from obstacles in 5 directions

6. 3

 3 Multi-robot path planning (MRPP)6.3.2 Simulation results of MRPPSix maps of size 500 × 500 pixels with different complexity are designed to verify the effectiveness of the ABCL algorithm in MRPP. In the simulations, mobile robots are represented by squares with a side length of 10 pixels and are colored differently. The start points and target points of all the robots are assigned beforehand. In addition, the obstacles are black and in various shapes. The complexity of a workspace is related to the size and number of obstacles.The performance of ABCL is examined by comparing it with three other ABC algorithms in cases of six and twelve robots. More precisely, the standard ABC, GABC (Zhu & Kwong, 2010), and DABC (Abbas & Ali, 2014) are compared with ABCL. Note that DABC was designed for mobile RPP as well. The parameter settings are the same as those in the previous comparisons.

  (a) Map 1 (b) Map (c) Map 3 (d) Map (e) Map 5 (f ) Map

Fig. 6 .

 6 Fig.6.12. The initial configurations of MRPP workspaces for six robots

6. 3 3 Fig. 6 .

 336 Fig. 6.13. The examples of detours in Map 5 planned by ABC algorithm

6 .

 6 Fig. 6.14. Comparison of running time in six robots path planning problems

Figure 6 . 16 .

 616 Figure 6.16. All robots are able to avoid obstacles efficiently. Meanwhile, when there are no teammates or obstacles nearby, the robots can move in almost the shortest straight line toward the destinations, r1 and r3 as examples.

Fig. 6 . 15 .

 615 Fig. 6.15. MRPP process for six robots via ABCL algorithm in Map 2

  premièrement, RL est adopté pour élargir et ajuster intelligemment la fréquence de perturbation de la phase d'abeille employée en tenant compte de la récompense immédiate des résultats de mise à jour de la solution. Deuxièmement, deux équations de recherche de solution améliorées sont utilisées. De plus, un type de distribution à queue lourde, la distribution de Mittag-Leffler, est utilisé pour générer des facteurs d'échelle des équations de recherche. Enfin, l'ABC_RL proposé est comparé à d'autres algorithmes ABC améliorés sur un groupe de fonctions de référence et montre ses performances exceptionnelles.. Dans le chapitre 4, la praticité et la complexité de l'algorithme sont considérées comme plus importantes puisque l'un des objectifs les plus essentiels de l'amélioration de ces algorithmes est de résoudre des problèmes pratiques. Bien que de nombreuses stratégies de modification soient efficaces pour résoudre les problèmes d'optimisation fonctionnelle, elles n'aident pas toujours à obtenir rapidement la solution optimale dans les applications pratiques. Par conséquent, il est également important d'améliorer les performances de l'algorithme sans le compliquer à l'excès. C'est pourquoi le chapitre 4 étudie l'amélioration des performances de l'ABC tout en évitant qu'il ne devienne trop complexe. Dans ce contexte, un algorithme ABC basé sur l'apprentissage (ABCL) est proposé. Ainsi, il est possible d'économiser de l'énergie et du temps lors de la résolution de problèmes tels que la planification de chemins locaux. Les principales contributions de ce chapitre sont les suivantes : premièrement, la meilleure solution globale est adoptée dans les phases d'abeille employée et d'abeille éclaireuse pour guider l'essaim dans une direction de recherche prometteuse. Deuxièmement, la phase d'apprentissage de l'algorithme TLBO est intégrée à la phase d'abeille spectatrice pour améliorer la capacité d'exploitation et simplifier la complexité de calcul. Les résultats expérimentaux et les simulations du dernier chapitre démontrent l'efficacité de l'ABCL tout en garantissant des solutions de haute qualité.Dans les variantes de l'ABC proposées aux chapitres 2 à 4, les équations de recherche de solutions sont améliorées en élargissant le nombre de dimensions à mettre à jour et en augmentant la quantité d'informations pouvant être obtenues à partir de la colonie. Cependant, ce type d'amélioration ignore en fait certaines informations utiles sur l'expérience de recherche historique. Il convient de souligner que, par rapport à la dérivée d'ordre entier, la dérivée d'ordre fractionnaire contient la mémoire entière de ses événements précédents. Par conséquent, contrairement aux résultats existants, le calcul d'ordre fractionnaire (FOC) est incorporé dans l'algorithme ABC en tenant compte des propriétés de mémoire du FOC. Dans l'algorithme FOABC proposé, chaque fois qu'une nouvelle solution candidate est générée, les comportements de fourrage précédents stockés en mémoire sont pris en compte.Les principales contributions de ce chapitre incluent l'incorporation du concept de FOC dans la phase d'abeille spectatrice pour améliorer la capacité de recherche locale. Parallèlement, une stratégie de recherche différentielle est utilisée dans la phase d'abeille employée pour renforcer la capacité d'exploration. Et les facteurs d'échelle de ces équations de recherche sont générés via la distribution de Lévy pour augmenter le caractère aléatoire du FOABC. Des comparaisons sont effectuées sur un ensemble de problèmes pour valider les performances de FOABC.

  The enhanced ABC algorithms have been applied in numerous fields until now, applications have been proposed such as image processing (Banharnsakun et al., 2011; Gao et al., 2019; Öztürk et al., 2020; Su et al., 2022), feature selection (Zhang et al., 2019; Zorarpacı & Özel, 2016), robot path planning (Contreras-Cruz et al., 2015; Liang & Lee, 2015; Lu et al., 2019; Xu et al., 2020), vehicle routing (Lei et al., 2022), parameter identification (Chen et al., 2019a; Hu et al., 2015, 2018), data clustering

  Algorithm 3 Pseudo-code of the standard PSO algorithm 1: Initialize the positions and velocities for the swarm

2: Evaluate the objective function values of the initial population 3: Initialize the pbest i with the particles' current positions 4: Set gbest as the particle position with the best objective function value 5: repeat 6:

  The Lévy distribution 1. INTRODUCTION is a kind of heavy-tailed distribution with a quite high probability of generating large steps. Therefore, Lévy flights enable the algorithm to explore the search space efficiently. Moreover, the CS algorithm generates new solutions via Lévy flights around the best solution obtained so far, which can accelerate the local search. At the same time, to prevent the algorithm from becoming stuck in a local optimum, part of the new solutions should be produced by randomization at locations distant from the current best solution. Correspondingly, the principle steps of the basic CS algorithm are presented in Algorithm 4.

Algorithm 4 Pseudo-code of the CS via Lévy flights 1: Initialize the positions of host nests x i , i = 1, • • • , NP 2: Evaluate the objective function values of x i 3: repeat 4:

  • • • , N employed } and j ∈ {1, • • • , D} is randomly chosen. x best,j is the jth dimension of the current best individual. The parameters F employed_i and F onlooker_i are associated with

	2. IMPROVED ABC ALGORITHM WITH DYNAMIC
	POPULATION COMPOSITION (ABCDC)
	solution search equations, the control parameters F employed_i and F onlooker_i are
	adaptively adjusted based on the successful experiences.
	The parameter adaptation method proposed by Zhang & Sanderson (2009) in
	algorithm JADE is adopted in ABCDC. The parameters F employed_i and F onlooker_i
	are independently created by Cauchy distribution of mean (µ F _employed or µ F _onlooker )
	and standard deviation 0.1. Each time, if the new position v i is better than the
	previous position x i (i.e., f
	each employed bee and each onlooker bee, respectively. Different from other

D} and k1 = k2 = i from the colony 20: Generate v s with Eq.(2.4) 21: if f (v s ) ≤ f (x s ) then 22: Replace x s with v s

	2. IMPROVED ABC ALGORITHM WITH DYNAMIC
	POPULATION COMPOSITION (ABCDC) 2.3 Experiments on function optimization problems
	2.3 Experiments on function optimization prob-
		lems
	In this section, experiments have been done to validate and demonstrate the per-
	formance of ABCDC. Firstly, the sensitive tests and analysis of control parameters
	are presented in subsection 2.3.2. Then two series of experiments are applied on
	22 benchmark functions in different dimensions. The results of ABCDC are com-
	pared with several ABC variants and other effective meta-heuristic algorithms,
	respectively.
	14: 15:	end for Evaluate the probability values prob i with Eq.(1.5) % Enhanced onlooker bee phase %
	16: 17:	while t < N onlooker do Select x s by roulette wheel method according prob
	18: 19: Randomly select j ∈ {1, • • • , 23: F onlooker_s = randCauchy(µ F _onlooker , 0.1) Add F onlooker_s into S F _onlooker
	24:	end if
	25:	end while
	26:	F ES = F ES + 2SN
	27:	if best current ≥ best history then
	28:	nb_f ail = nb_f ail + 1
	29: 30: 31:	end if Update the µ F _employed and µ F _onlooker with Eq.(2.5) Update the number of employed bees and onlooker bees with Algorithm 6. Fig. 2.2. The flowchart of ABCDC algorithm
	32: end while

Table 2

 2

		.1: 22 benchmark optimization functions
	Function	Range	Min

Table 2 .

 2 4: Comparison between ABCDC and other ABC variants with D = 30

	Function		ABC	DEABC	APABC	ILTD_ABC	NSABC	ABCDC
	f 1	Mean 0.00E+00 -1.37E-15 + 6.63E-70 + Std 0.00E+00 2.27E-15 1.79E-69	0.00E+00 0.00E+00	-1.67E-25 + 1.84E-197 1.15E-25 0.00E+00
	f 2	Mean Std	5.75E-09 + 1.70E-12 + 1.35E-64 + 8.70E-09 1.97E-12 6.51E-64	0.00E+00 0.00E+00	-2.70E-22 + 4.22E-145 2.98E-22 2.11E-144
	f 3	Mean 0.00E+00 -9.77E-17 + 4.35E-69 + Std 0.00E+00 1.38E-16 2.10E-68	0.00E+00 0.00E+00	-2.02E-26 + 2.91E-188 1.52E-26 0.00E+00
	f 4	Mean 0.00E+00 = 6.39E-35 + 1.21E-78 + Std 0.00E+00 3.19E-34 3.22E-78	0.00E+00 0.00E+00	= 3.16E-79 + 0.00E+00 9.96E-79 0.00E+00
	f 5	Mean 0.00E+00 -1.14E-05 + 2.61E-35 + Std 0.00E+00 2.99E-05 9.99E-35	0.00E+00 0.00E+00	-3.23E-14 + 1.40E-103 1.08E-14 5.89E-103
	f 6	Mean 1.17E+01 + 2.11E+00 + 6.11E-01 + Std 2.63E+00 2.39E+00 1.16E-01	0.00E+00 0.00E+00	-8.64E+00 + 1.12E-299 1.75E+00 0.00E+00
	f 7

  ABCDC surpass most of the other ABC variants. Unlike the uni-modal functions, multi-modal functions have more than one local optima. So it is worth to point out that multi-modal functions are adequate for evaluating the exploration ability of an algorithm. It can be found that, the

diversification of ABCDC is excellent in this comparison. For functions f 11f 16 and f 21 , ABCDC achieves the corresponding global optimum. For f 17 , f 18 , f 20 and f 22 , the basic ABC obtains the optimal solutions, however, the results of ABCDC are competitive.

In addition, according to the results of the Wilcoxon tests, the advantage of ABCDC is remarkable compared to DEABC. Meanwhile, ABCDC performs better than the original ABC on 10 functions, especially on f 2 , f 6 , f 13f 15 and f 21 . Compared with the NSABC algorithm, ABCDC performs better than it on 17 out of 22 problems. As for APABC, there are 7 functions where their corresponding errors are similar and 15 functions where ABCDC has significant advantages. Moreover, the proposed algorithm performs as good as ILTD_ABC algorithm on 11 functions.

Besides, it is essential to consider the overall performances of 22 benchmark functions together. In this context, it is worth pointing out that ABCDC is outstanding in term of solution precision for various kinds of problems. Compared to basic ABC and ILTD_ABC, ABCDC has very small errors when the other two algorithms get better solutions. Nevertheless, when the solutions of ABCDC are better, their errors are relatively large. So in Figure

2

.3, ABCDC is in the first place under Friedman test. We can conclude that the proposed algorithm achieves excellent results considering solution quality in the case of low-dimension.

Table 2 .

 2 5: Comparison between ABCDC and other ABC variants with D = 50

	Function		ABC	DEABC	APABC	ILTD_ABC	NSABC	ABCDC
	f 1	Mean 0.00E+00 -1.85E-15 + 3.22E-68 + Std 0.00E+00 2.09E-15 5.97E-68	0.00E+00 0.00E+00	-1.23E-23 + 4.10E-241 7.88E-24 0.00E+00
	f 2	Mean Std	2.90E-08 + 2.63E-12 + 2.32E-63 + 1.95E-08 2.86E-12 6.70E-63	0.00E+00 0.00E+00	-2.77E-20 + 4.43E-211 2.12E-20 0.00E+00
	f 3	Mean 0.00E+00 -3.07E-16 + 1.35E-67 + Std 0.00E+00 3.34E-16 4.44E-67	0.00E+00 0.00E+00	-2.84E-24 + 1.95E-246 1.99E-24 0.00E+00
	f 4	Mean 0.00E+00 = 1.20E-21 + 6.45E-74 + Std 0.00E+00 5.97E-21 2.40E-73	0.00E+00 0.00E+00	= 2.79E-77 + 0.00E+00 1.39E-76 0.00E+00
	f 5	Mean 0.00E+00 -7.76E+00 + 6.56E-36 + Std 0.00E+00 2.29E+01 1.21E-35	0.00E+00 0.00E+00	-5.06E-13 + 4.53E-135 2.07E-13 2.04E-134
	f 6	Mean 2.95E+01 + 1.02E+01 + 3.89E+00 + Std 2.83E+00 5.92E+00 3.90E-01	0.00E+00 0.00E+00	= 2.44E+01 + 0.00E+00 2.56E+00 0.00E+00
	f 7

  is competitive with ILTD_ABC and basic ABC. Meanwhile, the advantage of ABCDC is significant on most muti-modal problems.According to the results of the Wilcoxon tests, there are 10 out of 22 functions that ABCDC performs better than the original ABC. And they have similar results on 4 functions. At the same time, ABCDC obtains better solutions in 21 problems compared to DEABC. Meanwhile, ABCDC performs better than the APABC on 15 functions. Compared with the ILTD_ABC algorithm, the numbers that ABCDC performs better than it and worse than it are both 6.

	2. IMPROVED ABC ALGORITHM WITH DYNAMIC
	POPULATION COMPOSITION (ABCDC)
								67E-109
		Std	0.00E+00	5.15E-123	9.65E-125	9.65E-125	0.00E+00	9.65E-125
	f 9	Mean 0.00E+00 = 1.69E-26 + 4.74E-135 + Std 0.00E+00 4.79E-26 1.24E-134	0.00E+00 0.00E+00	= 4.71E-52 + 0.00E+00 7.71E-52 0.00E+00
	f 10	Mean Std	1.12E-01 + 1.25E-01 + 3.52E-02 + 1.99E-02 2.28E-02 6.53E-03	3.60E-06 2.90E-06	-7.52E-02 + 1.31E-05 1.52E-02 9.41E-06
	f 11	Mean Std	1.66E-01 + 4.75E+01 + 4.77E-01 + 1.28E-01 1.88E+01 3.82E-01	4.62E+01 1.36E-01	+ 1.28E+01 + 0.00E+00 2.63E+01 0.00E+00
	f 12	Mean Std	3.24E-04 + 4.38E+02 + 0.00E+00 = 1.62E-03 1.49E+01 0.00E+00	0.00E+00 0.00E+00	= 3.98E-02 + 0.00E+00 1.99E-01 0.00E+00
	f 13	Mean Std	8.00E-02 + 4.17E+02 + 0.00E+00 = 2.77E-01 1.77E+01 0.00E+00	0.00E+00 0.00E+00	= 0.00E+00 = 0.00E+00 0.00E+00 0.00E+00
	f 14	Mean 0.00E+00 = 6.90E-04 + 1.06E-11 + Std 0.00E+00 2.41E-03 5.30E-11	0.00E+00 0.00E+00	= 0.00E+00 = 0.00E+00 0.00E+00 0.00E+00
	f 15	Mean 2.33E+02 + 5.56E+02 + 9.48E+00 + Std 9.35E+01 1.55E+03 3.28E+01	4.26E+02 2.42E+02	+ 3.53E+47 + 1.82E-11 1.45E+48 0.00E+00
	f 16	Mean Std	1.35E-03 + 2.73E-08 + 1.46E-01 + 1.80E-03 2.51E-08 3.56E-01	8.88E-16 0.00E+00	= 9.14E-11 + 8.88E-16 2.26E-10 0.00E+00
	f 17	Mean 0.00E+00 -5.65E+00 + 9.42E-33 = Std 0.00E+00 1.54E+00 1.40E-48	1.15E-32 1.05E-32	+ 1.49E-25 + 9.42E-33 7.53E-26 1.40E-48
	f 18	Mean 0.00E+00 -5.80E+00 + 9.42E-33 = Std 0.00E+00 1.89E+00 1.40E-48	1.17E-32 1.05E-32	+ 1.50E-25 + 9.42E-33 1.39E-25 1.40E-48
	f 19	Mean Std	1.20E-04 + 7.69E-04 + 8.33E-32 + 9.04E-05 2.08E-03 3.68E-31	0.00E+00 0.00E+00	-5.41E-13 + 2.60E-117 3.12E-13 1.29E-116
	f 20	Mean 0.00E+00 -2.03E+01 + 6.61E-30 -Std 0.00E+00 1.18E+00 1.43E-45	6.18E+01 2.50E+01	+ 1.08E-21 -8.06E-11 1.18E-21 4.02E-10
	f 21	Mean Std	1.98E-06 + 8.42E-04 + 5.68E-15 + 1.30E-06 8.32E-04 8.20E-15	0.00E+00 0.00E+00	= 5.97E-14 + 0.00E+00 2.36E-14 0.00E+00
	f 22	Mean 0.00E+00 -7.46E-01 + 2.86E-05 + Std 0.00E+00 5.34E-01 2.58E-14	1.89E-04 4.67E-05	+ 2.86E-05 + 2.86E-05 1.57E-14 0.00E+00
	Total	+/=/-	10/4/8	21/1/0	15/6/1	6/10/6	17/3/2
					51

2.67E-109 + 2.67E-109 = 2.67E-109 = 0.00E+00 -2.

There are 10 out of 22 functions that they have similar results. As for NSABC, there are 17 functions where ABCDC has significant advantages.

In Table

2

.6, the proposed ABCDC, basic ABC and ILTD_ABC are competitive with each other in solving uni-modal functions. For f 1f 3 , the ILTD_ABC algorithm attains the global optima. For f 4 ABCDC and the basic ABC achieve the optimum while ABCDC and ILTD_ABC find the optimum of f 6 . Notice that, on the functions where ABCDC doesn't attain the global optimum, the corresponding errors are much smaller than those of other competitors. At the same time, the advantages of ABCDC, basic ABC and ILTD_ABC algorithms are also remarkable on multi-modal functions. For f 12f 14 , ABCDC and ILTD_ABC attain the global optima. For f 15 and f 16 , all competitors fail to reach the optimum, and the ABCDC algorithm generates the best solutions. The original ABC perform the best on f 17 , f 18 and f 22 whereas ABCDC attain the best results for f 20 and f 21 .

Considering the Wilcoxon test results, ABCDC performs better than the original ABC on 11 functions. DEABC fails to surpass the proposed algorithm on all the benchmarks. ABCDC obtains smaller errors than APABC does on 15 out of 22 functions. Meanwhile, ABCDC is competitive to ILTD_ABC as their results

Table 2 .

 2 6: Comparison between ABCDC and other ABC variants with D = 100

	Function		ABC	DEABC	APABC	ILTD_ABC	NSABC	ABCDC
	f 1	Mean Std	0.00E+00 -2.04E-13 + 2.45E-63 + 0.00E+00 5.67E-13 4.78E-63	0.00E+00 0.00E+00	-1.04E-21 + 5.46E-67 4.43E-22 2.73E-66
	f 2	Mean Std	1.47E-07 + 1.19E-10 + 4.00E-59 + 8.09E-08 1.69E-10 1.22E-58	0.00E+00 0.00E+00	-2.72E-18 + 2.56E-251 1.80E-18 0.00E+00
	f 3	Mean Std	0.00E+00 -1.65E-14 + 3.35E-63 + 0.00E+00 2.19E-14 9.17E-63	0.00E+00 0.00E+00	-4.99E-22 + 9.31E-170 3.70E-22 0.00E+00
	f 4	Mean Std	0.00E+00 = 3.16E-15 + 1.26E-74 + 0.00E+00 1.54E-14 6.27E-74	0.00E+00 0.00E+00	= 3.92E-77 + 0.00E+00 1.93E-76 0.00E+00
	f 5	Mean Std	5.55E-09 + 1.54E+02 + 1.62E-33 + 5.94E-09 6.58E+01 2.01E-33	0.00E+00 0.00E+00	-7.52E-12 + 1.33E-106 1.77E-12 6.67E-106
	f 6	Mean Std	5.28E+01 + 7.53E+01 + 1.84E+01 + 2.85E+00 3.97E+00 1.22E+00	0.00E+00 0.00E+00	= 5.00E+01 + 0.00E+00 2.22E+00 0.00E+00
	f 7	Mean Std	0.00E+00 = 2.00E-01 + 0.00E+00 = 0.00E+00 4.08E-01 0.00E+00	0.00E+00 0.00E+00	= 0.00E+00 = 0.00E+00 0.00E+00 0.00E+00
	f 8	Mean Std	0.00E+00 -7.12E-218 + 7.12E-218 = 0.00E+00 0.00E+00 0.00E+00	7.12E-218 = 0.00E+00 -7.12E-218 0.00E+00 0.00E+00 0.00E+00
	f 9	Mean Std	0.00E+00 = 7.43E-20 + 3.46E-122 + 0.00E+00 1.83E-19 7.46E-122	0.00E+00 0.00E+00	= 3.85E-48 + 0.00E+00 9.00E-48 0.00E+00
	f	Mean	2.70E-01 + 2.71E-01 + 7.57E-02 +	2.84E-06	-2.19E-01 + 6.61E-06
		Std	3.35E-02	3.47E-02	1.01E-02	2.25E-06	2.29E-02	4.17E-06
	f	Mean	2.81E-01 -1.61E+02 + 1.17E+00 -	9.56E+01	+ 1.84E+01 + 3.92E+00
		Std	1.59E-01	4.47E+01	1.28E+00	1.53E-01	2.57E+01	1.96E+01
	f	Mean	4.88E-02 + 9.76E+02 + 0.00E+00 =	0.00E+00	= 2.39E-01 + 0.00E+00
		Std	2.02E-01	2.35E+01	0.00E+00	0.00E+00	4.34E-01	0.00E+00
	f	Mean	8.05E-01 + 9.67E+02 + 0.00E+00 =	0.00E+00	= 0.00E+00 = 0.00E+00
		Std	6.82E-01	3.22E+01	0.00E+00	0.00E+00	0.00E+00	0.00E+00
	f	Mean	0.00E+00 = 7.88E-04 + 2.12E-13 +	0.00E+00	= 0.00E+00 = 0.00E+00
		Std	0.00E+00	2.82E-03	6.60E-13	0.00E+00	0.00E+00	0.00E+00
	f	Mean	7.15E+02 + 1.16E+04 + 4.26E+01 +	1.14E+03	+ 5.18E+51 + 1.09E-10
		Std	1.85E+02	1.03E+04	6.73E+01	3.33E+02	2.59E+52	0.00E+00
	f	Mean	1.54E-03 + 1.99E-01 + 3.51E-01 +	8.88E-16	= 7.19E-10 + 8.88E-16
		Std	1.01E-03	4.08E-01	3.98E-01	0.00E+00	6.31E-10	0.00E+00
	f	Mean	0.00E+00 -2.83E+03 + 4.71E-33 =	1.06E-31	+ 5.61E-24 + 4.71E-33
		Std	0.00E+00	2.94E+03	6.98E-49	4.97E-31	2.24E-24	6.98E-49
	f	Mean	0.00E+00 -2.29E+03 + 4.71E-33 =	1.14E-31	+ 5.82E-24 + 4.71E-33
		Std	0.00E+00	2.75E+03	6.98E-49	4.97E-31	2.66E-24	6.98E-49
	f	Mean	3.85E-03 + 1.24E-07 + 1.52E-21 +	0.00E+00	-3.21E-06 + 2.95E-125
		Std	4.96E-03	1.67E-07	7.60E-21	0.00E+00	1.60E-05	1.35E-124
	f	Mean	5.87E-10 + 3.29E+01 + 8.79E-03 +	1.56E+02	+ 4.39E-03 + 1.04E-15
		Std	2.94E-09	5.18E+00	3.04E-02	4.52E+01	2.20E-02	3.54E-15
	f	Mean	1.07E-05 + 7.54E-01 + 6.59E-14 +	0.00E+00	= 2.25E-05 + 0.00E+00
		Std	3.54E-06	7.80E-01	2.28E-14	0.00E+00	1.13E-04	0.00E+00
	f	Mean	0.00E+00 -2.26E+00 + 2.86E-05 +	4.58E-04	+ 2.86E-05 + 2.86E-05
		Std	0.00E+00	6.78E-01	2.29E-14	1.02E-04	1.72E-14	0.00E+00
	Total	+/=/=	11/4/7	22/0/0	15/6/1	6/10/6	18/3/1

2. IMPROVED ABC ALGORITHM WITH DYNAMIC POPULATION COMPOSITION (ABCDC) Fig. 2.3. Average rankings of ABC algorithms by Friedman test with D = 30, 50, and 100

Table 2

 2 CS also find the global optimal solution of f 7 . As mentioned above, the unimodal problem can detect the exploitation ability, so the exploitation capability of proposed ABCDC algorithm is outstanding among the concerned algorithms.As for the multi-modal functions f 11f 22 , ABCDC outperforms the other non-ABC methods. DE algorithm obtains the competitive results as ABCDC does on f 18 and f 21 . Regarding to the Wilcoxon test, the PSO and FA algorithms fail to surpass ABCDC on all the benchmark functions. Meanwhile, solutions found by DE are comparable to ABCDC on 4 functions. And ABCDC performs better

	Function		PSO	DE	CS	FA	ABCDC
	f 1	Mean Std	1.33E-18 + 7.77E-39 + 6.65E-05 + 1.72E-09 + 1.84E-197 3.75E-18 9.06E-39 3.62E-05 3.46E-10 0.00E+00
	f 2	Mean Std	5.02E-16 + 4.85E-36 + 2.74E-01 + 9.83E-05 + 4.22E-145 1.01E-15 7.98E-36 1.18E-01 2.51E-05 2.11E-144
	f 3	Mean Std	8.93E-20 + 5.16E-40 + 7.84E-06 + 2.22E-10 + 2.91E-188 1.60E-19 5.32E-40 2.81E-06 3.94E-11 0.00E+00
	f 4	Mean Std	6.27E-40 + 6.10E-82 + 4.53E-24 + 2.48E-17 + 0.00E+00 2.47E-39 3.05E-81 1.15E-23 2.65E-17 0.00E+00
	f 5	Mean Std	2.80E-13 + 2.76E-21 + 1.09E-01 + 1.44E-05 + 1.40E-103 4.60E-13 1.98E-21 4.39E-02 1.07E-06 5.89E-103
	f 6	Mean 2.81E+00 + 2.32E-01 + 6.84E-01 + 1.05E+01 + 1.12E-299 Std 1.23E+00 7.18E-01 3.09E-01 4.39E+00 0.00E+00
	f 7	Mean Std	4.00E-02 + 0.00E+00 = 0.00E+00 = 8.00E-02 + 0.00E+00 2.00E-01 0.00E+00 0.00E+00 2.77E-01 0.00E+00
	f 8	Mean Std	7.86E-59 + 7.18E-66 = 7.18E-66 + 7.18E-66 + 7.18E-66 2.13E-58 3.23E-81 3.45E-70 1.23E-72 3.23E-81
	f 9	Mean Std	4.41E-28 + 1.38E-62 + 2.70E-14 + 3.21E-20 + 0.00E+00 1.13E-27 3.13E-62 2.23E-14 4.26E-20 0.00E+00
	f 10	Mean Std	1.99E-02 + 6.24E-03 + 2.32E-02 + 6.50E-03 + 1.99E-05 7.71E-03 1.92E-03 6.74E-03 1.85E-03 1.36E-05
	f 11	Mean 3.50E+01 + 1.91E+01 + 2.31E+01 + 4.63E+01 + 0.00E+00 Std 2.43E+01 1.02E+00 4.09E+00 2.83E+01 0.00E+00
	f 12	Mean 3.50E+01 + 1.41E+02 + 7.10E+01 + 4.99E+01 + 0.00E+00 Std 1.22E+01 1.49E+01 1.14E+01 1.74E+01 0.00E+00
	f 13	Mean 3.72E+01 + 1.17E+02 + 6.24E+01 + 6.21E+01 + 0.00E+00 Std 1.31E+01 1.01E+01 1.27E+01 2.38E+01 0.00E+00
	f 14	Mean Std	1.89E-02 + 7.89E-04 + 4.14E-03 + 6.01E-03 + 0.00E+00 1.85E-02 2.82E-03 3.79E-03 8.16E-03 0.00E+00
	f 15	Mean 1.21E+03 + 5.41E+03 + 3.58E+03 + 3.36E+03 + 0.00E+00 Std 3.48E+02 4.15E+02 2.64E+02 5.86E+02 0.00E+00
	f 16	Mean 1.12E+01 + 2.00E+01 + 1.86E+01 + 1.54E-05 + 8.88E-16 Std 1.01E+01 7.87E-03 3.42E+00 2.11E-06 0.00E+00
	f 17	Mean Std	4.15E-03 + 4.15E-03 + 3.56E-03 + 5.81E-02 + 1.57E-32 2.07E-02 2.07E-02 5.89E-03 1.16E-01 5.59E-48
	f 18	Mean Std	4.15E-03 + 1.57E-32 = 1.66E-03 + 4.56E-02 + 1.57E-32 2.07E-02 5.59E-48 2.11E-03 9.02E-02 5.59E-48
	f 19	Mean Std	2.73E-12 + 1.67E-02 + 5.45E+00 + 7.31E-06 + 2.38E-92 3.60E-12 4.42E-03 1.85E+00 4.88E-06 1.19E-91
	f 20	Mean 1.78E+00 + 5.32E+00 + 6.51E+00 + 2.56E+00 + 4.96E-11 Std 5.16E-01 6.51E-01 8.29E-01 7.63E-01 2.13E-10
	f 21	Mean Std	6.52E-02 + 0.00E+00 = 2.99E+00 + 2.73E-03 + 0.00E+00 3.06E-01 0.00E+00 7.47E-01 1.51E-04 0.00E+00
	f 22	Mean 5.24E+00 + 7.54E-02 + 5.77E+00 + 6.97E+00 + 2.86E-05 Std 1.87E+00 2.61E-01 1.17E+00 3.02E+00 0.00E+00
	Total	+/=/-	22/0/0	18/4/0	21/1/0	22/0/0

.7 -Table

2

.9. In addition, results of Wilcoxon rank sum test are shown. And Figure

2

.4 presents the average rankings given by Friedman tests.

In Table

2

.7, the results of uni-modal functions f 1f 9 show that ABCDC contains significant advantages compared to other methods. Moreover, ABCDC is able to find the global optimum of f 2 , f 4 f 5 , f 7 and f 9 . Meanwhile, DE and

2.3 Experiments on function optimization problems

Table 2.7: Comparison between ABCDC and other meta-heuristic algorithms with D = 30 than DE on 18 functions. As for CS, there is only one problem where it performs

Table 2 .

 2 8: Comparison between ABCDC and other meta-heuristic algorithms with D = 50

	Function		PSO	DE	CS	FA	ABCDC
	f 1	Mean Std	2.12E-14 + 3.07E-34 + 5.90E-05 + 4.55E-09 + 4.10E-241 3.61E-14 2.59E-34 2.70E-05 2.55E-09 0.00E+00
	f 2	Mean Std	1.79E-11 + 2.12E-31 + 3.84E-01 + 6.06E-06 + 4.43E-211 4.55E-11 2.49E-31 1.79E-01 1.00E-05 0.00E+00
	f 3	Mean Std	2.15E-15 + 6.01E-35 + 1.28E-05 + 9.18E-10 + 1.95E-246 2.86E-15 8.96E-35 5.25E-06 7.49E-10 0.00E+00
	f 4	Mean Std	1.37E-30 + 3.50E-40 + 5.44E-24 + 3.36E-23 + 0.00E+00 4.37E-30 1.75E-39 2.67E-23 1.96E-23 0.00E+00
	f 5	Mean Std	1.47E-10 + 1.67E-18 + 2.40E+09 + 1.19E-06 + 4.53E-135 2.88E-10 1.65E-18 4.36E+09 3.91E-07 2.04E-134
	f 6	Mean 1.98E+01 + 6.42E+00 + 1.81E+00 + 3.94E+01 + 0.00E+00 Std 3.00E+00 5.71E+00 7.43E-01 6.94E+00 0.00E+00
	f 7	Mean Std	8.80E-01 + 0.00E+00 = 0.00E+00 = 2.72E+00 + 0.00E+00 9.27E-01 0.00E+00 0.00E+00 3.41E+00 0.00E+00
	f 8	Mean Std	7.35E-94 + 2.67E-109 = 2.68E-109 + 2.67E-109 + 2.67E-109 3.57E-93 9.65E-125 1.23E-111 2.87E-119 9.65E-125
	f 9	Mean Std	8.44E-20 + 4.51E-53 + 1.68E-12 + 2.66E-16 + 0.00E+00 1.84E-19 1.19E-52 2.55E-12 5.50E-16 0.00E+00
	f 10	Mean Std	5.71E-02 + 1.18E-02 + 3.32E-02 + 1.77E-02 + 1.31E-05 1.74E-02 2.46E-03 7.91E-03 5.23E-03 9.41E-06
	f 11	Mean 8.43E+01 + 3.98E+01 + 5.88E+01 + 1.01E+02 + 0.00E+00 Std 3.52E+01 1.05E+01 1.82E+01 3.15E+01 0.00E+00
	f 12	Mean 9.04E+01 + 3.05E+02 + 1.38E+02 + 1.19E+02 + 0.00E+00 Std 2.09E+01 1.94E+01 1.58E+01 3.09E+01 0.00E+00
	f 13	Mean 1.08E+02 + 2.72E+02 + 1.36E+02 + 1.53E+02 + 0.00E+00 Std 2.84E+01 1.86E+01 2.57E+01 2.98E+01 0.00E+00
	f 14	Mean Std	1.11E-02 + 0.00E+00 = 1.32E-03 + 5.61E-03 + 0.00E+00 1.38E-02 0.00E+00 3.72E-03 8.17E-03 0.00E+00
	f 15	Mean 2.28E+03 + 1.11E+04 + 6.90E+03 + 6.35E+03 + 1.82E-11 Std 3.77E+02 6.57E+02 4.67E+02 6.73E+02 0.00E+00
	f 16	Mean 1.83E+01 + 2.00E+01 + 1.94E+01 + 3.20E+00 + 8.88E-16 Std 5.50E+00 1.68E-03 9.15E-01 7.48E+00 0.00E+00
	f 17	Mean Std	2.99E-02 + 3.17E-32 + 6.45E+08 + 8.21E-02 + 9.42E-33 5.42E-02 1.09E-31 2.18E+09 8.93E-02 1.40E-48
	f 18	Mean Std	3.49E-02 + 4.98E-03 + 8.60E+08 + 1.25E-01 + 9.42E-33 5.40E-02 1.72E-02 2.76E+09 1.66E-01 1.40E-48
	f 19	Mean Std	1.17E-09 + 3.15E-02 + 1.30E+01 + 4.56E-07 + 2.60E-117 2.11E-09 7.68E-03 2.84E+00 1.58E-07 1.29E-116
	f 20	Mean 4.75E+00 + 1.19E+01 + 1.67E+01 + 7.85E+00 + 8.06E-11 Std 1.20E+00 1.95E+00 2.10E+00 2.44E+00 4.02E-10
	f 21	Mean 1.73E+00 + 0.00E+00 = 3.08E+00 + 1.18E-02 + 0.00E+00 Std 1.77E+00 0.00E+00 7.53E-01 2.46E-02 0.00E+00
	f 22	Mean 7.49E+00 + 2.04E-01 + 6.78E+00 + 8.89E+00 + 2.86E-05 Std 1.61E+00 3.22E-01 9.49E-01 1.74E+00 0.00E+00
	Total	+/=/-	22/0/0	18/4/0	21/1/0	22/0/0
				57

Fig. 2.4. Average rankings of non-ABC algorithms and ABCDC by Friedman test with D = 30, 50, and 100

  the search efficiency (Akay & Karaboga, 2012; Zhou et al., 2021a). In this case, the evolutionary operators of the DE algorithm have been widely concerned because they are able to update multiple dimensions at a time. Meanwhile,

the mutation operators can make individuals learn more information from their neighbors (Chen et al., 2019b; Cui et al., 2020, 2022; Jadon et al., 2017). For instance, Akay & Karaboga (2012) introduced a modification rate (MR) to control the number of variables that can be inherited from the previous solution, which has a similar role as the crossover rate (CR) in DE. Therefore, in this part of work, a DE-based search strategy is utilized in the employed bee phase. Nevertheless, different from the existing literature, we provide a novel way of varying the frequency of perturbation. The parameters CR and MR are pre-defined control parameters that play an essential role in corresponding algorithms. Yet, it is difficult to define the parameter value appropriately for all kinds of problems. These control parameters are usually constant or updated with predetermined adaptation methods (Chen et al., 2019b; Qin et al., 2009; Wu et al., 2016; Zhang & Sanderson, 2009

  via RL has not been sufficient so far. Considering the excellent performance of integrating RL with other meta-heuristic algorithms, one principle objective of this work is to overcome the shortcomings of ABC by incorporating RL appropri-

ately. Therefore, RL is adopted to vary the number of dimensions to be updated (nb up ) in solution search equation of ABC. The reward value of RL is defined based on the comparison result between the original solution and the newly generated one. In this case, more information can be learned appropriately from the previous updating experience. And nb up can be adjusted at different stages of the search process. Moreover, RL is adopted to set the nb up for each employed bee independently rather than setting the same value for all the population.

As summarized in the first chapter, numerous ABC variations involved the information about the global best solution to help the colony search in a promising direction

(Banharnsakun et

al., 2011; Cui et al., 2022; Gao et al., 2012, 2015a; Jadon et al., 2017; Li et al., 2015; Lin et al., 2018; Zhu & Kwong, 2010). Moreover, surprising results have been obtained when phases employed bees and onlooker bees utilize different search strategies (Cui et al., 2022; Gao et al., 2015a; Jadon et al., 2017; Karaboga & Gorkemli, 2014; Song et al., 2017;

[START_REF] Wang | Improving artificial bee colony algorithm using a new neighborhood selection mechanism[END_REF]

. It is important to note that the success of these kinds of strategies was also confirmed by the previous ABCDC algorithm's solid performance. Hence, in this chapter, the search behavior of onlooker bees in the ABC_RL algorithm is different from that of the employed bees. More precisely, the local search in the onlooker bee phase is boosted by using the global best solution.

table 7 :

 7 Update current state s t ← s t+1

a) 2: Select an initial state s t randomly 3: repeat 4:

Choose an action a t for current state s t from Q table via policy π (e.g., ǫgreedy)

5:

Perform action a t and get reward r t+1

6:

Evaluate Q new (s t , a t ) with Eq.(3.1) and update Q 8: until the termination condition is met

  Choose an action a t for current state s t from Q-table via ǫgreedy Eq.(3.7) Calculate nb up and randomly select nb up dimensions to update Update the j-th dimension of candidate solution via Eq.(3.5)

	5:	Perform action a t on adjusting d i ratio
	6:
	7:	for j ∈ selected dimensions do
	8:
	9:	end for
	10:

1: Initialize Q-tables for all the employed bees and initialize state s t randomly 2: Initialize d i ratio = 0.2, i = 1, • • • , SN 3: for each employed bee do 4:

  table i via Eq.(3.7)

	8:	Take action by adjusting d i ratio and calculate nb up
	9:

Randomly select nb up dimensions from {1, • • • , D} 10: for j ∈ selected dimensions do 11: Generate τ via Eq.(3.3) 12:

Table 3 .

 3 2: Comparison of initialization methods for parameter d ratio

	Function	ABC	ABC_RL rand ABC_RL 0.2 ABC_RL 0.5 ABC_RL 0.8
	f 1 f 3 f 4 f 5 f 6 f 7 f 8 f 9 f f f f f f f f f f f f f f f f f f f f f	2.88E+02 6.66E+02 2.28E-01 7.71E+00 3.35E-09 1.75E+01 8.00E+00 6.04E-03 2.50E+02 4.05E+00 4.29E+04 5.98E+02 1.72E+02 9.76E+01 8.06E+00 2.35E+00 1.53E+03 9.75E+01 2.84E-01 1.10E+02 7.54E+01 3.02E+02 1.04E+02 2.19E+02 1.07E+02 3.95E+02 2.77E+02 2.53E+02 1.87E+04	1.60E+03 2.99E+00 3.26E+00 4.08E+00 0.00E+00 1.35E+01 3.77E+00 0.00E+00 1.20E+02 2.08E+00 1.10E+04 2.76E+03 5.93E+01 6.25E+01 2.70E+00 1.22E+00 2.07E+03 1.96E+01 4.99E-02 1.40E+02 8.27E+01 3.08E+02 2.23E+02 4.06E+02 2.64E+02 3.90E+02 3.35E+02 2.48E+02 3.88E+03	1.10E+03 7.97E+00 3.24E+00 3.55E+00 0.00E+00 0.00E+00 0.00E+00 9.29E+02 3.04E+03 4.24E+00 3.23E-01 3.17E+00 3.62E+00 3.56E+00 3.43E+00 1.42E+01 1.39E+01 1.39E+01 3.40E+00 3.87E+00 3.34E+00 4.55E-15 0.00E+00 0.00E+00 1.67E+02 1.23E+02 9.11E+01 2.04E+00 1.97E+00 1.91E+00 1.14E+04 1.66E+04 2.38E+04 1.34E+03 1.74E+03 4.92E+03 2.79E+01 2.47E+01 2.19E+01 4.13E+01 1.34E+02 3.72E+01 1.20E+01 1.06E+01 2.49E+00 1.84E+00 2.07E+00 1.11E+00 1.61E+03 2.98E+03 1.20E+03 5.98E+01 2.77E+01 6.20E+01 1.25E-02 8.74E-02 1.25E-02 1.53E+02 1.71E+02 1.02E+02 7.75E+01 8.02E+01 9.03E+01 3.07E+02 3.07E+02 3.07E+02 1.58E+02 2.82E+02 2.82E+02 3.82E+02 4.06E+02 4.08E+02 2.18E+02 2.84E+02 3.04E+02 3.90E+02 3.90E+02 3.90E+02 2.87E+02 3.62E+02 3.96E+02 2.44E+02 2.43E+02 2.43E+02 4.68E+03 5.70E+03 7.23E+03
	Wilcoxon +/=/-		18/0/11	19/0/10	18/0/11	16/0/13
	Mean ranking	3.45	2.88	2.36	2.95	3.36

Table 3 .

 3 4: Comparison between ABC_RL and other ABC variants with D = 10In this comparison, the search effectiveness of ABC_RL can be proved by solving different types of problems rather than only being good at solving one kind of problem. It is worth pointing out that, the ABC_RL algorithm's exploitation and exploration abilities have been well balanced during the search process compared to other competitors.Considering the results of Wilcoxon tests, ABC_RL surpasses the original ABC on 21 out of 29 problems, and its advantages on f 3 , f 6 , f 9 , f 12 , f 17 , and f 20 are remarkable. sdABC is slightly better than ABC_RL on 4 problems. And ABC_RL outperforms sdABC on the rest 25 problems. Meanwhile, ABC_RL has more accurate solutions than ARABC on 27 problems while ARABC out-ILTD_ABC obtains smaller errors than ABC_RL on 9 functions, while ABC_RL achieves more accurate solutions on the rest 20 problems.MGABC performs better than ABC_RL on 10 functions and fails to outperform the proposed algorithm on the rest 19 functions.

	Function		ABC	sdABC	ARABC	ILTD_ABC MGABC	ABC_RL
	f 1	Mean 2.88E+02 -1.66E+09 + 1.88E+03 + 1.66E+04 + 2.41E+02 -9.80E+02 Std 2.39E+02 2.89E+09 2.79E+03 4.06E+04 3.46E+02 1.41E+03
	f 3	Mean 6.66E+02 + 6.03E+03 + 1.59E+04 + 1.89E+03 + 1.49E+03 + 5.40E+00 Std 3.85E+02 5.63E+03 5.55E+03 6.74E+02 8.26E+02 1.31E+01
	f 4	Mean 2.28E-01 -1.96E+01 + 3.70E+00 + 4.97E+00 + 6.09E+00 + 2.91E+00 Std 2.38E-01 5.11E+01 2.53E+00 1.41E+00 8.39E-01 1.70E+00
	f 5	Mean 7.71E+00 + 4.25E+01 + 5.02E+00 + 5.22E+00 + 4.76E+00 + 3.42E+00 Std 1.96E+00 1.76E+01 1.61E+00 1.80E+00 1.35E+00 1.10E+00
	f 6	Mean 3.35E-09 + 4.57E+01 + 2.22E-07 + 3.70E-04 + Std 7.72E-09 9.48E+00 1.59E-06 2.01E-04	0.00E+00 = 0.00E+00 0.00E+00 0.00E+00
	f 7	Mean 1.75E+01 + 1.09E+02 + 1.53E+01 + 1.66E+01 + 1.58E+01 + 1.42E+01 Std 2.33E+00 1.72E+01 1.83E+00 2.32E+00 1.39E+00 1.40E+00
	f 8	Mean 8.00E+00 + 4.42E+01 + 5.74E+00 + 6.59E+00 + 4.26E+00 + 3.78E+00 Std 2.51E+00 1.17E+01 1.93E+00 3.35E+00 1.04E+00 1.37E+00
	f 9	Mean 6.04E-03 + 7.12E+01 + 6.15E-06 + 7.72E-05 + Std 1.83E-02 1.36E+02 2.13E-05 1.06E-04	1.17E-05 + 4.46E-15 1.46E-05 2.23E-14
	f	Mean 2.50E+02 + 1.24E+03 + 2.05E+02 + 1.86E+02 + 2.00E+02 + 1.38E+02
		Std	8.62E+01	4.14E+02	1.21E+02	1.26E+02	1.00E+02	9.46E+01
	f	Mean 4.05E+00 + 1.49E+01 + 6.03E+00 + 3.94E+00 + 2.13E+00 + 1.95E+00
		Std	1.54E+00	2.80E+01	6.74E+00	2.38E+00	1.08E+00	1.21E+00
	f	Mean 4.29E+04 + 4.65E+07 + 1.51E+05 + 1.67E+04 + 2.27E+04 + 1.66E+04
		Std	2.41E+04	9.75E+07	2.40E+05	2.02E+04	2.85E+04	1.45E+04
	f	Mean 5.98E+02 -1.12E+03 -2.89E+03 + 7.19E+03 + 6.38E+03 + 1.23E+03 Std 4.90E+02 4.23E+03 2.95E+03 5.57E+03 3.09E+03 2.19E+03
	f	Mean 1.72E+02 + 7.79E+00 -1.80E+03 + 2.99E+03 + 2.04E+03 + 2.82E+01 Std 1.99E+02 1.16E+01 3.05E+03 1.92E+03 1.81E+03 4.50E+01
	f	Mean 9.76E+01 + 1.15E+01 -1.05E+03 + 1.45E+03 + 3.65E+02 + 8.30E+01 Std 2.03E+02 3.10E+01 1.43E+03 1.11E+03 6.00E+02 1.62E+02
	f	Mean 8.06E+00 + 1.83E+02 + 1.24E+01 + 4.33E+01 + 2.59E+00 -3.17E+00 Std 1.89E+01 1.25E+02 2.53E+01 5.93E+01 3.39E+00 9.08E+00
	f	Mean 2.35E+00 + 1.12E+02 + 1.64E+00 + 1.75E+01 + 2.80E+00 + 6.08E-01
		Std	1.29E+00	3.77E+01	1.15E+00	1.93E+01	3.94E+00	4.83E-01
	f	Mean 1.53E+03 + 4.31E+03 + 4.84E+03 + 7.77E+03 + 1.34E+03 + 9.88E+02
		Std	9.88E+02	1.55E+04	4.28E+03	6.61E+03	9.26E+02	1.11E+03
	f	Mean 9.75E+01 + 2.95E+02 + 8.41E+02 + 4.58E+03 + 6.59E+02 + 4.97E+01
		Std	1.18E+02	1.93E+03	1.21E+03	4.29E+03	8.10E+02	1.13E+02
	f	Mean 2.84E-01 + 1.20E+02 + 4.76E+00 + 5.24E+00 + 3.58E-02 + 3.06E-02
		Std	4.01E-01	1.05E+02	1.86E+01	2.33E+01	1.09E-01	9.38E-02
	f	Mean 1.10E+02 -1.04E+02 -1.55E+02 + 1.85E+02 + 1.08E+02 -1.20E+02 Std 3.36E+00 1.02E+01 4.11E+01 4.37E+01 2.55E+01 3.86E+01
	f	Mean 7.54E+01 + 4.05E+02 + 8.08E+01 + 9.43E+01 + 9.24E+01 + 7.39E+01
		Std	2.86E+01	2.63E+02	2.94E+01	2.38E+01	2.46E+01	3.41E+01
	f	Mean 3.02E+02 + 3.14E+02 + 3.10E+02 + 3.10E+02 + 3.03E+02 + 3.01E+02
		Std	5.95E+01	5.45E+00	2.65E+00	2.82E+00	4.29E+01	4.31E+01
	f	Mean 1.04E+02 -1.86E+02 + 2.56E+02 + 3.23E+02 + 1.60E+02 + 1.42E+02 Std 1.81E+01 1.17E+02 7.85E+01 6.57E+01 9.44E+01 8.29E+01
	f	Mean 2.19E+02 -4.71E+02 + 3.91E+02 -4.16E+02 + 4.16E+02 + 4.05E+02 Std 1.10E+02 1.21E+02 6.67E+01 2.23E+01 2.00E+01 1.68E+01
	f	Mean 1.07E+02 -4.80E+02 + 2.34E+02 -2.76E+02 + 2.42E+02 -2.47E+02 Std 9.92E+01 3.70E+02 1.03E+02 5.51E+01 1.08E+02 1.06E+02
	f	Mean 3.95E+02 + 3.93E+02 + 3.95E+02 + 3.91E+02 + 3.92E+02 + 3.90E+02
		Std	2.30E+00	9.45E+00	3.27E+00	2.02E+00	1.87E+00	1.16E+00
	f	Mean 2.77E+02 -5.20E+02 + 4.46E+02 + 4.44E+02 + 2.99E+02 -3.06E+02 Std 8.66E+01 1.36E+02 1.11E+02 1.54E+02 9.22E+01 7.97E+01
	f	Mean 2.53E+02 + 2.73E+02 + 2.58E+02 + 2.57E+02 + 2.61E+02 + 2.44E+02
		Std	2.60E+01	5.37E+01	2.30E+01	9.39E+00	7.25E+00	5.56E+00
	f	Mean 1.87E+04 + 1.47E+06 + 2.09E+04 + 1.74E+04 + 2.04E+04 + 3.85E+03
		Std	2.03E+04	3.33E+06	2.86E+04	1.88E+04	1.85E+04	3.13E+03
	Wilcoxon +/=/-21/0/8	25/0/4	27/0/2	29/0/0	23/1/5

performs ABC_RL on 2 functions. ILTD_ABC fails to surpass ABC_RL on all the benchmarks in the case of D = 10. In fact, only MGABC and ABC_RL achieve the exact optimum on f 6 . MGABC gets smaller errors than ABC_RL on 5 functions while ABC_RL performs better on the remaining 23 functions.

The comparison results with D = 30 are listed in Table

3

.5. Notice that the search difficulty will increase as the problem scale augments, but the population size remains the same. In this context, the advantages of ABC_RL compared to the other effective ABC algorithms haven't been affected greatly. Moreover, its superiorities are obvious on f 6 , f 9 , and f 11 . The Wilcoxon tests show that ABC_RL outperforms ABC on 19 functions and ABC has better results on 10 functions. The number of functions where ABC_RL has more accurate solutions than sdABC is 28 problems and sdABC achieves a smaller error on only 1 function, f 14 . And ABC_RL gains better results than ARABC on 21 problems and fails on 8 problems. ILTD_ABC and MGABC become more competitive in the case of D = 30.

Table 3 .

 3 6 presents the comparison with D = 50. According to the Wilcoxon test results, similar conclusions can be derived from this comparison. The orig-

	3.4 Experiments on function optimization problems

inal ABC surpasses ABC_RL on 9 functions while ABC_RL performs better on the rest 20 functions. Compared with sdABC, ABC_RL obtains smaller errors on 26 functions and larger errors on 3 functions. Although ARABC has outstanding performance in this case, ABC_RL is still competitive to it.

Table 3 .

 3 5: Comparison between ABC_RL and other ABC variants with D = 30

	Function		ABC	sdABC	ARABC	ILTD_ABC MGABC	ABC_RL
	f 1	Mean 2.15E+02 -8.81E+09 + 2.77E+03 + 4.23E+03 + 1.70E+03 -1.74E+03 Std 2.66E+02 7.91E+09 4.95E+03 4.05E+03 1.06E+03 3.06E+03
	f 3	Mean 7.29E+04 + 5.35E+04 + 1.44E+05 + 5.63E+04 + 4.20E+04 -5.25E+04 Std 2.35E+04 2.25E+04 2.74E+04 5.27E+03 6.85E+03 1.33E+04
	f 4	Mean 3.54E+01 -2.59E+03 + 7.62E+01 -7.20E+01 -1.07E+02 + 7.86E+01 Std 2.75E+01 3.80E+03 1.80E+01 1.70E+01 1.19E+01 2.13E+01
	f 5	Mean 8.21E+01 + 1.81E+02 + 4.22E+01 -7.62E+01 + 6.21E+01 + 4.63E+01 Std 1.12E+01 5.13E+01 7.57E+00 1.98E+01 7.81E+00 7.32E+00
	f 6	Mean 7.22E-10 + 2.58E+01 + 1.43E-07 + 2.29E-03 + Std 8.03E-10 1.87E+01 6.82E-07 8.03E-04	1.14E-13 + 0.00E+00 0.00E+00 0.00E+00
	f 7	Mean 9.97E+01 + 3.50E+02 + 6.73E+01 -1.24E+02 + 8.72E+01 + 7.37E+01 Std 1.01E+01 1.11E+02 5.95E+00 4.70E+01 9.36E+00 7.60E+00
	f 8	Mean 9.17E+01 + 1.75E+02 + 4.47E+01 -7.70E+01 + 5.58E+01 + 5.28E+01 Std 1.37E+01 5.46E+01 7.67E+00 1.63E+01 9.13E+00 9.06E+00
	f 9	Mean 7.16E+02 + 6.62E+03 + 4.61E+01 + 3.22E+02 + 3.16E+01 + 4.44E-01 Std 3.95E+02 2.93E+03 3.90E+01 4.46E+02 3.30E+01 6.48E-01
	f	Mean 2.22E+03 + 5.98E+03 + 1.98E+03 -1.97E+03 -2.35E+03 + 2.08E+03 Std 2.66E+02 8.29E+02 2.80E+02 4.36E+02 2.99E+02 3.22E+02
	f	Mean 2.09E+02 + 8.76E+02 + 1.58E+03 + 9.59E+01 + 8.48E+01 + 4.67E+01
		Std	1.11E+02	1.87E+03	1.24E+03	2.01E+01	6.03E+01	2.98E+01
	f	Mean 5.46E+05 + 1.22E+08 + 1.29E+06 + 9.52E+05 + 9.98E+05 + 2.47E+05
		Std	2.99E+05	3.56E+08	9.02E+05	5.25E+05	6.54E+05	2.24E+05
	f	Mean 9.18E+03 -3.14E+05 + 2.64E+04 + 1.72E+04 + 1.43E+04 -1.54E+04 Std 6.39E+03 2.07E+06 2.42E+04 1.37E+04 8.36E+03 1.68E+04
	f	Mean 4.92E+04 + 4.81E+03 -2.60E+05 + 2.29E+05 + 2.50E+05 + 3.03E+04 Std 3.77E+04 3.38E+04 1.89E+05 1.92E+05 2.10E+05 2.74E+04
	f	Mean 1.84E+03 -9.46E+06 + 1.11E+04 + 2.24E+03 -1.10E+03 -9.25E+03 Std 1.86E+03 6.28E+07 1.06E+04 1.62E+03 1.14E+03 1.13E+04
	f	Mean 6.14E+02 + 8.73E+02 + 6.84E+02 + 6.24E+02 + 6.49E+02 + 4.65E+02
		Std	1.54E+02	4.03E+02	1.68E+02	2.40E+02	1.50E+02	1.87E+02
	f	Mean 2.02E+02 + 2.78E+02 + 1.62E+02 + 1.94E+02 + 1.34E+02 + 1.22E+02
		Std	8.70E+01	2.63E+02	9.67E+01	1.37E+02	7.26E+01	9.67E+01
	f	Mean 1.77E+05 -2.91E+06 + 3.90E+05 + 3.09E+05 + 1.23E+05 -1.84E+05 Std 1.00E+05 5.20E+06 2.27E+05 1.92E+05 8.14E+04 1.22E+05
	f	Mean 1.58E+03 -5.02E+07 + 1.29E+04 -3.30E+03 -3.60E+03 -1.29E+04 Std 1.38E+03 2.12E+08 1.10E+04 2.33E+03 2.52E+03 1.66E+04
	f	Mean 2.70E+02 + 5.20E+02 + 2.23E+02 + 2.92E+02 + 1.57E+02 + 1.21E+02
		Std	8.96E+01	2.11E+02	1.02E+02	1.50E+02	4.65E+01	7.26E+01
	f	Mean 2.65E+02 + 3.29E+02 + 2.47E+02 + 2.51E+02 + 2.47E+02 + 2.39E+02
		Std	6.31E+01	3.63E+01	9.95E+00	1.03E+01	7.29E+00	4.28E+01
	f	Mean 6.07E+02 + 1.70E+03 + 6.67E+02 + 1.48E+02 -1.00E+02 -2.39E+02 Std 1.09E+03 1.35E+03 9.91E+02 3.40E+02 2.43E-13 5.65E+02
	f	Mean 4.21E+02 + 5.05E+02 + 4.03E+02 + 4.07E+02 + 3.96E+02 -4.02E+02 Std 2.66E+01 3.38E+01 7.96E+00 1.07E+01 8.70E+00 1.06E+01
	f	Mean 4.80E+02 -7.66E+02 + 5.29E+02 + 5.34E+02 + 5.00E+02 + 4.85E+02 Std 1.88E+02 2.87E+02 2.90E+01 1.94E+01 1.91E+01 7.05E+01
	f	Mean 3.85E+02 -5.97E+02 + 3.86E+02 -3.86E+02 -3.97E+02 + 3.87E+02 Std 1.08E+00 1.13E+02 1.28E+00 3.03E+00 1.46E+01 7.52E-01
	f	Mean 4.31E+02 -2.88E+03 + 1.56E+03 + 1.25E+03 -5.84E+02 -1.36E+03 Std 5.35E+02 2.01E+03 2.24E+02 5.84E+02 6.32E+02 5.47E+02
	f	Mean 5.13E+02 + 5.45E+02 + 5.11E+02 + 5.04E+02 + 5.11E+02 + 5.02E+02
		Std	4.51E+00	1.99E+01	5.43E+00	5.32E+00	5.50E+00	6.76E+00
	f	Mean 4.02E+02 -1.39E+03 + 4.09E+02 -4.00E+02 -4.29E+02 + 4.09E+02 Std 2.80E+00 9.33E+02 9.84E+00 4.73E+00 1.37E+01 6.08E+00
	f	Mean 6.14E+02 + 9.08E+02 + 6.00E+02 + 5.45E+02 + 5.12E+02 + 5.11E+02
		Std	7.83E+01	3.31E+02	8.17E+01	6.93E+01	6.28E+01	6.51E+01
	f	Mean 7.17E+03 05E+03
		Std	1.84E+03	4.36E+05	5.36E+03	2.21E+03	1.84E+03	4.17E+03
	Wilcoxon +/=/-19/0/10	28/0/1	21/0/8	20/0/9	19/0/10

+ 1.11E+05 + 1.14E+04 + 5.27E+03 -5.65E+03 -7.

Table 3 .

 3 6: Comparison between ABC_RL and other ABC variants with D = 50

	Function		ABC	sdABC	ARABC	ILTD_ABC MGABC	ABC_RL
	f 1	Mean 1.76E+03 -3.18E+10 + 5.11E+03 + 6.13E+03 + 4.55E+02 -3.87E+03 Std 1.45E+03 1.75E+10 5.75E+03 9.25E+03 5.47E+02 6.14E+03
	f 3	Mean 2.14E+05 + 1.48E+05 -2.94E+05 + 1.23E+05 -1.16E+05 -1.83E+05 Std 3.78E+04 3.33E+04 3.04E+04 1.20E+04 1.19E+04 2.62E+04
	f 4	Mean 3.87E+01 -4.62E+03 + 7.03E+01 -4.56E+01 -9.41E+01 + 8.00E+01 Std 1.51E+01 3.72E+03 3.27E+01 2.47E+01 3.60E+01 3.98E+01
	f 5	Mean 1.94E+02 + 4.24E+02 + 9.20E+01 -2.27E+02 + 1.52E+02 + 1.13E+02 Std 2.29E+01 9.59E+01 1.05E+01 3.94E+01 2.05E+01 1.57E+01
	f 6	Mean 6.72E-10 + 4.23E+01 + 7.02E-07 + 8.34E-03 + Std 7.81E-10 1.81E+01 4.16E-06 2.51E-02	2.07E-13 + 3.34E-14 4.38E-14 5.23E-14
	f 7	Mean 2.12E+02 + 6.82E+02 + 1.30E+02 -3.79E+02 + 2.02E+02 + 1.53E+02 Std 1.91E+01 1.40E+02 1.07E+01 1.60E+02 1.75E+01 1.41E+01
	f 8	Mean 2.06E+02 + 4.35E+02 + 9.19E+01 -2.28E+02 + 1.47E+02 + 1.16E+02 Std 2.06E+01 8.31E+01 1.37E+01 5.03E+01 1.98E+01 1.61E+01
	f 9	Mean 5.41E+03 + 2.39E+04 + 2.88E+02 + 3.77E+03 + 5.28E+02 + 9.05E+01 Std 1.55E+03 9.37E+03 1.75E+02 2.34E+03 3.53E+02 9.58E+01
	f	Mean 4.21E+03 + 1.07E+04 + 3.77E+03 -4.15E+03 + 4.36E+03 + 3.95E+03 Std 3.30E+02 2.79E+03 2.99E+02 6.14E+02 3.54E+02 4.42E+02
	f	Mean 7.13E+02 + 4.87E+03 + 4.39E+03 + 1.62E+03 + 8.84E+02 + 9.06E+01
		Std	4.87E+02	4.58E+03	2.57E+03	5.44E+02	5.39E+02	2.70E+01
	f	Mean 3.33E+06 + 5.90E+09 + 6.35E+06 + 1.75E+06 -3.14E+06 + 1.88E+06 Std 1.43E+06 1.21E+10 2.55E+06 7.12E+05 1.36E+06 1.17E+06
	f	Mean 5.06E+03 -4.28E+08 + 1.97E+04 + 2.85E+03 -1.41E+03 -9.26E+03 Std 3.35E+03 2.20E+09 1.66E+04 3.59E+03 1.60E+03 1.16E+04
	f	Mean 7.10E+05 + 6.00E+04 -1.51E+06 + 1.13E+06 + 3.06E+06 + 1.18E+05 Std 5.36E+05 2.86E+05 1.15E+06 7.38E+05 1.55E+06 9.11E+04
	f	Mean 9.19E+03 + 8.20E+08 + 1.14E+04 + 1.46E+04 + 1.59E+04 + 8.14E+03
		Std	5.51E+03	1.68E+09	6.14E+03	6.50E+03	3.68E+03	7.87E+03
	f	Mean 1.27E+03 + 1.81E+03 + 1.26E+03 + 8.99E+02 -9.32E+02 -1.14E+03 Std 2.40E+02 4.13E+02 1.93E+02 2.85E+02 1.96E+02 2.78E+02
	f	Mean 9.36E+02 + 1.66E+03 + 8.09E+02 + 9.92E+02 + 8.20E+02 + 7.37E+02
		Std	1.64E+02	1.14E+03	1.86E+02	2.71E+02	2.18E+02	1.81E+02
	f	Mean 9.38E+05 + 2.48E+05 -2.88E+06 + 1.45E+06 + 2.21E+06 + 6.41E+05 Std 4.90E+05 2.36E+05 1.47E+06 5.63E+05 9.69E+05 5.11E+05
	f	Mean 7.47E+03 -1.46E+05 + 1.62E+04 + 1.82E+04 + 1.61E+04 + 1.46E+04 Std 2.99E+03 4.90E+05 8.10E+03 1.13E+04 4.28E+03 1.31E+04
	f	Mean 7.12E+02 + 1.35E+03 + 6.89E+02 + 5.63E+02 + 4.44E+02 -5.23E+02 Std 1.81E+02 4.14E+02 1.49E+02 2.14E+02 1.63E+02 1.71E+02
	f	Mean 4.12E+02 + 5.23E+02 + 3.05E+02 -3.52E+02 + 3.23E+02 -3.28E+02 Std 2.23E+01 6.04E+01 1.32E+01 2.09E+01 1.47E+01 1.39E+01
	f	Mean 4.65E+03 -8.55E+03 + 4.09E+03 -4.88E+03 + 2.65E+03 -4.75E+03 Std 1.54E+03 4.26E+03 1.06E+03 1.77E+03 2.45E+03 7.99E+02
	f	Mean 6.48E+02 + 8.53E+02 + 5.51E+02 -5.76E+02 + 5.46E+02 -5.68E+02 Std 4.82E+01 7.45E+01 1.49E+01 2.43E+01 1.70E+01 1.97E+01
	f	Mean 1.01E+03 + 1.12E+03 + 7.97E+02 + 8.21E+02 + 7.48E+02 + 6.85E+02
		Std	6.34E+01	3.64E+02	3.97E+01	4.57E+01	3.35E+01	3.11E+01
	f	Mean 5.08E+02 -4.22E+03 + 5.12E+02 -5.42E+02 + 5.97E+02 + 5.16E+02 Std 1.99E+01 2.66E+03 1.78E+01 3.18E+01 1.02E+01 2.24E+01
	f	Mean 1.92E+03 -6.32E+03 + 2.38E+03 -1.65E+03 -6.20E+02 -2.59E+03 Std 1.54E+03 2.32E+03 1.65E+02 1.54E+03 1.13E+03 1.76E+02
	f	Mean 6.49E+02 + 8.92E+02 + 6.41E+02 + 5.92E+02 + 6.20E+02 + 5.80E+02
		Std	3.03E+01	1.34E+02	3.42E+01	1.69E+01	1.96E+01	3.75E+01
	f	Mean 4.85E+02 -2.54E+03 + 4.84E+02 -4.95E+02 + 5.44E+02 + 4.92E+02 Std 1.19E+01 1.64E+03 1.68E+01 1.99E+01 1.56E+01 1.90E+01
	f	Mean 1.03E+03 + 1.96E+03 + 8.59E+02 + 6.60E+02 -8.18E+02 + 7.62E+02 Std 1.54E+02 3.19E+03 1.64E+02 1.62E+02 1.78E+02 1.34E+02
	f	Mean 7.23E+05 -1.13E+08 + 7.97E+05 -8.52E+05 -8.76E+05 -9.74E+05 Std 4.99E+04 4.22E+08 9.45E+04 1.05E+05 7.15E+04 2.70E+05
	Wilcoxon +/=/-20/0/9	26/0/3	17/0/12	21/0/8	19/0/10

Table 3 .

 3

	7: Effectiveness of each modification of ABC_RL on benchmarks with
	D= 10, 30 and 50
			D=10			D=30		D=50
	Function	ABC	ABC_RLeq ABC_RLQL	ABC	ABC_RLeq ABC_RLQL	ABC	ABC_RLeq ABC_RLQL
	f1 f3 f4 f5 f6 f7 f8 f9 f10 f11 f12 f13 f14 f15 f16 f17 f18 f19 f20 f21 f22 f23 f24 f25 f26 f27 f28 f29 f30 Wilcoxon (+/=/-)	2.88E+02 6.87E+02 6.66E+02 1.03E+02 2.28E-01 2.26E+00 7.71E+00 4.51E+00 3.35E-09 0.00E+00 1.75E+01 1.44E+01 8.00E+00 5.10E+00 6.04E-03 5.21E-08 2.50E+02 1.37E+02 4.05E+00 2.36E+00 4.29E+04 2.88E+04 5.98E+02 5.14E+02 1.72E+02 2.89E+02 9.76E+01 1.40E+02 8.06E+00 1.40E+00 2.35E+00 8.20E-01 1.53E+03 1.86E+03 9.75E+01 1.80E+02 2.84E-01 1.84E-02 1.10E+02 1.16E+02 7.54E+01 5.72E+01 3.02E+02 3.08E+02 1.04E+02 1.09E+02 2.19E+02 3.64E+02 1.07E+02 1.55E+02 3.95E+02 3.93E+02 2.77E+02 2.98E+02 2.53E+02 2.50E+02 1.87E+04 9.41E+03	3.21E+02 2.16E+02 4.91E-01 7.34E+00 5.32E-11 1.74E+01 8.06E+00 9.10E-08 2.83E+02 3.26E+00 4.45E+04 4.13E+02 8.39E+01 4.12E+01 2.91E+00 2.91E+00 1.60E+03 3.77E+01 2.12E-01 1.09E+02 6.26E+01 2.97E+02 1.19E+02 2.91E+02 9.72E+01 4.31E+02 8.52E+02 2.15E+02 5.10E+02 7.29E+04 6.04E+04 3.54E+01 5.66E+01 8.21E+01 4.58E+01 7.22E-10 0.00E+00 9.97E+01 7.22E+01 9.17E+01 5.20E+01 7.16E+02 5.16E+01 2.22E+03 1.90E+03 2.09E+02 8.16E+01 5.46E+05 4.90E+05 9.18E+03 1.63E+04 4.92E+04 5.74E+04 1.84E+03 7.51E+03 6.14E+02 5.64E+02 2.02E+02 1.53E+02 1.77E+05 1.89E+05 1.58E+03 6.20E+03 2.70E+02 2.00E+02 2.65E+02 2.29E+02 6.07E+02 2.31E+02 4.21E+02 4.03E+02 4.80E+02 4.91E+02 3.85E+02 3.87E+02 3.91E+02 5.13E+02 5.10E+02 2.91E+02 4.02E+02 4.09E+02 2.64E+02 6.14E+02 5.49E+02 1.22E+04 7.17E+03 7.86E+03	5.03E+02 6.01E+04 5.03E+01 7.75E+01 7.74E-07 1.01E+02 8.27E+01 5.34E+02 2.42E+03 5.10E+01 5.37E+05 6.01E+03 2.84E+04 7.08E+02 5.99E+02 1.39E+02 1.93E+05 1.21E+03 7.47E+03 1.08E+04 1.76E+03 3.65E+03 2.14E+05 2.04E+05 3.87E+01 4.04E+01 1.94E+02 1.11E+02 6.72E-10 2.01E-14 2.12E+02 1.47E+02 2.06E+02 1.11E+02 5.41E+03 5.19E+02 4.21E+03 3.79E+03 7.13E+02 4.49E+02 3.33E+06 2.78E+06 5.06E+03 6.50E+03 7.10E+05 4.03E+05 9.19E+03 8.45E+03 1.27E+03 1.15E+03 9.36E+02 7.31E+02 9.38E+05 9.79E+05 1.63E+02 7.12E+02 5.93E+02 2.40E+02 4.12E+02 3.16E+02 4.38E+02 4.65E+03 4.16E+03 4.20E+02 6.48E+02 5.68E+02 4.67E+02 1.01E+03 8.04E+02 3.86E+02 5.08E+02 5.17E+02 4.77E+02 1.92E+03 2.51E+03 5.11E+02 6.49E+02 6.32E+02 4.09E+02 4.85E+02 4.96E+02 6.03E+02 1.03E+03 8.18E+02 5.08E+03 7.23E+05 7.32E+05	1.45E+03 1.86E+05 6.05E+01 1.89E+02 1.30E-05 2.10E+02 1.81E+02 4.42E+03 4.39E+03 1.33E+02 3.82E+06 1.36E+03 3.60E+05 5.57E+03 1.24E+03 8.49E+02 8.97E+05 7.85E+03 6.28E+02 3.92E+02 4.86E+03 6.27E+02 7.40E+02 5.16E+02 2.30E+03 6.40E+02 4.96E+02 8.67E+02 7.31E+05
	v.s. ABC		17/0/12	18/0/11		17/0/12	20/0/9	20/0/9	19/0/10
	Firstly, in

Table 3 .

 3 7, the comparison between variant ABC_RL eq and the basic ABC is concerned. It is observed that ABC_RL eq achieves significant improvement in functions f 6 and f 9 . According to the Wilcoxon test results, for D = 10 and 30, ABC_RL eq obtains smaller errors than ABC does on 17 benchmarks. As for D = 50, it outperforms the basic ABC algorithm on 20 functions. In this case, the effectiveness of utilizing improved search equations with heavy-tailed based scale factors can be verified.

Secondly, in order to demonstrate the effect of RL strategy, ABC_RL QL is compared with ABC algorithm. It can be found that, for all the dimension cases, ABC_RL QL achieves better results on more than half of the 29 functions. So the

3.4 Experiments on function optimization problems

algorithm's performance can be improved by using RL method to enlarge and adjust the nb up . Moreover, similar conclusion can be derived from the number of best results in boldface.

  Proposed ABCL algorithm has been widely embedded into ABC variants (Gao et al., 2014; Wang et al., 2020;

		Xiao et al., 2021; Zhu & Kwong, 2010). In this way, the colony can be guided in
		a hopeful direction while the algorithm's exploitation ability can be improved.
		In addition, limited useful information is learned from the swarm since only
		one individual is considered in Eq.(1.2). To this point, various modifications
	5: 6: 7: 8: 9:	Produce new learner x new i have been investigated. One of the most effective modifications is inspired by with Eq.(4.2) if f (x new i ) < f (x i ) then the evolutionary operators of the Differential Evolution (DE) algorithm. The Replace x i with x new i mutation and crossover strategies can not only allow the ABC to learn more else Learner x i remains the same information from the population but also enlarge the dimensions being updated
	10:	end if
	11:	end for % Learning phase %
	16:	Replace x i with x new i
	17: 18:	else Learner x i remains the same
	19:	end if
	20:	end for
	21:
	4.3.1 Enhanced employed bee phase
	In original ABC algorithm, the new candidate solutions are generated via Eq.(1.2)

12:

for i = 1 → NP do 13: Select a learner k = i randomly 14:

Update x i with Eq.(4.3)

15: if f (x new i ) < f (x i ) then

Update the T eacher and the Mean

22: until the termination condition is reached. this chapter introduces an enhanced ABC algorithm with three improvements, named ABCL. Firstly, a modified solution search equation based on the global best individual is adopted in employed bee phase of the proposed ABCL algorithm. Meanwhile, considering the strong exploitation ability of TLBO, its learning strategy is incorporated into the onlooker bee phase. And the global best information is involved in scout bee phase as well. with a randomly selected neighbor x k and a random number θ i,j . Hence, this search strategy is good at exploration thanks to the randomness (Zhu & Kwong, 2010). Based on the literature, being inspired by PSO, the global best information 4.3 each time (Banharnsakun et al., 2011; Chen et al., 2019b; Gao et al., 2012, 2016; Liang et al., 2017).

Table 4

 4

	.1: Benchmark functions
	Function	Range	Min

Table 4 .

 4

		ABC		DABC		GABC		ABCL
	Function	mean	± std	mean	± std	mean	± std	mean	± std
	f 1 f 2	3.79E-25 8.29E-25 1.37E-39 5.41E-39 5.36E-50 1.46E-49 4.48E-50 6.03E-50 4.14E-45 2.06E-44 3.43E-37 1.72E-36 4.85E-83 2.39E-82 1.27E-82 4.26E-82
	f 3

2: Comparison between ABCL and other ABC variants with D = 10

Table 4 .

 4 3: Comparison between ABCL and other ABC variants with D = 30

		ABC		GABC		DABC		ABCL
	Function	mean	± std	mean	± std	mean	± std	mean	± std
	f 1 f 2 f 3 f 4 f 5 f 6 f 7 f 8 f 9 f 10 f 11 f 12	4.68E-23 6.32E-23 1.35E-44 1.48E-44 1.91E-37 5.05E-37 9.99E-45 1.80E-44 1.14E-46 4.59E-46 9.79E-82 4.89E-81 3.31E-25 1.64E-24 9.60E-82 2.61E-81 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 7.18E-66 3.23E-81 7.18E-66 3.23E-81 7.18E-66 3.23E-81 7.18E-66 3.23E-81 5.29E-02 6.36E-02 3.81E+00 7.43E+00 8.16E-02 1.38E-01 1.85E-02 2.21E-02 0.00E+00 0.00E+00 0.00E+00 0.00E+00 7.11E-17 3.55E-16 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 7.82E-15 3.80E-14 0.00E+00 0.00E+00 1.19E-09 5.64E-09 1.42E+01 3.93E+01 1.62E-11 7.09E-11 1.09E-12 9.09E-13 2.43E-31 3.55E-31 1.57E-32 5.59E-48 1.57E-32 5.59E-48 1.57E-32 5.59E-48 7.63E-31 2.05E-30 1.57E-32 5.59E-48 1.57E-32 5.59E-48 1.57E-32 5.59E-48 3.31E-25 8.20E-25 3.91E-30 2.15E-45 5.74E-30 9.14E-30 3.91E-30 2.15E-45 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00
	+/=/-	7/5/0		4/8/0		7/5/0		-

According to Table

4

.3, the ABCL is found effective and competitive on both uni-modal and multi-modal problems with D = 30. The advantages are significant, especially on f 1 , f 5 , and f 8 . Based on the Wilcoxon tests, there is no case that ABCL performs worse than any other competitor on the considered benchmarks. The proposed algorithm obtains better solutions than ABC does on 7 functions. And they have similar results on the rest 5 problems. At the same time, the differences between results of GABC and ABCL are not very large since they obtain similar errors on 8 functions. And ABCL has better performance on the other 4 functions, which can help to prove the effectiveness of embedding the learning strategy in the ABC algorithm. Compared with DABC, ABCL is able to achieve smaller errors on 7 benchmarks.

In Table

4

.4, when D = 50, ABCL is able to maintain its effectiveness in this case. Based on the Wilcoxon tests, ABCL outperforms the standard ABC Furthermore, based on the average rankings given by the Friedman test in Figure 4.2, the proposed ABCL algorithm has the best rankings in all three cases. Therefore, it can be concluded that ABCL has the best overall performance among 4.4 Experiments on function optimization problems

Table 4 .

 4 4: Comparison between ABCL and other ABC variants with D = 50

		ABC		DABC		GABC		ABCL
	Function	mean	± std	mean	± std	mean	± std	mean	± std
	f 1 f 2 f 3 f 4 f 5 f 6 f 7 f 8 f 9 f 10 f 11 f 12	2.70E-22 2.94E-22 1.51E-36 5.15E-36 5.18E-43 4.08E-43 3.71E-43 3.97E-43 1.96E-46 9.61E-46 6.08E-21 2.71E-20 1.25E-83 4.33E-83 8.93E-82 2.52E-81 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 2.67E-109 9.65E-125 2.67E-109 9.65E-125 2.67E-109 9.65E-125 2.67E-109 9.65E-125 5.50E-02 6.10E-02 4.61E-02 4.59E-02 7.54E+00 1.95E+01 5.42E-02 7.11E-02 0.00E+00 0.00E+00 5.68E-16 2.22E-15 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 1.49E-14 5.42E-14 0.00E+00 0.00E+00 0.00E+00 0.00E+00 4.74E+00 2.37E+01 7.81E-10 3.79E-09 2.41E+01 6.00E+01 1.53E-04 7.64E-04 3.65E-31 4.65E-31 9.42E-33 1.40E-48 9.42E-33 1.40E-48 9.42E-33 1.40E-48 7.86E-31 1.62E-30 9.42E-33 1.40E-48 9.42E-33 1.40E-48 9.42E-33 1.40E-48 4.90E-24 6.64E-24 1.39E-28 6.63E-28 6.61E-30 1.43E-45 6.61E-30 1.43E-45 5.12E-15 8.08E-15 8.53E-15 1.00E-14 1.25E-14 1.18E-14 1.08E-14 1.18E-14
	+/=/-	7/4/1		5/4/3		4/7/1		-

Table 5 .

 5 1: Comparison of FOABC variants with number of terms r = 4 and q taked values from 0.1 to 0.9 with ABC

	r = 4

Table 5 .

 5 4: Friedman test results of three competitive FOABC variants

	Number of terms r	Derivative order q	Average ranking
	4	0.8	2.21
	8	0.8	1.93
	12	0.8	1.86

Table 5 .

 5 4 of them whereas MGABC attains two. As for the hybrid functions (f 11f 20 ), the advantage of FOABC is notable by comparing not only the values of errors but also the number of functions in boldface. FOABC achieves the best on 7 among 10 hybrid functions. Meanwhile, FOABC doesn't keep the same superiority in solving the rest benchmarks which can be explained according to the No Free Lunch Theorem. The iff-ABC algorithm performs the best on composition functions (f 21f 30 ). Nevertheless, it is worth mentioning that all the algorithms involved in this type of problem obtain similar results. The advantage of FOABC is also remarkable based on the results of the Wilcoxon tests. Compared to the original ABC algorithm, FOABC attains better solutions on 17 out of 29 functions. And it outperforms NSABC on 15 problems. 6: Comparison between FOABC and other ABC variants with D = 10

	Functions		ABC	NSABC	iff-ABC	MGABC	ILTD_ABC	sdABC	FOABCr=12,q=0.8
	f1 f3 f4 f5 f6 f7 f8 f9 f10 f11 f12 f13 f14 f15 f16 f17 f18 f19 f20 f21 f22 f23 f24 f25 f26 f27 f28 f29 f30	Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean 6.18E+01 -3.07E+02 + 2.52E+03 + 2.64E+02 + 4.20E+02 + 2.46E+02 2.55E+03 3.12E+02 5.72E+02 6.26E+02 + 3.17E+03 + 4.19E+03 + 1.32E+03 + 2.97E+02 2.49E+03 3.07E+03 6.60E+02 2.42E-01 -5.50E+00 + 3.05E+00 + 1.58E-01 -1.89E-01 1.38E+00 2.05E+00 2.26E-01 7.30E+00 + 4.67E+00 -7.37E+00 + 3.35E+00 -1.79E+00 2.08E+00 1.18E+00 1.13E+00 3.17E-09 -3.41E-14 -5.95E-09 -0.00E+00 -5.40E-09 5.30E-14 7.33E-09 0.00E+00 1.74E+01 + 1.57E+01 + 1.78E+01 + 1.34E+01 -2.32E+00 3.47E+00 1.07E+00 1.71E+00 9.14E+00 + 4.88E+00 -8.09E+00 + 3.71E+00 -2.53E+00 2.12E+00 1.17E+00 1.35E+00 5.26E-03 -1.12E-05 -5.73E-02 + 3.46E-06 -1.71E-02 3.27E-05 1.07E-01 7.46E-06 2.11E+02 + 1.92E+02 + 2.72E+02 + 1.67E+02 -8.96E+01 9.08E+01 8.27E+01 1.28E+02 3.86E+00 + 3.57E+00 + 4.97E+00 + 2.31E+00 + 1.93E+00 1.53E+00 1.91E+00 1.26E+00 3.89E+04 + 2.49E+04 + 2.33E+04 + 3.55E+04 + 2.32E+04 1.61E+04 1.45E+04 6.57E+04 6.83E+02 + 5.71E+03 + 4.07E+02 + 6.11E+03 + 6.10E+02 6.83E+03 3.65E+02 3.71E+03 1.81E+02 + 1.33E+03 + 1.57E+02 + 1.39E+03 + 1.75E+02 2.04E+03 2.28E+02 1.66E+03 1.69E+02 + 8.48E+02 + 1.61E+02 + 4.43E+02 + 2.10E+02 1.58E+03 3.19E+02 7.51E+02 8.88E+00 + 1.11E+01 + 1.31E+01 + 2.24E+00 -1.22E+01 2.52E+01 3.00E+01 3.24E+00 3.74E+00 + 2.89E+00 + 3.28E+00 + 2.03E+00 -1.74E+00 5.76E+00 5.15E+00 8.07E-01 1.26E+03 + 4.61E+03 + 1.16E+03 + 1.15E+03 + 9.39E+02 4.10E+03 9.51E+02 1.21E+03 8.69E+01 + 3.28E+03 + 1.09E+02 + 7.59E+02 + 1.16E+02 3.18E+03 1.87E+02 9.07E+02 3.14E-01 -2.28E+00 + 9.21E-01 + 4.34E-02 -4.11E-01 5.14E+00 5.75E-01 9.95E-02 1.13E+02 -1.15E+02 1.13E+02 --1.01E+02 -1.99E+01 3.62E+01 2.62E+01 2.72E+01 8.16E+01 -6.36E+01 -9.49E+01 -Std 3.84E+01 3.34E+01 2.04E+01 3.38E+01 3.08E+02 -3.04E+02 -3.08E+02 -Mean 2.87E+02 -Std 2.93E+00 5.75E+01 1.65E+00 8.25E+01 Mean 1.01E+02 -2.37E+02 1.62E+02 --9.00E+01 -Std 2.50E+01 1.14E+02 1.01E+02 2.53E+01 Mean 1.86E+02 -4.12E+02 4.22E+02 + -1.53E+02 -Std 1.05E+02 2.14E+01 2.20E+01 6.78E+01 Mean 1.20E+02 -2.76E+02 2.43E+02 --1.17E+02 -Std 1.00E+02 9.81E+01 1.04E+02 9.94E+01 Mean 3.84E+02 3.94E+02 + 3.92E+02 + -3.72E+02 -Std 5.12E+01 2.03E+00 1.93E+00 7.45E-01 Mean 2.57E+02 -3.28E+02 -4.33E+02 + 2.52E+02 -Std 1.08E+02 9.20E+01 6.79E+01 1.05E+02 Mean 2.49E+02 + 2.47E+02 + 2.47E+02 + 2.59E+02 + Std 3.05E+01 8.33E+00 2.72E+01 1.17E+01 Mean 1.24E+04 + 3.05E+04 + 1.76E+04 + 3.70E+02 -Std 1.56E+04 1.68E+02 2.12E+04 2.68E+04	9.65E+03 2.40E+04 2.16E+03 9.26E+02 5.32E+00 1.22E+00 5.00E+00 1.96E+00 2.91E-04 1.53E-04 1.57E+01 2.14E+00 5.05E+00 2.81E+00 9.36E-05 1.47E-04 1.67E+02 1.38E+02 3.10E+00 1.18E+00 9.75E+03 5.75E+03 7.36E+03 5.30E+03 2.41E+03 1.38E+03 1.38E+03 9.98E+02 2.74E+01 5.19E+01 1.46E+01 1.42E+01 6.64E+03 6.19E+03 4.71E+03 3.91E+03 7.20E-01 2.20E+00 1.76E+02 4.85E+01 1.00E+02 3.34E-01 3.09E+02 2.47E+00 3.02E+02 9.19E+01 4.18E+02 2.27E+01 2.73E+02 7.85E+01 3.91E+02 1.48E+00 4.11E+02 1.68E+02 2.56E+02 1.57E+01 1.68E+04 2.76E+04	+ 1.87E+09 + 3.30E+09 + 4.77E+03 + 3.36E+03 + 1.40E+01 + 2.59E+01 + 4.53E+01 + 1.76E+01 + 4.55E+01 + 1.05E+01 + 1.06E+02 + 1.94E+01 -4.27E+01 + 1.22E+01 -4.09E+01 + 9.31E+01 -1.06E+03 + 5.54E+02 + 1.16E+01 + 1.39E+01 + 3.43E+07 + 7.07E+07 + 2.86E+02 + 1.28E+03 + 9.37E+01 + 4.55E+02 + 2.43E+01 + 6.10E+01 + 1.84E+02 + 1.19E+02 + 1.13E+02 + 4.18E+01 + 3.55E+03 + 1.02E+04 + 2.40E+02 + 1.30E+03 + 1.24E+02 + 7.00E+01 + 1.12E+02 -3.00E+01 + 5.33E+02 + 2.65E+02 + 3.16E+02 + 9.12E+00 -1.84E+02 -1.20E+02 + 4.40E+02 + 6.48E+01 -5.27E+02 + 4.40E+02 + 3.94E+02 + 1.08E+01 + 5.22E+02 + 1.24E+02 + 2.92E+02 + 6.72E+01 + 1.04E+06 + 2.67E+06	4.33E-09 1.50E-08 1.71E-14 3.04E-14 8.01E-01 4.66E-01 4.68E+00 1.50E+00 3.78E-07 2.07E-06 1.40E+01 1.51E+00 5.21E+00 1.88E+00 1.51E-02 8.29E-02 1.87E+02 9.31E+01 1.13E+00 7.72E-01 6.74E+01 1.33E+02 4.78E+00 1.99E+00 1.65E+00 1.13E+00 5.95E-01 8.15E-01 3.12E+00 4.70E+00 2.72E+00 5.13E+00 3.74E-01 4.50E-01 5.49E-02 1.82E-01 4.44E-01 5.51E-01 1.58E+02 5.46E+01 9.61E+01 1.76E+01 3.09E+02 3.17E+00 3.39E+02 2.54E+00 4.12E+02 2.19E+01 3.09E+02 2.56E+01 3.90E+02 1.34E+00 3.45E+02 1.01E+02 2.35E+02 3.56E+00 7.41E+02 2.79E+02
	Total	+/=/-	17/0/12	15/0/14	19/0/10	17/0/12	24/0/5	27/0/2
					131

Comparing to iff-ABC and MGABC, FOABC surpasses them on 19 and 17 functions, respectively. ILTD_ABC fails to achieve better results than FOABC does

5.3 Experiments on function optimization problems

Table 5 .

 5 7: Comparison between FOABC and other ABC variants with D = 30

	Functions		ABC	NSABC	if f -ABC	MGABC	ILTD_ABC	sdABC	FOABCr=12,q=0.8
	f1 f3 f4 f5 f6 f7 f8 f9 f10 f11 f12 f13 f14 f15 f16 f17 f18 f19 f20 f21 f22 f23 f24 f25 f26 f27 f28 f29 f30	Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std Mean 4.37E+02 -1.73E+02 -1.32E+02 7.69E+04 + 5.47E+04 + 1.13E+05 + 4.13E+04 + 1.71E+03 + 2.38E+03 + 1.29E+02 -4.02E+03 1.37E+03 1.46E+02 1.97E+04 1.20E+04 1.52E+04 6.83E+03 2.38E+01 -4.57E+01 9.69E+01 + -1.72E+01 -2.49E+01 2.82E+01 2.16E+01 2.53E+01 8.23E+01 + 4.13E+01 + 8.42E+01 + 5.95E+01 + 1.01E+01 7.09E+00 1.53E+01 1.03E+01 4.80E-10 3.19E-09 -1.14E-13 --2.65E-14 -5.32E-10 3.57E-09 0.00E+00 8.28E-14 9.89E+01 + 8.88E+01 + 9.87E+01 + 6.94E+01 -1.09E+01 1.05E+01 9.94E+00 9.02E+00 9.14E+01 + 5.72E+01 + 9.20E+01 + 3.88E+01 -1.43E+01 1.40E+01 9.65E+00 6.79E+00 6.33E+02 + 7.93E+00 + 6.92E+02 + 5.73E+01 + 3.85E+02 8.46E+00 3.97E+02 5.79E+01 2.33E+03 + 2.40E+03 + 2.24E+03 -2.14E+03 -2.11E+03 -5.82E+03 + 6.76E+03 + 7.90E+09 + 8.77E+03 4.91E+09 5.24E+04 + 5.37E+04 + 7.66E+03 2.24E+04 7.38E+01 -1.93E+03 + 1.75E+01 2.64E+03 6.78E+01 + 1.90E+02 + 9.96E+00 5.23E+01 1.34E-03 -3.24E+01 + 4.69E-04 2.26E+01 1.18E+02 + 3.46E+02 + 3.66E+01 1.18E+02 7.30E+01 + 1.75E+02 + 1.66E+01 5.20E+01 1.24E+02 + 6.92E+03 + 2.81E+02 1.96E+03 2.03E+02 4.85E+02 2.19E+02 2.17E+02 7.76E+02 4.30E+02 1.90E+02 + 1.97E+02 + 3.51E+02 + 6.42E+01 + 9.61E+01 + 1.14E+03 + 9.13E+01 1.26E+02 2.68E+02 3.79E+01 2.51E+01 1.66E+03 4.65E+05 + 5.92E+05 + 3.69E+05 + 9.61E+05 + 6.25E+05 + 2.12E+08 + 2.11E+05 4.21E+05 2.74E+05 5.72E+05 3.05E+05 7.62E+08 7.83E+03 -1.08E+04 -1.85E+04 + 1.20E+08 + 1.73E+04 + 3.12E+03 -4.90E+03 1.76E+04 7.88E+03 1.47E+04 6.56E+08 2.40E+03 6.70E+04 + 4.46E+04 + 6.90E+04 + 4.26E+05 + 2.18E+05 + 1.74E+02 + 5.27E+04 6.03E+04 4.41E+04 3.80E+05 2.21E+05 5.32E+02 1.36E+03 -1.02E+03 -2.80E+03 + 3.07E+07 + 7.24E+03 + 7.63E+02 -1.06E+03 7.53E+03 1.30E+03 1.80E+03 1.17E+08 6.99E+02 6.72E+02 + 5.31E+02 + 6.50E+02 + 6.18E+02 + 6.44E+02 + 8.62E+02 + 1.30E+02 1.74E+02 1.47E+02 1.23E+02 2.33E+02 4.30E+02 1.86E+02 + 3.20E+02 + 1.98E+02 + 1.34E+02 + 2.47E+02 + 1.17E+02 -8.71E+01 8.02E+01 9.15E+01 1.51E+02 3.00E+02 6.12E+01 1.77E+05 + 2.26E+05 + 1.68E+05 + 2.22E+05 + 2.88E+05 + 2.55E+06 + 9.35E+04 1.44E+05 9.02E+04 1.99E+05 1.99E+05 5.49E+06 1.37E+03 + 1.38E+04 + 9.30E+02 + 3.21E+03 + 3.55E+03 + 7.33E+07 + 1.29E+03 1.64E+04 1.13E+03 2.56E+03 2.73E+03 1.74E+08 2.39E+02 + 1.59E+02 -2.82E+02 + 5.78E+02 + 2.81E+02 + 1.56E+02 -8.22E+01 9.80E+01 1.03E+02 1.38E+02 2.06E+02 4.55E+01 2.52E+02 + 2.49E+02 + 2.50E+02 + 3.27E+02 + 2.58E+02 + 2.39E+02 -6.70E+01 7.29E+01 9.45E+00 9.87E+00 4.11E+01 2.32E+01 4.68E+02 -1.00E+02 -3.89E+02 1.00E+02 -1.58E+03 + -1.00E+02 -9.43E+02 7.25E-07 8.61E+02 7.03E-03 8.58E+02 0.00E+00 4.08E+02 + 5.10E+02 + 4.24E+02 + 3.97E+02 + 4.11E+02 + 3.87E+02 -2.43E+01 8.64E+00 2.82E+01 1.13E+01 3.65E+01 5.50E+01 5.18E+02 + 4.74E+02 + 4.99E+02 + 5.32E+02 + 8.32E+02 + Std 2.25E+01 2.02E+02 7.65E+01 2.08E+01 3.32E+02 2.04E+02 Mean 3.85E+02 3.84E+02 -3.94E+02 + 3.86E+02 -6.17E+02 + -3.78E+02 -Std 1.13E+00 1.21E+00 1.44E+01 2.01E+00 1.94E+02 1.06E+00 1.36E+03 -3.59E+02 -5.57E+02 -1.29E+03 -2.69E+03 + Mean 3.41E+02 -Std 2.71E+02 4.04E+02 6.53E+02 5.73E+02 1.74E+03 3.73E+02 Mean 5.12E+02 + 5.10E+02 -5.03E+02 -5.40E+02 + 5.12E+02 + 4.97E+02 -Std 5.36E+00 8.29E+00 5.23E+00 4.51E+00 2.10E+01 1.06E+01 Mean 4.04E+02 + 4.95E+02 + 3.97E+02 + 4.06E+02 + 4.00E+02 + 1.42E+03 + Std 3.77E+00 1.63E+01 1.28E+01 4.59E+00 5.00E+00 8.94E+02 Mean 5.87E+02 + 5.11E+02 + 5.40E+02 + 8.42E+02 + 5.91E+02 + 4.27E+02 -Std 8.52E+01 9.03E+01 6.60E+01 6.55E+01 3.24E+02 8.21E+01 Mean 5.52E+03 + 5.37E+03 + 4.87E+03 + 1.23E+05 + 6.55E+03 + 3.13E+03 -Std 2.32E+03 3.14E+03 1.65E+03 1.21E+03 1.61E+03 4.58E+05	1.50E+03 2.35E+03 1.10E+04 5.59E+03 8.71E+01 1.76E+01 3.73E+01 9.07E+00 3.55E-03 1.18E-02 7.84E+01 9.53E+00 4.15E+01 1.01E+01 6.11E+00 7.86E+00 2.30E+03 2.58E+02 2.71E+01 2.19E+01 2.77E+04 1.54E+04 1.11E+04 1.33E+04 3.93E+01 2.55E+01 1.59E+03 3.22E+03 4.46E+02 2.00E+02 1.33E+02 1.18E+02 9.43E+03 7.41E+03 4.48E+02 1.81E+03 1.85E+02 1.23E+02 2.42E+02 8.45E+00 8.48E+02 1.18E+03 3.94E+02 8.04E+00 4.72E+02 9.01E+00 3.87E+02 1.45E+00 1.49E+03 1.40E+02 5.11E+02 7.45E+00 3.65E+02 5.86E+01 4.95E+02 7.33E+01 4.50E+03 2.64E+03
	Total	+/=/-	20/0/9	17/0/12	20/0/9	19/0/10	22/0/7	29/0/0

Table 5 .

 5 7 shows the comparison at D = 30, where FOABC obtains the best results on 10 functions. And it is also followed by NSABC. Note that the difficulty of searching will increase when the size of dimensions augments and the swarm5. FRACTIONAL-ORDER ABC ALGORITHM (FOABC)

Table 5 .

 5 8: Comparison between FOABC and other ABC variants with D = 50

	Functions	ABC	NSABC	if f -ABC	MGABC	ILTD_ABC	sdABC	FOABC r=12,q=0.8
	f 1 f 3 f 4 f 5 f 6 f 7 f 8 f 9 f 10 f 11 f 12 f 13 f 14 f 15 f 16 f 17 f 18 f 19 f 20 f 21 f 22 f 23 f 24 f 25 f 26 f 27 f 28 f 29 f 30	Mean 1.85E+03 -Std 1.91E+03 Mean 2.07E+05 + 2.67E+05 + 2.61E+05 + 1.16E+05 + 5.87E+03 -1.11E+03 -5.30E+02 -7.17E+03 1.23E+03 7.17E+02 Std 3.23E+04 2.97E+04 2.23E+04 1.28E+04 Mean 4.06E+01 -9.12E+01 -5.67E+01 -3.33E+01 -Std 1.33E+01 2.48E+01 4.83E+01 9.37E+00 Mean 1.98E+02 + 1.19E+02 + 2.01E+02 + 1.59E+02 + Std 1.69E+01 1.68E+01 1.98E+01 2.29E+01 Mean 5.20E-10 -9.40E-13 -1.44E-09 -1.10E-13 -Std 3.95E-10 1.81E-13 1.25E-09 2.08E-14 2.09E+02 + 2.01E+02 + Mean 2.13E+02 + 1.52E+02 -Std 1.90E+01 1.79E+01 2.35E+01 1.68E+01 Mean 1.96E+02 + 1.16E+02 + 2.00E+02 + 1.55E+02 + Std 2.17E+01 1.44E+01 2.66E+01 1.65E+01 Mean 4.52E+03 + 5.38E+02 + 4.40E+03 + 6.33E+02 + Std 1.49E+03 2.07E+02 1.18E+03 5.64E+02 Mean 4.29E+03 -4.23E+03 -4.16E+03 -3.98E+03 -Std 3.46E+02 5.60E+02 3.74E+02 3.87E+02 Mean 7.17E+02 + 4.40E+03 + 9.19E+02 + 8.24E+02 + Std 5.26E+02 2.87E+03 7.47E+02 4.70E+02 Mean 3.40E+06 + 6.73E+06 + 2.70E+06 + 2.26E+06 + Std 1.30E+06 3.05E+06 1.13E+06 8.71E+05 Mean 4.46E+03 -6.11E+03 -2.01E+03 -1.50E+03 -Std 2.17E+03 8.22E+03 1.69E+03 1.69E+03 Mean 6.22E+05 + 1.29E+06 + 7.31E+05 + 2.44E+06 + Std 5.16E+05 7.46E+05 5.04E+05 1.21E+06 Mean 6.64E+03 + 2.18E+04 + 8.08E+03 + 1.42E+04 + Std 3.86E+03 4.68E+04 5.70E+03 4.95E+03 Mean 1.25E+03 + 1.29E+03 + 1.25E+03 + 9.59E+02 -9.20E+02 -1.89E+03 + 6.79E+03 + 3.13E+10 + 1.10E+04 1.98E+10 1.18E+05 + 1.47E+05 + 9.03E+03 2.86E+04 4.54E+01 -5.84E+03 + 2.16E+01 5.65E+03 2.19E+02 + 4.08E+02 + 4.84E+01 9.31E+01 2.90E-03 -3.75E+01 + 1.13E-03 8.72E+00 3.40E+02 + 7.59E+02 + 1.14E+02 1.14E+02 1.86E+02 + 4.40E+02 + 3.13E+01 8.54E+01 3.09E+03 + 2.51E+04 + 2.53E+03 1.06E+04 4.23E+03 -1.08E+04 + 8.69E+02 2.69E+03 1.67E+03 + 3.75E+03 + 5.91E+02 3.53E+03 1.69E+06 + 2.86E+09 + 5.34E+05 3.04E+09 2.76E+03 -1.21E+08 + 3.82E+03 1.89E+08 9.57E+05 + 8.02E+04 + 5.85E+05 3.94E+05 1.45E+04 + 5.89E+08 + 6.74E+03 1.56E+09 Std 2.33E+02 2.19E+02 1.75E+02 2.41E+02 4.64E+02 2.53E+02 Mean 8.71E+02 + 9.21E+02 + 8.92E+02 + 8.72E+02 + 8.82E+02 + 2.15E+03 + Std 2.02E+02 2.12E+02 1.61E+02 1.97E+02 2.07E+02 3.34E+03 Mean 1.03E+06 + 2.17E+06 + 8.36E+05 + 2.40E+06 + 1.27E+06 + 2.92E+06 + Std 5.53E+05 1.69E+06 5.06E+05 9.96E+05 5.62E+05 1.06E+07 Mean 9.14E+03 -1.70E+04 + 1.98E+04 + 1.17E+05 + 1.59E+04 + 6.38E+03 -Std 4.92E+03 1.16E+04 5.00E+03 1.15E+04 4.52E+05 3.44E+03 5.61E+02 -1.27E+03 + Mean 7.45E+02 + 7.26E+02 + 6.94E+02 + 4.67E+02 -Std 1.44E+02 2.02E+02 1.54E+02 2.40E+02 4.55E+02 1.74E+02 Mean 4.04E+02 + 3.29E+02 + 4.09E+02 + 3.28E+02 + 3.41E+02 + 5.37E+02 + Std 4.87E+01 2.14E+01 2.48E+01 1.59E+01 2.37E+01 4.97E+01 Mean 4.76E+03 -4.74E+03 -4.47E+03 -8.33E+03 + 4.67E+03 -3.89E+03 -Std 1.02E+03 4.54E+02 1.30E+03 2.27E+03 4.88E+03 1.96E+03 Mean 6.48E+02 + 5.55E+02 + 6.55E+02 + 5.53E+02 + 5.69E+02 + 8.52E+02 + Std 3.77E+01 3.13E+01 2.89E+01 1.96E+01 1.86E+01 6.84E+01 Mean 9.60E+02 + 8.87E+02 + 1.03E+03 + 7.61E+02 + 8.14E+02 + 1.06E+03 + Std 1.49E+02 5.22E+01 6.92E+01 3.71E+01 3.85E+01 1.17E+02 4.94E+02 -5.91E+02 + 5.59E+02 + 3.58E+03 + Mean 5.12E+02 -4.39E+02 -Std 1.73E+01 2.32E+01 1.29E+01 2.27E+01 2.01E+03 1.51E+01 Mean 1.87E+03 -1.80E+03 -6.88E+03 + 2.55E+03 + 1.31E+03 -8.62E+02 -Std 1.56E+03 4.93E+02 1.46E+03 1.25E+03 2.64E+03 1.47E+03 6.52E+02 -6.26E+02 -5.98E+02 -8.83E+02 + Mean 6.43E+02 -5.00E+02 -Std 4.10E+01 3.62E+01 2.88E+01 2.45E+01 1.34E+02 2.61E-04 Mean 4.83E+02 -5.32E+02 + 5.00E+02 -2.60E+03 + 4.83E+02 -4.74E+02 -Std 1.26E+01 2.34E+01 1.97E+01 1.94E+01 1.39E+03 1.26E+01 Mean 1.01E+03 + 8.11E+02 + 1.08E+03 + 8.58E+02 + 6.89E+02 + 1.85E+03 + Std 1.48E+02 1.33E+02 2.26E+02 1.53E+02 1.91E+02 1.59E+03 7.13E+05 -8.45E+05 -8.81E+05 + 9.89E+06 + Mean 7.24E+05 -7.02E+03 -Std 5.09E+04 8.54E+03 5.18E+04 6.47E+04 1.14E+05 9.46E+06	6.42E+03 6.87E+03 1.15E+05 1.74E+04 1.19E+02 6.27E+01 1.03E+02 1.95E+01 5.50E-02 6.02E-02 1.78E+02 2.01E+01 1.06E+02 1.59E+01 2.96E+02 2.68E+02 4.50E+03 2.69E+02 7.12E+01 2.33E+01 4.26E+05 2.87E+05 8.38E+03 9.85E+03 4.45E+03 3.95E+03 5.38E+03 5.54E+03 1.02E+03 2.46E+02 6.05E+02 1.89E+02 9.10E+04 6.00E+04 1.52E+04 1.27E+04 5.65E+02 2.45E+02 3.09E+02 1.47E+01 5.17E+03 4.45E+02 5.43E+02 1.83E+01 6.16E+02 1.98E+01 5.45E+02 3.44E+01 2.34E+03 1.88E+02 6.68E+02 6.80E+01 5.06E+02 2.34E+01 6.62E+02 1.64E+02 8.47E+05 2.54E+05
	Total	+/=/-17/0/12	18/0/11	17/0/12	18/0/11	19/0/10	29/0/0

  As shown in Table5.12, there are 12 functions where FOABC obtains the best results while CLPSO achieves the best on 10 functions. FOCS algorithm manages to obtain the best on 5 problems. It can be found that the proposed algorithm is able to find competitive solutions on multi-modal, hybrid and composition functions. Considering the results of Wilcoxon tests, the CLPSO obtains better

	5.3 Experiments on function optimization problems 5. FRACTIONAL-ORDER ABC ALGORITHM (FOABC)
	Table 5.10: Comparison between FOABC and other improved meta-heuristic modal and composition problems. According to the Wilcoxon test results, there Table 5.11: Comparison between FOABC and other improved meta-heuristic algorithms with D = 10 algorithms with D = 30 are 18 functions that FOABC exceeds CLPSO. In addition, FOABC outperforms
	It is crucial to further compare FOABC with other FO-based algorithms and improved versions of other meta-heuristics in order to completely evaluate its efficacy. As presented in Table 5.9, five effective enhanced algorithms are involved in the following comparisons. It is worth pointing out that the FODPSO, FOCS, and FOFA algorithms were chosen because they also incorporate FOC in their basic algorithms. In this case, by comparing FOABC with them, it is possible to check whether the proposed manner of incorporating FOC into ABC is more effective. And the algorithms LIPS and CLPSO are two excellent algorithms that Functions CLPSO LIPS FODPSO FOFA FOCS FOABC r=12,q=0.8 f 1 Mean 1.52E+02 + 4.90E+02 + 3.43E+01 + 2.95E+10 + 5.44E+01 + 4.33E-09 Std 1.16E+02 7.65E+02 5.59E+01 9.68E+07 1.12E+02 1.50E-08 f 3 Mean 2.97E+00 + 6.10E-07 + 4.67E-10 + 3.13E+05 + 2.86E-02 + 1.71E-14 Std 3.02E+00 1.99E-06 7.45E-10 1.37E+05 3.62E-02 3.04E-14 f 4 Mean 2.58E+00 + 3.18E+00 + 6.02E-01 -5.38E+03 + 4.15E-01 -8.01E-01 Std 1.33E+00 1.55E+00 8.01E-01 3.30E+01 2.03E-01 4.66E-01 f 5 Mean 5.29E+00 + 6.60E+00 + 3.11E+01 + 2.15E+02 + 1.35E+01 + 4.68E+00 Std 1.41E+00 2.59E+00 7.73E+00 2.88E+00 3.46E+00 1.50E+00 f 6 Mean 1.69E-10 -8.14E-02 + 6.59E+00 + 1.21E+02 + 3.53E-02 + 3.78E-07 Std 1.22E-10 2.21E-01 4.56E+00 4.28E+00 2.18E-02 2.07E-06 f 7 Mean 1.82E+01 + 1.42E+01 + 2.27E+01 + 1.95E+02 + 2.87E+01 + 1.40E+01 Std 2.10E+00 1.21E+00 3.97E+00 7.51E+00 5.16E+00 1.51E+00 f 8 Mean 5.60E+00 + 6.53E+00 + 1.17E+01 + 1.35E+02 + 1.32E+01 + 5.21E+00 Std 1.12E+00 2.69E+00 3.98E+00 2.43E+00 3.60E+00 1.88E+00 f 9 Mean 1.38E-08 -6.33E-13 -6.35E+00 + 1.78E+03 + 1.09E-02 -1.51E-02 Std 1.72E-08 4.14E-13 1.96E+01 2.41E+02 2.87E-02 8.29E-02 f 10 Mean 2.30E+02 + 4.66E+02 + 6.14E+02 + 4.94E+03 + 4.51E+02 + 1.87E+02 Std 9.12E+01 1.39E+02 1.81E+02 4.64E+01 1.45E+02 9.31E+01 f 11 Mean 2.46E+00 + 1.56E+01 + 1.36E+01 + 5.89E+07 + 4.78E+00 + 1.13E+00 Std 9.22E-01 1.13E+01 5.37E+00 1.73E+06 1.23E+00 7.72E-01 f 12 Mean 1.84E+04 + 4.08E+04 + 1.59E+03 + 5.57E+09 + 8.93E+02 + 6.74E+01 Std 1.11E+04 1.30E+05 7.31E+02 4.68E+07 2.77E+02 1.33E+02 f 13 Mean 8.40E+01 + 9.16E+02 + 6.73E+02 + 2.74E+09 + 1.25E+01 + 4.78E+00 Std 1.11E+02 1.77E+03 5.32E+02 2.32E+07 3.10E+00 1.99E+00 f 14 Mean 4.56E+01 + 4.27E+02 + 7.83E+01 + 2.12E+09 + 1.18E+01 + 1.65E+00 Std 4.04E+01 7.56E+02 4.59E+01 2.31E+07 4.65E+00 1.13E+00 f 15 Mean 2.31E+01 + 4.30E+02 + 1.00E+02 + 6.73E+08 + 2.92E+00 + 5.95E-01 Std 2.15E+01 7.95E+02 8.28E+01 2.65E+07 9.28E-01 8.15E-01 f 16 Mean 1.71E+00 -6.73E+01 + 1.22E+02 + 1.77E+03 + 5.83E+00 + 3.12E+00 Std 5.85E-01 8.66E+01 8.41E+01 2.09E+01 7.94E+00 4.70E+00 f 17 Mean 4.82E+00 + 3.68E+01 + 2.97E+01 + 1.46E+03 + 2.75E+01 + 2.72E+00 Std 3.06E+00 1.42E+01 8.34E+00 1.91E+01 2.89E+00 5.13E+00 f 18 Mean 5.14E+02 + 1.99E+03 + 1.80E+03 + 1.41E+10 + 1.30E+01 + 3.74E-01 Std 4.52E+02 1.75E+03 1.82E+03 8.86E+07 4.97E+00 4.50E-01 f 19 Mean 1.02E+01 + 1.32E+03 + 1.97E+02 + 1.19E+10 + 2.39E+00 + 5.49E-02 Std 9.33E+00 2.22E+03 2.66E+02 9.92E+07 5.50E-01 1.82E-01 f 20 Mean 5.51E-02 -3.10E+01 + 4.28E+01 + 1.04E+03 + 1.14E+01 + 4.44E-01 Std 1.41E-01 1.71E+01 1.75E+01 2.79E+01 5.97E+00 5.51E-01 f 21 Mean 1.07E+02 -1.89E+02 + 1.00E+02 -7.12E+02 + 1.67E+02 + 1.58E+02 Std 4.84E+00 4.08E+01 1.26E+00 5.29E+00 5.71E+01 5.46E+01 f 22 Mean 7.71E+01 -9.55E+01 -9.24E+01 -2.98E+03 + 8.56E+01 -9.61E+01 Std 2.31E+01 1.87E+01 1.95E+01 3.75E+01 3.17E+01 1.76E+01 f 23 Mean 3.06E+02 -3.03E+02 -3.51E+02 + 1.94E+03 + 3.13E+02 + 3.09E+02 Std 5.09E+00 3.89E+01 1.24E+01 2.61E+01 3.51E+00 3.17E+00 f 24 Mean 1.50E+02 -2.54E+02 -1.34E+02 -9.83E+02 + 3.29E+02 -3.39E+02 Std 6.09E+01 1.05E+02 1.00E+02 2.73E+00 6.11E+01 2.54E+00 f 25 Mean 3.73E+02 -3.91E+02 -3.88E+02 -2.28E+03 + 4.04E+02 -4.12E+02 Std 5.94E+01 8.45E+01 5.44E+01 1.48E+01 1.59E+01 2.19E+01 f 26 Mean 1.62E+02 -2.65E+02 -1.41E+02 -3.09E+03 + 3.00E+02 -3.09E+02 Std 1.08E+02 1.96E+02 1.09E+02 1.18E+01 2.81E-06 2.56E+01 f 27 Mean 3.92E+02 + 4.04E+02 + 4.17E+02 + 2.26E+03 + 3.90E+02 + 3.90E+02 Std 1.70E+00 5.89E+00 1.67E+01 2.12E+01 2.75E+00 1.34E+00 f 28 Mean 2.68E+02 -4.35E+02 + 3.16E+02 -1.69E+03 + 4.04E+02 + 3.45E+02 Std 1.02E+02 1.42E+02 4.33E+01 7.22E+00 1.53E+02 1.01E+02 f 29 Mean 2.59E+02 + 2.88E+02 + 2.91E+02 + 4.06E+04 + 2.67E+02 + 2.35E+02 Std 8.73E+00 2.40E+01 1.45E+01 1.24E+03 2.75E+01 3.56E+00 f 30 Mean 9.17E+03 + 1.44E+05 + 2.66E+03 + 4.88E+08 + 1.26E+05 + 7.41E+02 Std 7.47E+03 6.07E+05 8.45E+02 5.08E+06 2.81E+05 2.79E+02 Functions CLPSO LIPS FODPSO FOFA FOCS both LIPS and FOCS on 24 problems, while it surpasses FODPSO on 22 out of FOABC r=12,q=0.8 f 1 Mean 1.49E+01 -3.78E+02 -4.87E+06 + 8.43E+10 + 1.00E+10 + 1.50E+03 Std f 3 Mean Std 5.35E+03 8.02E+03 f 4 Std f 5 Mean Std 7.89E+00 1.17E+01 2.10E+01 3.15E+00 f 6 Mean Std f 7 Mean Std 7.91E+00 1.80E+01 3.69E+01 1.02E+01 f 8 Mean Std 7.18E+00 1.71E+01 1.60E+01 2.29E+00 f 9 Mean Std 2.06E+01 2.84E+02 3.89E+02 1.47E+03 f 10 Std f 11 Mean Std 2.08E+01 1.27E+02 1.77E+01 1.18E+07 f 12 Mean Std 1.78E+05 4.56E+06 2.66E+04 5.81E+07 f 13 Std f 14 Mean Std 2.95E+04 8.23E+03 2.16E+03 1.21E+07 f 15 Std f 16 Mean Std 1.56E+02 1.78E+02 1.79E+02 1.27E+02 f 17 Mean Std 6.94E+01 9.06E+01 1.46E+02 3.72E+03 f 18 Mean Std 7.49E+04 1.02E+05 1.71E+04 3.86E+07 f 19 Mean 5.09E+01 -Std 3.69E+01 1.41E+03 2.23E+02 4.46E+07 f 20 Mean Std 5.97E+01 9.96E+01 8.26E+01 2.23E+01 f 21 Std f 22 Mean Std 4.82E+02 f 23 Mean Std 1.07E+01 2.12E+01 8.18E+01 3.47E+01 f 24 Mean Std 9.81E+01 3.19E+01 1.40E+02 2.34E+00 f 25 Std f 26 Std f 27 Mean Std 4.60E+00 2.10E+01 2.82E+01 4.39E+01 f 28 Mean Std 8.60E+00 8.76E+01 2.24E+01 2.02E+01 f 29 Mean Std 6.01E+01 1.20E+02 1.34E+02 5.13E+03 f 30 Mean Std 1.84E+03 1.27E+05 4.31E+03 3.03E+07 2.64E+03 6.97E+03 4.50E+03 6.19E+03 + 8.33E+04 + 8.93E+03 + 1.02E+10 + 1.63E+04 + 7.33E+01 1.46E+02 4.95E+02 5.52E+02 + 9.44E+02 + 1.01E+03 + 2.23E+05 + 9.01E+02 + 5.86E+01 5.46E+01 3.65E+02 4.22E+02 + 4.73E+02 + 4.54E+02 + 7.39E+03 + 3.76E+02 + 7.45E+00 1.81E+01 5.11E+02 5.12E+02 + 6.07E+02 + 6.00E+02 + 7.79E+03 + 5.20E+02 + 5.18E+02 8.61E+02 1.09E+03 3.29E+01 4.96E+02 1.40E+02 Mean 7.67E+02 -1.28E+03 -1.00E+03 -1.35E+04 + 2.41E+03 + 1.49E+03 1.04E+00 2.52E+01 1.01E+01 3.42E+01 9.97E-01 1.45E+00 Mean 3.87E+02 -4.26E+02 + 4.07E+02 + 6.65E+03 + 3.87E+02 -3.87E+02 9.01E+00 3.35E+01 4.72E+02 4.73E+02 + 4.96E+02 + 6.82E+02 + 2.79E+03 + 5.61E+02 + 8.04E+00 1.68E+01 3.94E+02 3.98E+02 + 4.41E+02 + 7.32E+02 + 5.64E+03 + 4.76E+02 + 1.01E+00 1.79E+01 5.57E+01 2.20E+03 1.18E+03 2.20E+02 -1.00E+02 -1.35E+02 -1.09E+04 + 3.04E+03 + 8.48E+02 4.64E+01 1.55E+01 2.47E+01 2.75E+00 2.06E+01 8.45E+00 Mean 2.28E+02 -2.63E+02 + 3.54E+02 + 1.12E+03 + 3.12E+02 + 2.42E+02 1.23E+02 1.03E+02 1.85E+02 2.00E+02 + 3.01E+02 + 3.65E+02 + 3.41E+03 + 2.28E+02 + 1.17E+01 1.81E+03 1.11E+03 + 2.91E+02 -6.51E+09 + 4.68E+01 -4.48E+02 7.41E+03 6.55E+03 9.43E+03 1.39E+05 + 1.67E+05 + 3.92E+04 + 4.63E+09 + 1.17E+04 + 1.18E+02 6.79E+01 1.33E+02 1.56E+02 + 2.55E+02 + 3.51E+02 + 2.72E+05 + 2.41E+02 + 2.00E+02 1.97E+02 4.46E+02 5.40E+02 + 6.85E+02 + 8.62E+02 + 2.54E+04 + 8.83E+02 + 4.87E+01 1.90E+03 4.68E+02 5.64E+07 3.36E+01 3.22E+03 Mean 9.76E+01 -1.52E+03 -9.48E+02 -6.34E+09 + 1.38E+02 -1.59E+03 2.55E+01 1.05E+01 3.93E+01 3.61E+04 + 1.02E+04 + 1.90E+03 + 1.21E+09 + 8.02E+01 + 2.99E+02 2.40E+03 2.31E+03 8.06E+07 9.61E+02 1.33E+04 Mean 3.16E+02 -2.77E+03 -7.15E+03 -4.38E+10 + 2.08E+03 -1.11E+04 1.54E+04 4.66E+09 2.77E+04 3.69E+05 + 1.46E+06 + 4.81E+04 + 2.93E+10 + 3.00E+09 + 2.19E+01 2.96E+01 2.71E+01 6.01E+01 + 1.85E+02 + 8.78E+01 + 5.76E+08 + 1.22E+02 + 2.55E+02 4.28E+02 3.96E+02 7.53E+01 6.91E+02 2.58E+02 Mean 2.21E+03 -2.70E+03 + 3.17E+03 + 1.01E+04 + 4.26E+03 + 2.30E+03 7.86E+00 1.57E+02 6.11E+00 3.99E+01 + 3.79E+02 + 2.43E+03 + 2.72E+04 + 1.86E+02 + 1.01E+01 1.55E+01 4.15E+01 5.18E+01 + 6.59E+01 + 1.08E+02 + 5.11E+02 + 1.05E+02 + 9.53E+00 1.91E+01 7.84E+01 8.13E+01 + 9.33E+01 + 1.99E+02 + 9.18E+02 + 1.68E+02 + 4.22E-14 3.58E+00 3.59E+00 1.71E+00 1.41E-01 1.18E-02 2.27E-13 -7.57E+00 + 4.41E+01 + 1.41E+02 + 1.49E-01 + 3.55E-03 9.07E+00 1.74E+01 3.73E+01 4.21E+01 + 5.50E+01 + 1.60E+02 + 6.16E+02 + 1.10E+02 + 2.28E+01 6.22E+01 2.42E+01 1.29E+02 1.43E+01 1.76E+01 Mean 5.56E+01 -1.37E+02 + 9.95E+01 + 3.46E+04 + 7.40E+01 -8.71E+01 2.09E+03 1.46E+08 9.26E+03 5.59E+03 1.92E+04 + 2.06E+04 + 8.07E+03 -4.15E+08 + 3.11E+04 + 1.10E+04 2.89E+01 9.29E+02 6.56E+06 1.74E+08 0.00E+00 2.35E+03 29 functions. And FOFA fails to exceed FOABC on the concerned benchmarks.
	have been widely compared in various experiments. The tests are also conducted Total +/=/-18/0/11 23/0/6 22/0/7 29/0/0 23/0/6 Total +/=/-18/0/11 24/0/5 23/0/6 29/0/0 24/0/5
	on the CEC 2017 benchmarks with D = 10, 30, and 50.

Table 5

 5 In this part, the effectiveness of each proposed improvement is tested and analyzed. Hence, two variants of FOABC are constructed, namely FOABC DE and FOABC F O . The employed bee phase of FOABC DE adopts the DE-based search strategy, i.e., Eq.(5.7), with scale factors following Lévy distribution. And the rest of FOABC DE remains the same as the standard ABC algorithm. Moreover, in order to verify the efficacy of FOC in aiding the search process of ABC, FOABC F O = 50 are presented in Table 5.13. The best results of each function are highlighted in boldface. And the results of the Wilcoxon and Friedman tests are given at the bottom of the table. In the Wilcoxon tests, these two variants are compared to the standard ABC and the final FOABC, respectively. In Table 5.13, the variant FOABC DE and the basic ABC are first observed and compared. It can be noticed that FOABC DE has significant improvements in functions f 11 , f 14 , f 18 , and f 29 . Meanwhile, the advantage of FOABC DE is verified by the Wilcoxon test results between it and the basic ABC. FOABC DE variant

	.12: Comparison between FOABC and other improved meta-heuristic
	algorithms with D = 50
	Functions		CLPSO	LIPS	FODPSO	FOFA	FOCS	FOABC r=12,q=0.8
	f 1 f 3 f 4 f 5 f 6 f 7 f 8 f 9 f 10 f 11 f 12 f 13 f 14 f 15 f 16 f 17 f 18 f 19 f 20 f 21 f 22 f 23 f 24 f 25 f 26 f 27 f 28 f 29 f 30	1.01E+09 + 1.35E+11 + 1.00E+10 + 7.67E+08 1.36E+08 0.00E+00 7.05E+04 -1.04E+05 -4.60E+04 -1.77E+14 + 1.06E+05 -Mean 2.89E+01 -4.86E+02 -Std 1.03E+03 4.32E+01 Mean Std 1.26E+04 2.08E+04 3.78E+12 2.63E+04 7.72E+03 Mean 8.21E+01 -6.13E+02 + 4.55E+02 + 5.66E+04 + 7.92E+01 -Std 2.61E+01 2.39E+02 8.02E+01 9.49E+01 4.13E+01 Mean 1.14E+02 + 1.68E+02 + 2.69E+02 + 8.64E+02 + 2.50E+02 + Std 1.38E+01 2.75E+01 2.41E+01 3.07E+00 2.36E+01 Mean 6.43E-01 + 3.07E-13 -2.13E+01 + 5.24E+01 + 1.45E+02 + Std 5.83E+00 3.88E+00 8.94E-01 4.32E-01 9.03E-14 Mean 1.65E+02 -3.49E+02 + 4.86E+02 + 1.48E+03 + 3.91E+02 + Std 7.47E+01 5.85E+01 1.42E+01 4.06E+01 1.09E+01 Mean 1.13E+02 + 1.62E+02 + 2.84E+02 + 9.05E+02 + 2.62E+02 + Std 1.43E+01 3.15E+01 2.36E+01 2.99E+00 2.62E+01 Mean 1.01E+03 + 3.40E+03 + 8.75E+03 + 6.73E+04 + 2.02E+03 + Std 3.25E+02 1.18E+03 8.12E+02 3.25E+03 1.28E+03 Mean 3.92E+03 -5.22E+03 + 5.62E+03 + 2.06E+04 + 9.24E+03 + Std 4.38E+02 5.66E+02 8.72E+01 7.63E+02 3.43E+02 Mean 1.30E+02 + 1.53E+03 + 2.34E+02 + 1.24E+06 + 2.89E+02 + Std 3.72E+01 1.45E+03 4.26E+01 1.88E+05 5.40E+01 Mean 3.79E+06 + 6.93E+06 + 2.07E+07 + 1.43E+11 + 1.00E+10 + Std 1.56E+06 2.52E+07 1.56E+07 1.93E+08 0.00E+00 1.98E+04 + 1.13E+11 + 1.00E+10 + Mean 4.16E+02 -6.78E+03 -Std 4.79E+03 8.22E+03 1.46E+08 0.00E+00 2.20E+02 Mean 3.93E+05 + 8.90E+04 + 9.92E+03 + 1.45E+09 + 2.86E+02 -Std 2.37E+05 5.12E+04 6.31E+03 4.99E+06 4.84E+01 4.29E+03 -2.38E+10 + 3.33E+08 + Mean 1.82E+02 -1.49E+03 -Std 1.17E+03 1.71E+03 6.80E+07 1.83E+09 1.32E+02 Mean 1.18E+03 + 1.49E+03 + 1.45E+03 + 2.29E+04 + 1.99E+03 + Std 2.08E+02 3.08E+02 1.94E+02 5.29E+01 2.52E+02 Mean 7.95E+02 + 1.12E+03 + 1.15E+03 + 1.69E+05 + 1.28E+03 + Std 1.27E+02 2.55E+02 1.41E+02 2.67E+03 1.81E+02 Mean 8.29E+05 + 1.03E+06 + 7.73E+04 -2.08E+09 + 2.07E+05 + Std 4.05E+05 1.19E+06 1.67E+07 1.51E+05 2.45E+04 Mean 3.14E+02 -2.09E+03 -5.08E+03 -1.39E+10 + 1.46E+02 -Std 3.66E+02 2.97E+03 3.13E+03 5.10E+07 2.61E+01 Mean 6.12E+02 + 6.67E+02 + 8.83E+02 + 3.39E+03 + 1.06E+03 + Std 1.66E+02 1.67E+02 1.69E+02 2.95E+01 2.09E+02 Mean 3.23E+02 + 3.61E+02 + 5.38E+02 + 2.24E+03 + 4.57E+02 + Std 1.06E+01 2.66E+01 3.50E+01 5.01E+00 2.49E+01 6.79E+03 + 1.89E+04 + 9.29E+03 + Mean 4.13E+03 -4.97E+03 -Std 2.05E+03 5.94E+02 5.49E+01 1.31E+03 1.57E+03 Mean 5.66E+02 + 6.96E+02 + 1.28E+03 + 7.30E+03 + 7.48E+02 + Std 1.15E+01 6.43E+01 1.25E+02 2.04E+01 6.19E+01 Mean 7.64E+02 + 7.71E+02 + 1.14E+03 + 4.45E+03 + 8.44E+02 + Std 4.07E+01 8.53E+01 8.22E+01 2.62E+00 7.81E+01 Mean 5.42E+02 -8.64E+02 + 7.61E+02 + 1.75E+04 + 5.28E+02 -Std 1.57E+01 1.93E+02 7.99E+01 3.34E+01 3.45E+01 Mean 2.11E+03 -3.64E+03 + 2.41E+03 + 1.76E+04 + 4.62E+03 + Std 4.53E+02 7.33E+02 3.71E+01 6.03E+02 7.20E+02 Mean 6.38E+02 -1.13E+03 + 1.16E+03 + 1.64E+04 + 8.15E+02 + Std 8.40E+01 1.73E+02 5.25E+01 1.63E+02 2.70E+01 Mean 5.28E+02 + 1.26E+03 + 9.42E+02 + 1.74E+04 + 4.79E+02 -Std 1.54E+01 2.71E+02 1.53E+02 2.54E+01 2.45E+01 Mean 7.90E+02 + 1.93E+03 + 1.96E+03 + 6.60E+06 + 1.80E+03 + Std 1.38E+02 3.21E+02 2.39E+02 6.91E+04 3.07E+02 Mean 7.03E+05 -3.38E+07 + 1.21E+07 + 2.49E+10 + 2.19E+06 + Std 5.73E+04 1.46E+07 2.32E+06 5.65E+07 5.77E+05	6.42E+03 6.87E+03 1.15E+05 1.74E+04 1.19E+02 6.27E+01 1.03E+02 1.95E+01 5.50E-02 6.02E-02 1.78E+02 2.01E+01 1.06E+02 1.59E+01 2.96E+02 2.68E+02 4.50E+03 2.69E+02 7.12E+01 2.33E+01 4.26E+05 2.87E+05 8.38E+03 9.85E+03 4.45E+03 3.95E+03 5.38E+03 5.54E+03 1.02E+03 2.46E+02 6.05E+02 1.89E+02 9.10E+04 6.00E+04 1.52E+04 1.27E+04 5.65E+02 2.45E+02 3.09E+02 1.47E+01 5.17E+03 4.45E+02 5.43E+02 1.83E+01 6.16E+02 1.98E+01 5.45E+02 3.44E+01 2.34E+03 1.88E+02 6.68E+02 6.80E+01 5.06E+02 2.34E+01 6.62E+02 1.64E+02 8.47E+05 2.54E+05
	Total	+/=/-	15/0/14	23/0/6	25/0/4	29/0/0	23/0/6

Table 5 .

 5 13: Effectiveness of each modification of FOABC on benchmarks with D = 50Secondly, in order to test the effectiveness of the embedding of FOC, the variant FOABC F O is compared with ABC algorithm. It can be found that, FOABC F O achieves better results on 22 benchmarks. Hence, the algorithm's performance can be improved by using FOC in the onlooker bee phase. Furthermore, these two versions are compared with the final FOABC algorithm as well. From the Wilcoxon test results, these two variants can be found to be better than the basic ABC but less effective than the final version we proposed.FOABC DE is unable to outperform FOABC on 17 functions, while FOABC F O also fails on 19 benchmarks. And a similar conclusion can be drawn from the5.3 Experiments on function optimization problemsnumber of best results marked in boldface.According to the average rankings given by the Friedman test, the final FOABC obtains the best ranking. Meanwhile, the rankings of FOABC DE and FOABC DE are both better than those of ABC.

	Functions	ABC	FOABC DE	FOABC F O	FOABC r=12,q=0.8
		Mean	Std	Mean	Std	Mean	Std	Mean	Std
	f 1	1.56E+03 1.10E+03 6.67E+03	7.10E+03	1.51E+04	4.35E+03	6.42E+03	6.87E+03
	f 3	2.14E+05	3.60E+04 1.12E+05 2.20E+04 2.12E+05	2.93E+04	1.15E+05	1.74E+04
	f 4	4.20E+01	1.51E+01	9.83E+01	5.59E+01 3.37E+01 1.22E+01 1.19E+02	6.27E+01
	f 5	1.98E+02	1.95E+01	1.10E+02	1.58E+01	1.80E+02	1.72E+01 1.03E+02 1.95E+01
	f 6	7.59E-10	5.36E-10	2.84E-01	3.44E-01	2.66E-04	4.96E-05	5.50E-02	6.02E-02
	f 7	2.11E+02	1.60E+01	1.96E+02	3.67E+01	1.98E+02	1.51E+01 1.78E+02 2.01E+01
	f 8	2.04E+02	2.13E+01	1.07E+02	1.92E+01	1.84E+02	2.19E+01 1.06E+02 1.59E+01
	f 9	5.26E+03	1.24E+03	4.35E+02	4.43E+02	4.52E+03	1.63E+03 2.96E+02 2.68E+02
	f 10	4.09E+03	3.99E+02	4.59E+03	3.63E+02 4.01E+03 2.99E+02 4.50E+03	2.69E+02
	f 11	7.92E+02	9.93E+02	7.78E+01	2.29E+01	5.63E+02	4.40E+02 7.12E+01 2.33E+01
	f 12	3.58E+06	1.23E+06 1.77E+05 1.30E+05 3.48E+06	1.34E+06	4.26E+05	2.87E+05
	f 13	5.21E+03 3.77E+03 6.45E+03	8.09E+03	1.14E+04	4.68E+03	8.38E+03	9.85E+03
	f 14	6.81E+05	4.74E+05 3.40E+03 3.75E+03 5.98E+05	3.71E+05	4.45E+03	3.95E+03
	f 15	7.03E+03	4.93E+03	8.10E+03	9.14E+03	9.59E+03	4.67E+03 5.38E+03 5.54E+03
	f 16	1.28E+03	1.77E+02	1.06E+03	2.89E+02	1.22E+03	2.77E+02 1.02E+03 2.46E+02
	f 17	8.82E+02	1.65E+02	6.92E+02	2.14E+02	8.01E+02	1.51E+02 6.05E+02 1.89E+02
	f 18	1.01E+06	5.77E+05	9.84E+04	7.13E+04	9.20E+05	4.85E+05 9.10E+04 6.00E+04
	f 19	8.90E+03 4.35E+03 9.54E+03	9.91E+03	9.53E+03	4.06E+03	1.52E+04	1.27E+04
	f 20	7.44E+02	1.27E+02 5.64E+02 2.15E+02 7.59E+02	1.45E+02	5.65E+02	2.45E+02
	f 21	4.05E+02	4.75E+01 3.08E+02 1.73E+01 3.98E+02	1.98E+01	3.09E+02	1.47E+01
	f 22	5.19E+03	3.68E+02	5.17E+03	1.03E+03 4.86E+03 9.54E+02 5.17E+03	4.45E+02
	f 23	6.53E+02	5.07E+01	5.46E+02	2.79E+01	6.46E+02	3.77E+01 5.43E+02 1.83E+01
	f 24	1.02E+03	5.98E+01	6.19E+02	2.85E+01	9.83E+02	5.72E+01 6.16E+02 1.98E+01
	f 25	5.12E+02	1.84E+01	5.43E+02	3.67E+01 5.07E+02 1.73E+01 5.45E+02	3.44E+01
	f 26	1.39E+03 1.47E+03 2.44E+03	2.12E+02	1.73E+03	1.50E+03	2.34E+03	1.88E+02
	f 27	6.56E+02	2.16E+01	6.82E+02	7.66E+01 6.45E+02 3.21E+01 6.68E+02	6.80E+01
	f 28	4.87E+02	1.29E+01	5.03E+02	1.25E+01 4.82E+02 1.33E+01 5.06E+02	2.34E+01
	f 29	1.08E+03	1.49E+02	7.53E+02	1.59E+02	9.91E+02	1.25E+02 6.62E+02 1.64E+02
	f 30	7.51E+05	6.66E+04	7.14E+05	1.13E+05 7.14E+05 4.57E+04 8.47E+05	2.54E+05
	Wilcoxon (+/=/-)
	v.s. ABC			18/0/11		22/0/7
	v.s. FOABC			12/0/17		10/0/19
	Friedman	3		2.31	2.55	2.14

  algorithm, a new fractionalorder ABC algorithm (FOABC for short) is proposed in this chapter. Firstly, a DE-based search strategy is used in the employed bee phase to keep a nice balance between the exploration and exploitation. Secondly, in order to improve the local search ability, the FOC is incorporated into the search strategy of onlooker bees. In this way, the memory feature of FOC enriches the amount of available information for onlookers. Precisely, the information of the last several steps is considered when generating new solutions. Moreover, the random number used in the search equations is drawn from Lévy distribution in order to increase the randomness of the searching process.Experiments are carried out to study the sensitivity of FOABC with respect to the parameters of FOC. Groups of experiments are conducted on CEC 2017 benchmark functions to evaluate the performance of FOABC from different perspectives. In the first two groups, FOABC is compared with six ABC algorithms and five other enhanced meta-heuristics, respectively. According to the results, One of the most popular problems in domains linked to robotics is robot path planning (RPP). It has attracted considerable attention since path planning issues have widely existed for different types of robots, including industrial, mobile, etc. RPP problem aims at finding out an optimal collision-free path from the start point to the target point in an environment with obstacles (Fragapane et al., 2021; Nazarahari et al., 2019). And a series of optimization problems can be formulated by considering different goals and constraints. For instance, energy cost (or fuel cost) is one of the essential terms that needs to be minimized.

	6.1 Introduction
	After investigating different improvement strategies to enhance the effectiveness
	of the ABC algorithm in the previous chapters, applying them to some more
	meaningful practical problems is to be expected. At the same time, observing the
	importance and booming development of robotics-related fields, certain problems
	in this area are selected to be investigated in this chapter.
	As mentioned in the first chapter, path planning algorithms can be generally
	classified into three categories: classical, graph-based, and (meta-)heuristic ap-
	FOABC outperforms the other ABC algorithms in terms of solution accuracy proaches (Koubâa et al., 2018). The classical methods like the artificial potential
	and robustness. Meanwhile, it also performs outstandingly in the second group fields method were popular in the period when the path planning problem was
	of comparisons. Thirdly, redundancy elimination experiments are done to verify just appearing. They were found to be effective in finding feasible paths, yet the
	the effectiveness of each proposed strategy. As a result, FOABC has excellent shortcomings became increasingly evident. One of the widely known limitations
	performance in handling numerical optimization problems. is that they are time-consuming, as the generated solutions are computationally
	expensive. Besides, these kinds of methods might fall into local optima (Das

& Jena, 2020;

[START_REF] Koubâa | Robot path planning and cooperation: foundations[END_REF]

. As for the second category, graph-based search methods are also popular, such as the A* algorithm

[START_REF] Hart | A formal basis for the heuristic determination of minimum cost paths[END_REF]

Table 6 .

 6 Besides, it is interesting to see that as the complexity rises, the performance of the iff -ABC algorithm becomes more noticeable. Particularly in Map 6, it outperforms the ABCL and ILTD_ABC in terms of average path length. However, it should be noted that the average running time of the iff -ABC algorithm is not as impressive as the aforementioned three ABC methods.

		2: (continued ) Comparison of 15 ABC algorithms for solving SRPP prob-
	lems
		ABC algorithms Avg. path length	Std.	Best path Worst path Avg. time
		ABC	1374.173	751.638 575.924	3825.581	17.695
		NSABC	795.414	214.625 536.574	1339.219	16.123
		iff -ABC	537.987	8.294	525.905	558.539	13.751
		MGABC	562.355	94.386	527.268	1033.116	13.660
		ILTD_ABC sdABC	526.093 557.500	1.150 13.579	524.806 532.505	530.745 590.765	11.228 15.461
		DEABC	819.613	32.598	749.023	877.275	21.789
	Map 4	APABC ARABC	1629.682 544.193	318.632 1160.452 9.119 528.528	2502.570 570.100	11.982 13.914
		GABC	540.554	6.945	527.041	554.018	14.092
		DABC	537.939	3.625	534.510	551.403	15.203
		ABCL ABCDC	537.731 554.008	8.400 7.225	525.540 540.861	555.251 569.026	10.342 12.316
		ABC_RL	538.458	9.590	525.358	557.892	12.909
		FOABC	525.535	1.725	524.716	532.401	11.101
		ABC	1499.821	569.741 759.605	2983.055	16.025
		NSABC	830.759	158.392 638.792	1311.277	15.863
		iff -ABC	620.451	0.796	619.283	622.048	13.533
		MGABC	633.949	53.919	619.095	873.485	15.294
		ILTD_ABC sdABC	619.212 687.238	0.099 41.668	619.044 633.728	619.451 788.478	11.668 19.736
		DEABC	731.605	24.496	662.920	774.724	26.610
	Map 5	APABC ARABC	1419.491 623.769	190.415 1035.945 7.135 619.565	1704.878 659.830	16.099 13.380
		GABC	620.706	0.923	619.272	622.854	13.480
		DABC	621.505	2.164	619.340	631.606	16.790
		ABCL ABCDC	620.701 648.908	1.551 6.538	619.175 632.713	626.336 666.528	11.186 15.344
		ABC_RL	625.622	25.916	619.162	762.696	13.634
		FOABC	619.142	0.137	619.013	619.710	12.364
		ABC	1563.006	781.150 651.615	3581.080	15.364
		NSABC	727.292	152.099 561.464	1076.331	14.519
		iff -ABC	539.038	0.283	538.668	539.683	11.914
		MGABC	772.555	283.193 544.013	1509.287	17.931
		ILTD_ABC	539.054	0.364	538.633	540.301	10.081
		sdABC	668.048	64.326	570.736	836.237	18.051
		DEABC	714.867	27.531	648.334	772.856	24.597
	Map 6	APABC ARABC	1300.744 539.095	198.569 1003.654 0.338 538.560	1738.505 540.087	14.852 12.548
		GABC	539.497	0.630	538.828	541.467	13.108
		DABC	539.998	0.747	538.694	542.101	15.038
		ABCL ABCDC	539.069 594.741	0.268 11.874	538.572 575.052	539.707 630.126	9.230 12.512
		ABC_RL	539.195	0.565	538.552	540.969	11.913
		FOABC	538.723	0.315	538.459	539.672	10.954

Table 6 .

 6 3. The best values are marked in boldface. Furthermore, the shortest paths planned by the FOABC algorithm and the four well-known path planning approaches are drawn in Figure6.7.From Table6.3, it is encouraging to find that our proposed approaches, especially the FOABC algorithm, are very comparable to those well-known methods.In other words, the performance of proposed algorithms in searching for the optimal collision-free path has been well demonstrated. In terms of average path length, all approaches outperform the other four path planners in Maps 2, 3, and 5. And the average path length obtained by FOABC is the shortest in 5 out of 6 workspaces. As for Maps 1 and 6, excluding ABCDC, the other three improved ABC algorithms are also outstanding. Be aware that ABCDC performs slightly worse than PRM in Map 1 and A* in Map 6, respectively. A*, a kind of exact path planning method, is able to find a competitive path among the approaches. It is worth pointing out that it achieves the shortest average path length in Map 4. Nonetheless, A* is significantly time-consuming as it generates computationally expensive solutions. PRM is also comparative in terms of path length and running time. Its execution time is the least among the concerned methods. Additionally, because the degrees of complexity of concerned maps are different, the running times of RRT, BRRT, and A* have been significantly impacted. Meanwhile, it can be found that the increase in difficulty can hardly affect the running time of our proposed ABC algorithms as well as PRM. Therefore, it can be concluded that the proposed algorithms are effective methods for the SRPP problem considering their superiorities in the path length and robustness.

	6.2 Single robot path planning (SRPP)

Table 6 .

 6 4: Comparison of average required steps and average path lengths for six robots

				Average required steps		Average path length
		Robot ABC GABC DABC ABCL	ABC	GABC DABC	ABCL
	Map 1	r1	6.73	6.93	6.87	6.67	464.57 468.06 467.51	461.90
		r2	5.00	5.00	5.00	5.00	240.18 239.33 240.20	238.61
		r3	4.00	4.00	4.00	4.00	203.40 205.08 201.68	202.66
		r4	8.00	8.00	8.00	8.00	363.98 363.70 361.82	362.16
		r5	6.20	5.53	6.20	6.07	292.98 286.49 288.53	287.93
		r6	6.07	6.00	6.00	6.00	228.81 229.66 229.58	228.65
		Total 36.00 35.47 36.07	35.73 1793.91 1792.32 1789.31 1781.92
	Map 2	r1	3.00	3.00	3.00	3.00	185.32 187.31 186.51	185.57
		r2	5.20	5.00	5.00	5.00	338.10 346.47 343.84	347.17
		r3	8.33	12.87	7.60	8.00	286.56 365.90 283.34	285.07
		r4	7.20	6.93	8.07	5.00	293.82 292.56 309.29	277.27
		r5	4.00	4.00	4.00	4.00	199.07 201.46 199.90	199.72
		r6	3.00	3.13	3.20	3.00	69.15	77.11	65.34	69.31
		Total 30.73 34.93	30.87 28.00 1372.02 1470.82 1388.22 1364.11
	Map 3	r1	4.53	5.00	4.20	4.53	243.48 254.73 242.15	242.77
		r2	5.00	5.00	5.00	5.00	335.59 336.74 337.04	336.46
		r3	12.40 10.67	12.53	9.93	416.57 413.74 449.06	396.29
		r4	4.67	4.27	4.60	4.40	283.42 282.21 285.77	279.95
		r5	6.53	6.60	6.67	6.67	319.28 318.54 318.96	318.85
		r6	4.00	5.20	4.00	4.00	117.64 143.27 115.26	116.77
		Total 37.13 36.73	37.00 34.53 1715.98 1749.23 1748.25 1691.08
	Map 4	r1	5.00	5.00	5.07	5.00	306.84 307.88 305.76	305.80
		r2	7.00	6.93	7.00	6.73	214.65 212.69 218.25	215.38
		r3	4.00	4.00	4.00	4.00	287.76 287.27 286.36	286.94
		r4	4.07	4.07	4.20	4.00	198.51 194.42 203.83	193.94
		r5	11.20 10.67	9.87	9.73	325.53 327.65 316.88	300.90
		r6	3.00	3.00	3.00	3.00	223.99 225.86 225.30	223.31
		Total 34.27 33.67	33.13 32.47 1557.28 1555.75 1556.38 1526.27
	Map 5	r1	8.67	6.60	8.87	9.40	308.948 263.986 308.93 307.224
		r2	7.53	7.40	6.67	6.93	398.236 408.086 400.047 399.518
		r3	6.27	6.60	6.00	6.40	223.052 215.56 211.214 209.014
		r4	6.00	6.00	6.00	6.07	367.657 368.641 367.339 368.188
		r5	5.07	5.13	5.13	5.20	208.825 202.747 217.093 209.991
		r6	3.00	9.47	3.00	3.00	89.4593 215.795 87.761 84.1803
		Total 36.53 41.20 35.67 37.00 1596.18 1674.81 1592.38 1578.11
	Map 6	r1	6.67	7.20	6.33	7.27	468.25 478.65 460.41	471.77
		r2	4.00	4.00	4.00	4.00	281.43 278.02 279.16	279.92
		r3	12.87 16.93	13.67	13.13	340.01 386.93 358.04	320.07
		r4	10.60 11.60	12.80	8.93	316.46 315.82 357.54	265.03
		r5	6.13	7.93	5.33	5.93	281.86 302.74 271.76	272.44
		r6	5.00	5.00	5.00	5.00	356.11 355.46 356.46	356.30
		Total 45.27 52.67	47.13 44.27 2044.13 2117.62 2083.37 1965.52

Table 6 .

 6 5: Comparison of average required steps, average path lengths and running time for twelve robots

				Average required steps			Average path length
		Robot	ABC	GABC DABC ABCL	ABC	GABC DABC	ABCL
	Map 1	r1	6.73	5.40	6.07	6.33	367.20 368.08 364.37	368.60
		r2	7.33	6.07	6.27	6.80	279.54 253.97 258.85	257.79
		r3	5.40	5.20	5.60	5.73	244.82 249.06 249.16	248.31
		r4	6.80	6.87	6.87	6.33	279.67 279.11 277.13	270.77
		r5	4.67	4.07	4.40	5.13	261.44 265.09 261.88	270.40
		r6	7.13	5.67	6.80	6.60	334.41 326.76 329.10	328.00
		r7	4.87	4.07	4.67	4.60	154.24 142.09 152.68	139.62
		r8	5.20	4.07	4.80	4.40	278.54 274.67 281.49	267.90
		r9	4.47	3.87	4.27	4.40	157.61 152.90 152.63	155.05
		r10	5.33	4.33	5.40	5.13	213.15 216.24 221.94	216.98
		r11	6.27	4.93	5.80	5.73	278.62 274.98 275.79	276.54
		r12	5.00	4.60	4.60	4.87	288.74 289.69 287.10	287.09
		Total	69.20	59.13	65.53	66.07 3137.99 3092.66 3112.11 3087.05
			Average running time		47.24	58.91	49.33	43.35
	Map 2	r1	5.20	4.53	4.87	5.00	292.31 283.73 284.43	280.90
		r2	8.47	6.53	8.00	7.93	363.06 353.66 369.01	354.01
		r3	13.53	14.47	10.80	12.27	341.81 351.09 336.03	330.28
		r4	9.33	6.67	9.00	5.27	334.70 298.18 334.43	260.97
		r5	12.00	12.73	10.00	10.13	348.23 331.10 319.22	320.92
		r6	6.27	4.73	6.00	6.07	227.25 207.85 222.06	216.36
		r7	4.40	4.00	4.40	4.53	252.04 254.15 255.61	251.67
		r8	5.87	4.00	5.47	5.47	228.89 209.06 233.64	227.60
		r9	4.33	3.80	4.40	4.47	137.03 134.03 131.65	137.00
		r10	4.93	4.20	4.87	5.47	234.02 234.55 233.24	240.54
		r11	5.27	4.67	5.47	5.47	181.16 172.14 168.68	169.73
		r12	5.20	3.73	5.07	4.53	210.81 197.01 216.47	205.58
		Total	84.80	74.07	78.33	76.60 3151.31 3026.55 3104.46 2995.57
			Average running time		41.84	58.03	45.81	39.20

(a) f 4 (b) f 9 (c) f 10 (d) f 11 (e) f 17 (f ) f 20

(a) f 3 (b) f 8 (c) f 10 (d) f 12 (e) f 18 (f ) f 22

(a) f 3 (b) f 8 (c) f 10 (d) f 12 (e) f 18 (f ) f 22 Fig. 3.6. The influence of RL method on convergence rate of ABC algorithm

(a) Initial state (b) Step 1

Acknowledgements ACKNOWLEDGEMENTS a few. And special thanks to the staff at Centrale Lille and CRIStAL, Vanessa FLEURY, Bénédicte FIEVET, Dominique DEREMETZ, etc. for their kind help. I also could not have undertaken this journey without the support from the CSC (China Scholarship Council), who financed my research work.

Finally, and most importantly, I want to thank my dear family and friends for their encouragement, support and love, especially my parents. The trust from all these lovely people has kept my spirits and motivation high throughout this process.

Lille, France

Yibing CUI September, 2022 RESUME ETENDU d'exploration est principalement accomplie par les abeilles employées tandis que les abeilles spectatrices sont chargés d'exploiter dans certaines régions.

En plus de ces stratégies d'amélioration largement mentionnées, l'impact de la composition de la population est étudié dans le chapitre 2. En effet, la composition invariable de la population de la colonie d'abeilles ne peut pas satisfaire les besoins des différentes étapes de recherche. Dans ce contexte, l'amélioration de l'efficacité de l'ABC en ajustant la composition de la population est développée. Ainsi, un algorithme ABC avec composition dynamique de la population, à savoir ABCDC, est proposé. Les principales contributions de ce chapitre sont les suivantes : premièrement, le Symmetric Latin Hypercube Design (SLHD) est adoptée dans l'initialisation pour améliorer la diversité de la population. Ensuite, un nouveau mécanisme est proposé pour ajuster la composition de la population en fonction des expériences de recherche. Plus précisément, le nombre d'abeilles employées diminue périodiquement tandis que la taille des abeilles spectatrices augmente afin d'apporter plus d'énergie pour exploiter l'optimum global au stade moyen-tardif du processus de recherche. Dans l'ABCDC, la division du travail entre les différentes abeilles est plus claire, de sorte que l'optimum global peut être obtenu plus efficacement grâce à leur coopération. De plus, l'ABCDC maintient un bon équilibre entre la diversification et l'intensification. Des études expérimentales sur des problèmes d'optimisation fonctionnelle sont réalisées pour vérifier la performance d'ABCDC. Les comparaisons montrent que l'ABCDC a une meilleure précision de solution et un taux de convergence plus rapide. En fait, il est difficile de définir les valeurs des paramètres de contrôle de manière appropriée pour tous les types de problèmes. Ainsi, ces paramètres de contrôle sont généralement constants ou mis à jour avec des méthodes d'adaptation prédéterminées, comme celle adoptée dans l'ABCDC. Cependant, les démarches d'adaptation reposent encore largement sur l'expérience du concepteur. Dans ce contexte, différent de la littérature existante, une nouvelle façon de définir les valeurs des paramètres est proposée dans le chapitre 3. Un algorithme ABC basé sur l'apprentissage par renforcement (RL) est proposé, nommé ABC_RL. La méthode RL est utilisée pour faire varier le nombre de dimensions à mettre à jour dans l'équation de recherche de solution. La valeur de récompense de RL est définie en fonction des résultats de la mise à jour. Dans ce cas, plus d'informations

LEARNING BASED ABC ALGORITHM (ABCL)

the competitors. Since the proposed algorithm performs well in solving both unimodal and multi-modal functions, it keeps a nice balance between exploitation and exploration during the search process. The second best algorithm is given to GABC and followed by the DABC. Besides, it is normal that all the improved ABC variants achieve better performances than the basic ABC does. Therefore, the search effectiveness of ABCL can be proved by solving different types of problems.

Convergence behavior analysis

The convergence curves of compared algorithms were plotted in order to further compare their performance. The convergence process of solving different types of problems with D = 30 can be observed in Figure 4.3. Note that the values of log(f (•)) are presented because the objective function values are too large in the earlier stages of the searching process.

In Figure 4.3, (a) and (b) present the convergence curves of ABC algorithms on uni-modal functions f 1 and f 4 , respectively. Since the results of solving unimodal functions can indicate the exploitation ability of an algorithm, it can be found that ABCL's exploitation ability has been remarkably enhanced. And the proposed algorithm is able to achieve the best accuracy. Moreover, subfigures (c) and (d) show the cases of solving multi-modal problems f 6 and f 9 . A similar conclusion can be derived from these cases, where ABCL converges rapidly to the smallest value among the concerned methods. In this case, the diversification of ABCL is also outstanding compared to the other three methods. At the same time, the convergence rate of the GABC algorithm is competitive, which is the second best in the comparisons. Furthermore, the advantages in solution accuracy of ABCL are significant as shown in the figures. This indicates that it has the ability to find a better solution in a limited number of iterations.

size keeps the same. Accordingly, FOABC loses a bit of superiority in solving the uni-modal functions. Nevertheless, the performance of FOABC on multimodal functions becomes better as it achieves the best results on 2 functions.

As for solving the hybrid functions, FOABC still possesses significant advantages compared to the other methods. The comparison when D = 10 is presented in The comparison results of D = 30 and 50 are shown in Table 5.11 -Table 5.12, respectively. In Table 5.11, the advantages of algorithms CLPSO and FOABC are evident. CLPSO obtains the smallest errors on 9 functions while FOABC has RPP problems can be solved by transforming them into functional optimization problems. Then, the optimal solution can be found by using meta-heuristic algorithms, which is effective and powerful even in complex environments. In recent The 15 ABC algorithms have all been independently run 30 times for each map. The mean and standard deviation of the path lengths are calculated and presented in Table 6.1 and Table 6.2 (columns Avg. path length and Std., respectively). The shortest path and the longest path are also listed in the tables.

Moreover, in order to make a meaningful comparison, the average running time 6.2 Single robot path planning (SRPP) Based on the comparison in Table 6.5, the MRPP missions of twelve robots are accomplished by all the involved algorithms without collisions. And ABCL evidently outperforms the other methods in terms of average path lengths as well as execution time. Meanwhile, the steps required to reach the predefined goals via GABC are a bit smaller than the other competitors. It can be found that the proposed ABCL algorithm enables the robots to avoid long detours by taking more small steps in some complex situations. In this way, the average path lengths planned by ABCL are the shortest in comparison while it requires a bit more steps to achieve the targets compared with GABC. Besides, the average running time can be found in the last row of each case in the table. Although the execution times of all the algorithms do not have a large difference, ABCL is still the most efficient. Therefore, its effectiveness can be further proved through these complex tasks MRPP.

CONCLUSIONS AND PERSPECTIVES

In Chapter 3, a particular improvement strategy is investigated considering that it is difficult to define the parameter value appropriately for all kinds of problems. In fact, the control parameters are usually constant or updated with predetermined adaptation methods, which heavily rely on the designer's experience. In this context, a RL-based ABC algorithm is proposed, named ABC_RL.

The RL method is utilized to vary a newly introduced control parameter, i.e., number of dimensions to be updated in the solution search equation. The reward value of RL is defined based on the update results. In this case, more information can be learned appropriately from the previous experience. To sum up, the main contributions can be summarized as follows: firstly, RL is adopted to enlarge and adjust the frequency of perturbation of employed bee phase intelligently. Thirdly, in Chapter 4, enhancing the performance of ABC while avoiding it becoming too complex is studied. Hence, a learning-based ABC (ABCL) algorithm is proposed. Since one of the most essential goals of improving such algorithms is to solve more practical problems, then its practicality and complexity must be considered. Therefore, to improve the performance of algorithm without overcomplicating it, the following strategies are utilized: firstly, the global best solution is adopted in the employed bee and scout bee phases to better guide the colony. Secondly, the learning phase of the TLBO algorithm is embedded in the onlooker bee phase to improve the exploitation ability and simplify the computational complexity. Furthermore, comparisons with other ABC variants are carried out on functional optimization problems in different dimensions cases.

And the effectiveness of ABCL can be validated according to the results.

In Chapter 5, different from the proposed ABC variants in previous chapters, the FOC is incorporated into the ABC algorithm considering the memory properties of FOC. In the proposed FOABC algorithm, each time generating a new candidate solution, the previous foraging behaviors stored in memory are referred