Abstract
Nature-inspired algorithms attract many researchers worldwide for solving the hardest optimization problems. One of the newest members of this extensive family is the bat algorithm. To date, many variants of this algorithm have emerged for solving continuous as well as combinatorial problems. One of the more promising variants, a self-adaptive bat algorithm, has recently been proposed that enables a self-adaptation of its control parameters. In this paper, we have hybridized this algorithm using different DE strategies and applied these as a local search heuristics for improving the current best solution directing the swarm of a solution towards the better regions within a search space. The results of exhaustive experiments were promising and have encouraged us to invest more efforts into developing in this direction.
1. Introduction
Optimization has become more and more important, especially during these times of recession. It has been established throughout practically all spheres of human activities, for example, finances, industries, sport, pharmacy, and so forth. In each of these spheres, a goal of the optimization is to find the optimal input parameters according to known outcomes and a known model. This model represents a problem to be solved and transforms input parameters to output outcomes. Essentially, the optimal input parameters must be properly evaluated in order to determine the quality of a solution. Indeed, an objective function is used that mathematically describes this quality. In the praxis, the value of the objective function can be either minimized or maximized. For example, when buying a car either the minimum cost or maximum comfort is interesting objective for a potential buyer, where denotes the objective function. Using the equation , the maximization problem can be transformed into a minimization one, and vice versa. The function expressed in this way is also named as fitness function in evolutionary computation community. In place of the objective function, the minimization of the fitness function is assumed in this paper.
Most of the problems arising in practice today are NP-hard. This means that the time complexity of an exhaustive search algorithm running on a digital computer which checks all solutions within a search space increases exponentially by increasing the instance size as determined by the number of input parameters. In some situations, when the number of input parameters increases to a certain limit, it can be expected that a user never obtains the results from the optimization. As a result, algorithms solving the NP-problems approximately have arisen in the past. Although these algorithms do not find the exact optimal solution, in general, their solutions are good enough in practice. For instance, the well-known algorithms for approximately solving the hardest problems today are(i)artificial bee colony (ABC) [1],(ii)bat algorithm (BA) [2],(iii)cuckoo search (CS) [3],(iv)differential evolution (DE) [4],(v)firefly algorithm (FA) [5, 6],(vi)particle swarm optimization (PSO) [7],(vii)many more [8].
The developers of the above-mentioned algorithms were typically inspired by nature in a development phase. Namely, each nature-inspired algorithm mimics the natural behaviors of specific biological, chemical, or physical systems in order to solve particular problems using a digital computer. Most of the inspiration from nature emanates from biology. The greatest lasting impression on the developers has been left by Darwin’s evolution [9]. Darwin observed that nature is not static and therefore the fitter individuals have more chances of surviving within the changing environments. Holland in [10] connected this adaptive behavior of the natural systems to artificial systems that simulate their behavior by solving the particular problems (also optimization problems) on the digital computers.
Optimization algorithms are controlled by algorithm parameters that can be changed deterministically, adaptively, and self-adaptively [11]. Deterministic parameters are altered by using some deterministic rules (e.g., Rechenberg’s 1/5 success rule [12]). In contrast, adaptively controlled parameters are subject to feedback from the search process that serves as an input to the mechanism used which determines the direction and magnitude of the change [11]. Finally, the self-adaptively controlled parameters are encoded into a representation of the solution and undergo the actions of variation operators [13].
This paper focuses on the adaptation and hybridization of a swarm intelligence algorithm. Swarm intelligence (SI) belongs to an artificial intelligence discipline (AI) which first became popular over the last decade and still is [14]. It is inspired by the collective behavior of social swarms of ants, termites, bees, and worms, flock of birds, and schools of fish [15]. Although these swarms consist of relatively unsophisticated individuals, they exhibit coordinated behavior that directs the swarms towards their desired goals. This usually results in the self-organizing behavior of the whole system, and collective intelligence or swarm intelligence is in essence the self-organization of such multiagent systems, based on simple interaction rules.
The bat algorithm (BA) is one of the younger members of this family which was developed by Yang [16]. It is inspired by the microbats that use an echolocation for orientation and prey seeking. The original bat algorithm employs two strategy parameters: the pulse rate and the loudness. The former regulates an improvement of the best solution, while the latter influences an acceptance of the best solution. Both mentioned parameters are fixed during the execution of the original bat algorithm. In the self-adaptive bat algorithm (SABA) developed by Fister et al. [17] these parameters are self-adaptive. The aim of this self-adaptation is twofold. On the one hand, it is very difficult to guess the valid value of the parameter at the beginning of the algorithm. On the other hand, this value depends on the phase in which the search process is. This means that the parameter setting at the beginning of the search process can be changed when this process reaches maturity. In this paper, a further step forward has been taken.
Hybridization with local search heuristics has now been applied in order to further improve the results of the SABA algorithm. Domain-specific knowledge can be incorporated using the local search. Although the local search is as yet an ingredient of the original bat algorithm it can be replaced by different DE strategies [18]. Indeed, the further improvement of the current best solution is expected which would direct the swarm intelligence search towards the more promising areas of the search space. As a result, the hybrid self-adaptive bat algorithm (HSABA) that was applied to a benchmark suite consisting of ten well-known functions from the literature was developed. The results of HSABA obtained during extensive experiments showed that the HSABA improved the results of both the original BA and the SABA. Moreover, the results were also comparable with the other well-known algorithms, like firefly (FA) [5], differential evolution (DE) [19], and artificial bee colony (ABC) [20].
The structure of this paper is as follows. Section 2 presents an evolution of the bat algorithms from the original BA via the self-adaptive SABA to the hybrid self-adaptive HSABA algorithm. Section 3 describes the experimental work, while Section 4 illustrates the obtained results in detail. The paper concludes by summarizing the performed work and outlines directions for the further development.
2. Evolution of Bat Algorithms
The results of experiments regarding the original bat algorithm showed that this algorithm is efficient especially when optimizing the problems of lower dimensions. In line with this, Eiben and Smith in [11] asserted that 2-dimensional functions are not suitable for solving with the population-based stochastic algorithms (like evolutionary algorithms and swarm intelligence), because these can be solved optimally using traditional methods. On the other hand, these kinds of algorithms could play a role as general problem solvers, because they share the same performances when averaged over all the discrete problems. This fact is the essence of the so-called No-Free Lunch (NFL) theorem [21]. In order for this theorem to prevail, there are almost two typical mechanisms for improving the performance of the population-based algorithms as follows:(i)self-adaptation of control parameters,(ii)hybridization.The former enables the control parameters to be changed during the search process in order to better suit the exploration and exploitation components of this search process [22], while the latter incorporates the problem-specific knowledge within it.
In the rest of this paper, firstly the original BA algorithm is presented in detail, followed by describing the application of a self-adaptive mechanism within the original BA algorithm which leads to the emergence of a self-adaptive BA algorithm SABA. Finally, a hybridization of the SABA algorithm is broadly discussed which obtains a hybridized SABA algorithm named the HSABA.
2.1. The Original Bat Algorithm
The original BA is a population-based algorithm, where each particle within the bat population represents the candidate solution. The candidate solutions are represented as vectors for with real-valued elements , where each element is drawn from interval . Thus, and denote the corresponding lower and upper bounds, and determines a population size.
This algorithm consists of the following main components:(i)an initialization,(ii)a variation operation,(iii)a local search,(iv)an evaluation of a solution,(v)a replacement.
In the initialization, the algorithm parameters are initialized, then, the initial population is generated randomly, next, this population is evaluated, and finally, the best solution in this initial population is determined. The variation operator moves the virtual bats in the search space according to the physical rules of bat echolocation. In the local search, the current best solution is improved by the random walk direct exploitation heuristics (RWDE) [23]. The quality of a solution is determined during the evaluation of solution. The replacement replaces the current solution with the newly generated solution regarding the some probability. This component is similar to the simulated annealing [24], where the new solution is accepted by the acceptance probability function which simulates the physical rules of annealing. The pseudocode of this algorithm is presented in Algorithm 1.
| ||||||||||||||||||||||||||||||||||||||
In Algorithm 1 the particular component of the BA algorithm is denoted either by function name when it comprises one line or by a comment between two curly brackets when the components are written within a structured statement and/or it comprises more lines. In line with this, the initialization comprises lines 1–3 in Algorithm 1, the variation operation line 6 (function generate_new_solution), the local search lines 7–9, the evaluation of the solution (function evaluate_new_solution in line 10), and the replacement lines 12–14. In addition, the current best solution is determined in each generation (function find_best_solution in line 15).
The variation operation which is implemented in function generate_new_solution moves the virtual bats towards the best current bat’s position according to the following equations: where is a random number drawn from a Gaussian distribution with zero mean and a standard deviation of one. A RWDE heuristics [23] implemented in the function improve_the_best_solution modifies the current best solution according to the following equation: where denotes the random number drawn from a Gaussian distribution with zero mean and a standard deviation of one, being the scaling factor and the loudness.
A local search is launched with the probability of pulse rate . As already stated, the probability of accepting the new best solution in the component save_the_best_solution_conditionaly depends on loudness . Actually, the original BA algorithm is controlled by two algorithm parameters: the pulse rate and the loudness . Typically, the rate of pulse emission increases and the loudness decreases when the population draws nearer to the local optimum. Both characteristics imitate natural bats, where the rate of pulse emission increases and the loudness decreases when a bat finds a prey. Mathematically, these characteristics are captured using the following equations: where and are constants. Actually, the parameter controls the convergence rate of the bat algorithm and therefore plays a similar role as the cooling factor in the simulated annealing algorithm.
In summary, the original BA algorithm is based on a PSO algorithm [7] which is hybridized with RWDE and simulated annealing heuristics. The former represents the local search that directs the bat search process towards improving the best solution, while the latter takes care of the population diversity. In other words, the local search can be connected with exploitation, while simulated annealing uses the exploration component of the bat search process. The exploitation is controlled by the parameter and exploration by the parameter . As a result, the BA algorithm is able to explicitly control the exploration and exploitation components within its search process.
2.2. The Self-Adaptive Bat Algorithm
Almost two advantages can be expected when the self-adaptation of control parameters is applied to a population-based algorithm [11]:(i)control parameters need not be set before the algorithm’s run,(ii)the control parameters are adapted during the run to the fitness landscape defined by the positions of the candidate solutions within the search space and their corresponding fitness values [25].
Normally, the self-adaptation is realized by encoding the control parameters into representation of candidate solutions and letting them undergo an operation of the variation operators. In this way, the self-adaptation of the BA control parameters (the loudness and the pulse rate) is considered.
This means that the existing representation of the candidate solutions consisting of problem variables is widened by the control parameters and to where denotes the population size. The control parameters are modified according to the following equations:
Note that the parameters and denote the learning rates that were set, as , while for designate the randomly generated value from interval .
The self-adapting part of the SABA algorithm is performed by the generate_the_new_solution function (line 6 in Algorithm 1). Note that the control parameters are modified in this function according to the learning rates and . In the case , each 10th candidate solution is modified on average. The modified parameters influence the application of the local search and the probability of the replacement. The replacement function preserves the problem variables as well as the control parameters.
This self-adapting function was inspired by Brest et al. [4] who proposed the self-adaptive version of DE, better known as jDE. This self-adaptive algorithm improves the results of the original DE significantly by continuous optimization.
2.3. The Hybrid Self-Adaptive Bat Algorithm
The population-based algorithms, like evolutionary algorithms and swarm intelligence, can be seen as some kind of general problem solvers, because they are applicable for all classes of optimization problems. Thus, their results confirm the so-called No-Free Lunch (NFL) theorem by Wolpert and Macready [21]. According to this theorem any two optimization algorithms are equivalent when their performances are averaged across all possible problems.
In order to circumvent the NFL theorem by solving a specific problem, domain-specific knowledge must be incorporated within the algorithm for solving it. The domain-specific knowledge can be incorporated by the bat algorithm within each of its components, that is, an initialization, a variation operator, a local search, an evaluation function, and a replacement.
Although the SABA algorithm significantly outperformed the results of the original BA algorithm, it suffers from a lack of incorporated domain-specific knowledge of the problem to be solved. This paper focuses on hybridizing the SABA using a novel local search heuristics that better exploits the self-adaptation mechanism of this algorithm. The standard “rand/1/bin” DE strategy and three other DE strategies focusing on the improvement of the current best solution were used for this purpose, where the modification of the local search in a hybrid self-adaptive BA algorithm (HSABA) is illustrated in Algorithm 2.
| ||||||||||||||||||||||||
The local search is launched according to a threshold determined by the self-adapted pulse rate (line 1). This parameter is modified by each 10th virtual bat, on average. The local search is an implementation of the operators crossover and mutation borrowed from DE [19]. Firstly, the four virtual bats are selected randomly from the bat population (line 2) and the random position is chosen within the virtual bat (line 3). Then, the appropriate DE strategy modifies the trial solution (lines 4–9) as follows: where the DE_Strategy is launched according to the probability of crossover . The term ensures that almost one modification is performed on the trial solution. The DE_Strategy function is presented in Algorithm 3.
| ||||||||||||||||||||||||||||
Although more different DE strategies exist today, we have focused mainly on those that include the current best solution in the modification of the trial solution. These strategies are presented in Table 1. Note that the suitable DE strategy is selected using the global parameter strategy.
The strategies illustrated in the table that use the best solution in the modification operations, that is, “randToBest/1/bin,” “best/2/bin,” and “best/1/bin,” typically direct the virtual bats towards the current best solution. Thus, it is expected that the new best solution is found when the virtual bats move across the search space directed by the best solution. The “rand/1/bin” represents one of the most popular DE strategies today that introduces an additional randomness into a search process. Obviously, it was used also in this study.
3. Experiments
The goal of our experimental work was to show that the HSABA outperforms the results of the original BA as well as the self-adaptive SABA algorithms, on the one hand, and that these results were comparable with the results of other well-known algorithms, like firefly (FA) [5], differential evolution (DE) [19], and artificial bee colony (ABC) [20]. All the mentioned algorithms were applied to a function optimization that belongs to a class of combinatorial optimization problems.
Function optimization is formally defined as follows. Let us assume a fitness function , where denotes a vector of design variables from a decision space . The values of the design variables are drawn from the interval , where and are their corresponding lower and upper bounds, respectively. Then, the task of function optimization is to find the minimum of this objective function.
In the remainder of this paper, the benchmark function suite is presented, then the experimental setup is described, and finally, the configuration of the personal computer (PC) on which the tests were conducted is clarified in detail.
3.1. Benchmark Suite
The benchmark suite was composed of ten well-known functions selected from various publications. The reader is invited to check deeper details about test functions in the state-of-the-art reviews [26–28]. The definitions of the benchmark functions are summarized in Table 2 which consists of three columns denoting the function tag , the function name, and the function definition.
Each function from the table is tagged with its sequence number from to . Typically, the problem becomes heavier to solve when the dimensionality of the benchmark functions is increased. Therefore, benchmark functions of more dimensions needed to be optimized during the experimental work.
The properties of the benchmark functions can be seen in Table 3 which consists of five columns: the function tag , the value of the optimal solution , the optimal solution , the function characteristics, and domain. One of the more important characteristics of the functions is the number of local and global optima. According to this characteristic the functions are divided into either unimodal or multimodal. The former type of functions has only one global optimum, while the latter is able to have more local and global optima throughout the whole search space. Parameter domain limits the values of parameters to the interval between their lower and upper bounds. As a matter of fact, these bounds determine the size of the search space. In order to make the problems heavier to solve, the parameter domains were more widely selected in this paper than those prescribed in the standard publications.
3.2. Experimental Setup
The characteristics of the HSABA were observed during this experimental study. Then, the best parameter setting was determined. Finally, the results of the HSABA were compared with the results of the original BA and self-adaptive SABA algorithms. The results of the other well-known algorithms, like FA, DE, and ABC, were also added to this comparative study. Note that the best parameter settings for each particular algorithm were used in the tests, except BA algorithms, where the same setup was employed in order to make the comparison as fair as possible. All the parameter settings were found after extensive testing.
During the tests, the BA parameters were set as follows: the loudness , the pulse rate , minimum frequency , and maximum frequency . The same initial values for and were also applied by SABA and HSABA, while the frequency was captured from the same interval as by the original bat algorithm. Thus, the values for loudness are drawn from the interval and the pulse rate from the interval . The additional parameters controlling behavior of DE strategies were set as and . Both parameters had a great influence on the results of the HSABA algorithm, while this setting initiated the best performance by the self-adaptive SABA framework hybridized with DE strategies.
FA ran with the following set of parameters: , , and , while DE was configured as follows: the amplification factor of the difference vector and the crossover control parameter . In the ABC algorithm, the onlooker bees represented 50% of the whole colony, while the other 50% of the colony was reserved for the employed bees. On the other hand, the scout bee was generated when its value had not improved in 100 generations. In other words, the parameter was set to the value .
Each algorithm in the tests was run with a population size of 100. Each algorithm was launched 25 times. The obtained results of these algorithms were aggregated according to their best, the worst, the mean, the standard deviation, and the median values reached during 25 runs.
3.3. PC Configuration
All runs were made on HP Compaq with the following configurations.(1)Processor: Intel Core i7-2600 3.4 (3.8) GHz.(2)RAM: 4 GB DDR3.(3)Operating system: Linux Mint 12. Additionally, all the tested algorithms were implemented within the Eclipse Indigo CDT framework.
4. The Results
Our experimental work consisted of conducting the four experiments, in which the following characteristics of the HSABA were tested:(i)an influence of using the different DE strategies,(ii)an influence of the number of evaluations,(iii)an influence of the dimensionality of problems,(iv)a comparative study.
In the first test, searching was performed for the most appropriate DE strategy incorporated within the HSABA algorithm. This strategy was then used in all the other experiments that followed. The aim of the second experiment was to show how the results of the HSABA converge according to the number of fitness function evaluations. Dependence of the results on the dimensionality of the functions to be optimized is presented in the third experiment. Finally, the results of the HSABA were compared with the original BA and self-adaptive SABA algorithms as well as the other well-known algorithms, like FA, DE, and ABC.
In the remainder of this paper, the results of the experiments are presented in detail.
4.1. Influences of Using the Different DE Strategies
In this experiment, the qualities of four different DE strategies that were implemented within the HSABA algorithm were observed. The outcome of this experiment had an impact on the further tests, because all the tests that followed were performed with the most promising DE strategy found during the experiment. In line with this, each of the DE strategies, that is, “rand/1/bin,” “randToBest/1/bin,” “best/2/bin,” and “best/1/bin,” was tested by optimizing the benchmark function suite. The results of function optimization were averaged after 25 runs and are illustrated in Figure 1.
| (a) |
| (b) |
| (c) |
| (d) |
| (e) |
| (f) |
This figure is divided into six diagrams, two for each observed dimension of the function. Each diagram represents the progress of the optimization in percent on the -axis, where denotes the corresponding number of fitness function evaluations (e.g., 10,000 for 10-dimensional functions), while the value of the fitness function is placed alongside the -axis.
In summary, the “best/2/bin” strategy achieved the best result when compared with the other three strategies, on average. Interestingly, the “rand/1/bin” strategy was the most successful when optimizing the function of dimension . This behavior was in accordance with the characteristics of this function, because highly multimodal functions demand more randomness during the exploration of the search space.
4.2. Influence of the Number of Evaluations
A long-term lack of improvement regarding the best result during the run was one of the most reliable indicators of the stagnation. If the fitness value did not improve over a number of generations, this probably means that the search process got stuck within a local optimum. In order to detect this undesirable situation during the run of HSABA, the fitness values were tracked at three different phases of the search process, that is, at 1/25, at 1/5, and at the final fitness evaluation. Thus, HSABA runs with the “best/2/bin” strategy and parameters as presented in Section 3.2. The results of this test are collated in Table 4.
It can be seen from Table 4 that HSABA successfully progressed towards the global optimum according to all benchmark functions, except which fell into a local optimum very quickly and did not find any way of further improving the result.
4.3. Influence of the Dimensionality of the Problem
The aim of this experiment is to discover how the quality of the results depends on the dimension of the problem, in other words, the dimensionality of the functions to be optimized. In line with this, three different dimensions of the benchmark functions , , and were taken into account. The results of the tests according to five measures are presented in Table 5.
In this experiment, it was expected that the functions of the higher dimensions would be harder to optimize and therefore the obtained results would be worse. The results in Table 5 show that our assumption held for the average fitness values in general, except for function , where optimizing this function of dimension returned the worst fitness value as by functions of dimensions and .
4.4. Comparative Study
The true value of each algorithm is shown for only when we compared it with the other algorithms solving the same problems. Therefore, the HSABA was compared with algorithms such as BA, SABA, FA, DE, and ABC. All the algorithms were solved using the same benchmark functions as proposed in Section 3.1 and used the parameter setting as explained in Section 3.2. The results of this comparative study obtained by optimizing the functions of dimension are illustrated in Table 6. The best results in these tables are presented as bold.
It can be seen from Table 6 that the HSABA outperformed the results of the other algorithms sixfold (i.e., by , , , , , and ), DE twice (i.e., by and ), while the other algorithms once (i.e., SABA by and ABC by ). BA and FA did not outperform any other algorithm in the test.
In order to evaluate the quality of the results statistically, Friedman tests [29, 30] were conducted to compare the average ranks of the compared algorithms. Thus, a null hypothesis is placed that states the following: two algorithms are equivalent and therefore their ranks should be equal. When the null hypothesis is rejected, the Bonferroni-Dunn test [31] is performed. In this test, the critical difference is calculated between the average ranks of those two algorithms. If the statistical difference is higher than the critical difference, the algorithms are significantly different.
Three Friedman tests (Figure 2) were performed regarding data obtained by optimizing ten functions of three different dimensions according to five measures. As a result, each algorithm during the tests (also the classifier) was compared with regard to the 10 functions × 5 measures; this means 50 different variables. The tests were conducted at the significance level . The results of the Friedman nonparametric test can be seen in Figure 1 that is divided into three diagrams. Each diagram shows the ranks and confidence intervals (critical differences) for the algorithms under consideration with regard to the dimensions of the functions. Note that the significant difference between two algorithms is observed if their confidence intervals denoted by thickened lines in Figure 1 do not overlap.
The first two diagrams in Figure 1 show that the HSABA significantly outperforms the results of the other algorithms by solving the benchmark functions with dimensions and . Although the other algorithms are not significantly different between each other the algorithms DE, SABA, and ABC are substantially better than the BA and FA. The third diagram shows that the HSABA remains significantly better than the ABC, FA, and BA, while the critical difference between DE and SABA was reduced and therefore becomes insignificant. On the other hand, this reduction caused the difference between DE and the other algorithms in the test, except SABA, to become significant.
5. Conclusion
The original BA algorithm gained better results by optimizing the lower-dimensional problems. When this algorithm was tackled with the harder problems of higher dimensions the results became poorer. In order to improve performance on high dimensional problems, almost two mechanisms are proposed in the publications: firstly, self-adaptation of control parameters and secondly hybridization of the original algorithm with problem-specific knowledge in the form of local search.
In this paper, both mechanisms were incorporated within the original BA algorithm in order to obtain the hybrid self-adaptive HSABA. Thus, the self-adaptive mechanism was borrowed from the self-adaptive jDE, while four different DE strategies were used as a local search. Finally, the evolution of the original BA via self-adaptive SABA and finally to hybrid HSABA is presented in detail.
On the one hand, the experimental work focuses on discovering the characteristics of the HSABA, while on the other hand, it focuses on the comparative study, in which the HSABA was compared with its predecessors BA and SABA as well as with other well-known algorithms, like FA, DE, and ABC. The results of this extensive work showed that the HSABA significantly outperformed the results of all the other algorithms in the tests. Therefore, the assumption taken from publications that self-adaptation and hybridization are appropriate mechanisms for improving the results of population-based algorithms was confirmed.
In the future work, this started work will be continued with experiments on the large-scale global optimization problems.
Conflict of Interests
All authors declare that there is no conflict of interests regarding the publication of this paper.