Taxonomy of hybrid metaheuristics
1 / 51

Taxonomy of Hybrid Metaheuristics - PowerPoint PPT Presentation

  • Uploaded on

Taxonomy of Hybrid Metaheuristics. Presented by: Xiaojun Bao & Lijun Wang School of Engineering University of Guelph Paper review for ENGG*6140 Optimization. Outline. Introduction Design issues of Hybrid Metaheuristics

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Taxonomy of Hybrid Metaheuristics' - makaio

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Taxonomy of hybrid metaheuristics
Taxonomy of Hybrid Metaheuristics

Presented by: Xiaojun Bao & Lijun Wang

School of Engineering

University of Guelph

Paper review for ENGG*6140 Optimization


  • Introduction

  • Design issues of Hybrid Metaheuristics

  • Implementation issues of Hybrid Metaheuristics

  • A Grammar for extended hybrid schemes

  • Conclusion

    6. Reference


1.1 Single-solution algorithms:

- descent local search

- greedy heuristic

- simulated annealing

- tabu search


1.2 Metaheuristic algorithms:

- evolutionary algorithms

EA: genetic







- ant colonies

- scater search

- and so on.


1.3 Hybrid Metaheuristics

  • So far, many hybrid metaheuristic algorithms have been proposed and implemented to solve many combinatorial optimization problems, e.g. those known as NP-hard.

  • The best results found for various practical problems have proven that combination of different algorithms are very powerful, in case of large and difficult problems.


  • A taxonomy of hybrid algorithms is presented to provide a common terminology and classification mechanism

  • Based on the general taxonomy, we could make the comparison of the hybrid algorithms in a qualitative way.


  • Hybridization of heuristics involves a few major issues which may be classified as design and implementation respectively.





hardware platform

programming model



Design issues
Design issues

2.1 Hierarchical classification

The structure of the hierarchical portion of the taxonomy is shown as follows:

Design issues1
Design issues

Figure 1. Classification (design issues)

Hierarchical classification
Hierarchical classification

2.1.1 Low-level versus High-level

  • The low-level hybridization addresses the functional composition of a single optimization method. In this hybrid class, a given function of a metaheuristic is replaced by another metaheuristic.

Hierarchical classification1
Hierarchical classification

  • In high-level hybrid algorithms, the different metaheuristics are self-contained. We have no direct relationship to the internal workings of a metaheuristic.

Hierarchical classification2
Hierarchical classification

2.1.2 Relay versus Co-evolutionary

In relay hybridization, a set of meta-heuristics is applied one after another, each using the output of the previous as its input, acting in a pipeline fashion.

Hierarchical classification3
Hierarchical classification

Co-evolutionary hybridization represents cooperative optimization models, in which we have many parallel cooperating agents, each agent carries out a search in a solution space.

Hierarchical classification4
Hierarchical classification

Four classes are divided from this hierarchical taxonomy:

  • LRH (Low-level Relay Hybrid).

    This class of hybrids represents algorithms in which a given metaheuristic is embedded into a single-solution metaheuristic. Few examples from the literature belong to this class.

    Let us look at the following example:

Hierarchical classification5
Hierarchical classification

Figure 2. An example of LRH hybridization embedding local search in simulated


Hierarchical classification6
Hierarchical classification

  • LCH (Low-level Co-evolutionary Hybrid)

    Two competing goals govern the design of a metaheuristic: exploration and exploitation.

    In order to achieve the best performance, most efficient population-based heuristics (i.e., genetic algorithms, scatter search, ant colonies, etc.) have been coupled with local search method such as hill-climbing, simulated annealing and tabu search.

Hierarchical classification7
Hierarchical classification

An example:

Figure 3. LCH. For instance, a tabu search is used as a mutation operator and a greedy heuristic as a crossover operator in a genetic algorithm


Individuals individual

Crossover mutation




Hierarchical classification8
Hierarchical classification

  • HRH(High-level Relay Hybrid).

    In HRH hybrid, self-contained metaheuristics are executed in a sequence.

    For example, evolutionary algorithms are not well suited for fine-tuning structures which are very close to optimal solutions. Instead, the strength of EA is in quickly locating the high performance regions of vast and complex search spaces. Once those regions are located, it may be useful to apply local search heuristics to the high performance structures evolved by the EA.

Hierarchical classification9
Hierarchical classification

Three instances of this hybridization scheme:






Initial population

Population to exploit

Initial population



Population to exploit



Hierarchical classification10
Hierarchical classification

  • HCH (High-level Co-evolutionary Hybrid).

    The HCH scheme involves several self-contained algorithms performing a search in parallel, and cooperating to find an optimum. Intuitively, HCH will ultimately perform at least as well as one algorithm alone, more often perform better, each algorithm providing information to the others to help them.

Hierarchical classification11
Hierarchical classification

  • An example of HCH based on GA is the island model:

Figure 4. The island model of genetic algorithms as an example of High-level Co-evolutionary Hybrid.

Hierarchical classification12
Hierarchical classification

a) The population is partitioned into small subpopulations

by geographic isolation.

c) A GA evolves each subpopulation

b) Individuals can migrate between subpopulations

d) The model is controlled by several parameters:

- topology

- migration rate

- replacement strategy

- migration interval

Hierarchical classification13
Hierarchical classification

e) The results of many experiment done based on this model show that:

the global optimum was found more often when migration (with cooperation) was used than in completely isolated cases (without cooperation).

Flat classification
Flat classification

2.2 Flat classification

2.2.1 Homogeneous versus Heterogeneous

- In homogeneous hybrids, all the combined algorithms use the same metaheuristic. In general, different parameters are used for the algorithms.

- In heterogeneous algorithms, different metaheuristics are used.

Flat classification1
Flat classification

Figure 5. High-level Co-evolutionary Hybridization

HCH(heterogeneous, global, general). Several search

algorithms cooperate, co-adapt, and co-solve a solution.

Flat classification2
Flat classification

The GRASP method may be seen as an iterated heterogeneous HRH hybrid, in which local search is repeated from a number of initial solutions generated by randomized greedy heuristic.

Flat classification3
Flat classification

2.2.2 Global versus Partial

- In global hybrids, all the algorithms search in the whole research space. The goal is here to explore the space more thoroughly.

- In partial hybrids, the problem to be solved is decomposed into sub-problems, each one having its own search space. Then each algorithm is dedicated to the search in one of these sub-space.

Flat classification4
Flat classification

2.2.3 Specialist versus General

- In general hybrids, all the algorithms solve the same target optimization problem. All the above mentioned hybrids are general hybrids.

- Specialist hybrids combine algorithms which solve different problems. An example of such a HCH approach has been developed to solve the quadratic assignment problem(QAP).

Flat classification5
Flat classification

Figure 6. High-level Co-evolutionary hybridization HCH(Global, Heterogeneous, Specialist). Several search algorithms solve different problems

Flat classification6
Flat classification

Another approach of specialist hybrid HRH heuristic is to use an heuristic to optimize another heuristic, i.e. find the optimal values of the parameters of the heuristic. This approach has been used to optimize simulated annealing by GA, ant colonies by GA, and a GA by a GA.

Implementation issues
Implementation issues

The structure of the taxonomy concerning implementation issues is shown in Figure 7.

Implementation issues1
Implementation issues

Figure 7. Classification of hybrid metaheuristics(implementation issues).

Implementation issues2
Implementation issues

Specific versus General-purpose computers

- Application specific computers differ from general purpose ones in that they usually only solve a small range of problems, but often at much higher rates and lower costs. Their internal structure is tailored for a particular problem, and thus can achieve much higher efficiency and hardware utilization than a processor which must handle a wide range of tasks.

Implementation issues3
Implementation issues

Sequential versus Parallel

- Most of the proposed hybrid metaheuristics are sequential.

- According to the size of problems, parallel implementations of hybrid algorithms have been considered. Parallel hybrids may be classified using the different characteristics of the target parallel architecture:

  • SIMD versus MIMD

    In SIMD (Single Instruction stream, Multiple Data Stream) parallel machines, the processors are restricted to execute the same program. They are very efficient in executing synchronized parallel algorithms that contain regular computations and regular data structure.

Implementation issues4
Implementation issues

In parallel MIMD (Multiple Instruction stream, Multiple data stream), the processors are allowed to perform different types of instructions on different data. HCH hybrids based respectively on tabu search, simulated annealing, and genetic algorithms have been implemented on networks of transputers.

  • Shared-memory versus Distributed-memory

    Shared-memory parallel architecture simplicity

    Distributed-memory parallel architecture flexible and




Implementation issues5
Implementation issues

  • Homogeneous versus Heterogeneous

    - Most of massively parallel machines (MPP) and cluster of processors such as IBM SP/2, Cray T3D, and DEC Alpha-farms are composed of homogeneous processors.

    - Heterogeneous network of workstations comp up as platforms for high-performance computing due to the availability of powerful workstations and fast communication networks.

    Look at the following example:

Implementation issues6
Implementation issues

  • Figure 8. Parallel implementation of heterogeneous HCH algorithms.

Implementation issues7
Implementation issues

3.3 Static, Dynamic or Adaptive

  • static: This category represents parallel heuristics in which

    both the number of tasks of the application and the

    location of the work (tasks or data) are generated at

    compile time (static scheduling). The allocation of

    processors to tasks (or data) remains unchanged

    during the execution of the application regardless of

    the current state of the parallel machines. Most of

    the proposed parallel heuristics belong to this class.

The major disadvantage
The major disadvantage

When there are noticeable load or power differences between processors, a significant number of tasks are often idle waiting for other tasks to complete their work.

Implementation issues8
Implementation issues

  • dynamic: To improve the performance of parallel static

    heuristics, dynamic load balancing must be

    introduced. This class represents heuristics

    for which the number of tasks is fixed at

    compile-time, but the location of work (tasks,

    data) is determined and/or changed at run-

    time. For example, load-balancing requirement

    can be met by a dynamic redistribution of

    work between processors.


When the number of tasks exceeds the number of idle nodes, multiple tasks are assigned to the same node. Moreover, when there are more idle nodes than tasks , some of them will not be used.

Implementation issues9
Implementation issues

  • adaptive: Parallel adaptive programs are parallel

    computations with a dynamically changing set of

    tasks. Tasks may be created or killed as a function

    of the load state of the parallel machine. A task is

    created automatically when a node become idle.

    When a node becomes busy, the task is killed.

A grammar for extended hybrids
A grammar for extended hybrids

<hybrid – metaheuristic > <design – issues> <implementation issues>

<design – issues> < hierarchical >< flat >

< hierarchical > < LRH >| < LCH >| < HRH >| < HCH >

< LRH > LRH (< metaheuristic >(< metaheuristic >))

< LCH > LCH (< metaheuristic >(< metaheuristic >))

< HRH > HRH (< metaheuristic > + < metaheuristic >)

< HCH > HCH (< metaheuristic > )

< HCH > HCH (< metaheuristic >, < metaheuristic > )

< flat > ( < nature >, < optimization >, < function > )

< nature > homogeneous | heterogeneous

< optimization > global | partial

A grammar for extended hybrids1
A grammar for extended hybrids

< function > general | specialist

<implementation issues> sequential | parallel < scheduling>

< scheduling> static | dynamic | adaptive

< metaheuristic > LS | TS | SA | GA | ES | GP | NN |

< metaheuristic > GH | AC | SS | NM | CLP |<hybrid-metaheuristic>

Figure 9. A grammar for extended hybridization schemes

A grammar for extended hybrids2
A grammar for extended hybrids

Some hybridization examples:

  • HCH(HRH(GH+LCH(GA(LS)))) hierarchical scheme was used to solve set partitioning problem.

  • HRH(GH+LS+LCH(GA(GH+LS))) scheme was used for traveling salesman problem.

  • Three metaheuristics, GA, SA, and TS have been combined in a LCH(GA(HRH(SA+TS))) scheme to solve a scheduling problem.


  • A taxonomy for hybrid metaheuristics has been presented.

  • It considers solutions to design and implementation issues.

  • Treating the two problems orthogonally is beneficial because it allows one to study, understand, classify and evaluate the algorithms using a well defined set of criteria.


  • Hybrid metaheuristics that hybridize population-based metaheuristics with local search heuristics have been proved to be very efficient for large size and hard optimization problem.

  • The HCH proposes natural way to efficiently implement algorithms on heterogeneous computer environment


  • E-G. Talbi. A Taxonomy of Hybrid Metaheuristics. Journal of Combinatorial Optimization, 1-45, 1999

  • V. Bachelet, P. Preux, and E-G. Talbi. Parallel Hybrid Meta-heuristics: Application to the Quadratic Assignment Problem. May, 1996

  • Olivier C. Martin, and Steve W. Otto. Combining Simulated Annealing with Local Search Heuristics. Annals of Operations Research, 1996

  • D.E. Brown, C. L. Huntley, and A. R. Spillane. A parallel genetic heuristic for the quadratic assignment problem. In Third Int. Conf. On Genetic Algorithms ICGA’ 89, San Mateo, USA, July 1989

The end
The End

  • Thank you !