Taxonomy of hybrid metaheuristics
This presentation is the property of its rightful owner.
Sponsored Links
1 / 51

Taxonomy of Hybrid Metaheuristics PowerPoint PPT Presentation


  • 119 Views
  • Uploaded on
  • Presentation posted in: General

Taxonomy of Hybrid Metaheuristics. Presented by: Xiaojun Bao & Lijun Wang School of Engineering University of Guelph Paper review for ENGG*6140 Optimization. Outline. Introduction Design issues of Hybrid Metaheuristics

Download Presentation

Taxonomy of Hybrid Metaheuristics

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Taxonomy of hybrid metaheuristics

Taxonomy of Hybrid Metaheuristics

Presented by: Xiaojun Bao & Lijun Wang

School of Engineering

University of Guelph

Paper review for ENGG*6140 Optimization


Outline

Outline

  • Introduction

  • Design issues of Hybrid Metaheuristics

  • Implementation issues of Hybrid Metaheuristics

  • A Grammar for extended hybrid schemes

  • Conclusion

    6. Reference


Introduction

Introduction

1.1 Single-solution algorithms:

- descent local search

- greedy heuristic

- simulated annealing

- tabu search


Introduction1

Introduction

1.2 Metaheuristic algorithms:

- evolutionary algorithms

EA: genetic

algorithms

evolution

strategies

genetic

programming


Introduction2

Introduction

- ant colonies

- scater search

- and so on.


Hybridization

Hybridization

1.3 Hybrid Metaheuristics

  • So far, many hybrid metaheuristic algorithms have been proposed and implemented to solve many combinatorial optimization problems, e.g. those known as NP-hard.

  • The best results found for various practical problems have proven that combination of different algorithms are very powerful, in case of large and difficult problems.


Hybridization1

Hybridization

  • A taxonomy of hybrid algorithms is presented to provide a common terminology and classification mechanism

  • Based on the general taxonomy, we could make the comparison of the hybrid algorithms in a qualitative way.


Hybridization2

Hybridization

  • Hybridization of heuristics involves a few major issues which may be classified as design and implementation respectively.

functionality

design

architecture


Hybridization3

Hybridization

hardware platform

programming model

implementation

environment


Design issues

Design issues

2.1 Hierarchical classification

The structure of the hierarchical portion of the taxonomy is shown as follows:


Design issues1

Design issues

Figure 1. Classification (design issues)


Hierarchical classification

Hierarchical classification

2.1.1 Low-level versus High-level

  • The low-level hybridization addresses the functional composition of a single optimization method. In this hybrid class, a given function of a metaheuristic is replaced by another metaheuristic.


Hierarchical classification1

Hierarchical classification

  • In high-level hybrid algorithms, the different metaheuristics are self-contained. We have no direct relationship to the internal workings of a metaheuristic.


Hierarchical classification2

Hierarchical classification

2.1.2 Relay versus Co-evolutionary

In relay hybridization, a set of meta-heuristics is applied one after another, each using the output of the previous as its input, acting in a pipeline fashion.


Hierarchical classification3

Hierarchical classification

Co-evolutionary hybridization represents cooperative optimization models, in which we have many parallel cooperating agents, each agent carries out a search in a solution space.


Hierarchical classification4

Hierarchical classification

Four classes are divided from this hierarchical taxonomy:

  • LRH (Low-level Relay Hybrid).

    This class of hybrids represents algorithms in which a given metaheuristic is embedded into a single-solution metaheuristic. Few examples from the literature belong to this class.

    Let us look at the following example:


Hierarchical classification5

Hierarchical classification

Figure 2. An example of LRH hybridization embedding local search in simulated

annealing


Hierarchical classification6

Hierarchical classification

  • LCH (Low-level Co-evolutionary Hybrid)

    Two competing goals govern the design of a metaheuristic: exploration and exploitation.

    In order to achieve the best performance, most efficient population-based heuristics (i.e., genetic algorithms, scatter search, ant colonies, etc.) have been coupled with local search method such as hill-climbing, simulated annealing and tabu search.


Hierarchical classification7

Hierarchical classification

An example:

Figure 3. LCH. For instance, a tabu search is used as a mutation operator and a greedy heuristic as a crossover operator in a genetic algorithm

GA

Individuals individual

Crossover mutation

Greedy

heuristic

tabu


Hierarchical classification8

Hierarchical classification

  • HRH(High-level Relay Hybrid).

    In HRH hybrid, self-contained metaheuristics are executed in a sequence.

    For example, evolutionary algorithms are not well suited for fine-tuning structures which are very close to optimal solutions. Instead, the strength of EA is in quickly locating the high performance regions of vast and complex search spaces. Once those regions are located, it may be useful to apply local search heuristics to the high performance structures evolved by the EA.


Hierarchical classification9

Hierarchical classification

Three instances of this hybridization scheme:

Greedy

heuristic

GA

Greedy

heuristic

Initial population

Population to exploit

Initial population

GA

Tabu

Population to exploit

GA

Tabu


Hierarchical classification10

Hierarchical classification

  • HCH (High-level Co-evolutionary Hybrid).

    The HCH scheme involves several self-contained algorithms performing a search in parallel, and cooperating to find an optimum. Intuitively, HCH will ultimately perform at least as well as one algorithm alone, more often perform better, each algorithm providing information to the others to help them.


Hierarchical classification11

Hierarchical classification

  • An example of HCH based on GA is the island model:

Figure 4. The island model of genetic algorithms as an example of High-level Co-evolutionary Hybrid.


Hierarchical classification12

Hierarchical classification

a) The population is partitioned into small subpopulations

by geographic isolation.

c) A GA evolves each subpopulation

b) Individuals can migrate between subpopulations

d) The model is controlled by several parameters:

- topology

- migration rate

- replacement strategy

- migration interval


Hierarchical classification13

Hierarchical classification

e) The results of many experiment done based on this model show that:

the global optimum was found more often when migration (with cooperation) was used than in completely isolated cases (without cooperation).


Flat classification

Flat classification

2.2 Flat classification

2.2.1 Homogeneous versus Heterogeneous

- In homogeneous hybrids, all the combined algorithms use the same metaheuristic. In general, different parameters are used for the algorithms.

- In heterogeneous algorithms, different metaheuristics are used.


Flat classification1

Flat classification

Figure 5. High-level Co-evolutionary Hybridization

HCH(heterogeneous, global, general). Several search

algorithms cooperate, co-adapt, and co-solve a solution.


Flat classification2

Flat classification

The GRASP method may be seen as an iterated heterogeneous HRH hybrid, in which local search is repeated from a number of initial solutions generated by randomized greedy heuristic.


Flat classification3

Flat classification

2.2.2 Global versus Partial

- In global hybrids, all the algorithms search in the whole research space. The goal is here to explore the space more thoroughly.

- In partial hybrids, the problem to be solved is decomposed into sub-problems, each one having its own search space. Then each algorithm is dedicated to the search in one of these sub-space.


Flat classification4

Flat classification

2.2.3 Specialist versus General

- In general hybrids, all the algorithms solve the same target optimization problem. All the above mentioned hybrids are general hybrids.

- Specialist hybrids combine algorithms which solve different problems. An example of such a HCH approach has been developed to solve the quadratic assignment problem(QAP).


Flat classification5

Flat classification

Figure 6. High-level Co-evolutionary hybridization HCH(Global, Heterogeneous, Specialist). Several search algorithms solve different problems


Flat classification6

Flat classification

Another approach of specialist hybrid HRH heuristic is to use an heuristic to optimize another heuristic, i.e. find the optimal values of the parameters of the heuristic. This approach has been used to optimize simulated annealing by GA, ant colonies by GA, and a GA by a GA.


Implementation issues

Implementation issues

The structure of the taxonomy concerning implementation issues is shown in Figure 7.


Implementation issues1

Implementation issues

Figure 7. Classification of hybrid metaheuristics(implementation issues).


Implementation issues2

Implementation issues

Specific versus General-purpose computers

- Application specific computers differ from general purpose ones in that they usually only solve a small range of problems, but often at much higher rates and lower costs. Their internal structure is tailored for a particular problem, and thus can achieve much higher efficiency and hardware utilization than a processor which must handle a wide range of tasks.


Implementation issues3

Implementation issues

Sequential versus Parallel

- Most of the proposed hybrid metaheuristics are sequential.

- According to the size of problems, parallel implementations of hybrid algorithms have been considered. Parallel hybrids may be classified using the different characteristics of the target parallel architecture:

  • SIMD versus MIMD

    In SIMD (Single Instruction stream, Multiple Data Stream) parallel machines, the processors are restricted to execute the same program. They are very efficient in executing synchronized parallel algorithms that contain regular computations and regular data structure.


Implementation issues4

Implementation issues

In parallel MIMD (Multiple Instruction stream, Multiple data stream), the processors are allowed to perform different types of instructions on different data. HCH hybrids based respectively on tabu search, simulated annealing, and genetic algorithms have been implemented on networks of transputers.

  • Shared-memory versus Distributed-memory

    Shared-memory parallel architecture simplicity

    Distributed-memory parallel architecture flexible and

    fault-tolerant

    programming

    platform


Implementation issues5

Implementation issues

  • Homogeneous versus Heterogeneous

    - Most of massively parallel machines (MPP) and cluster of processors such as IBM SP/2, Cray T3D, and DEC Alpha-farms are composed of homogeneous processors.

    - Heterogeneous network of workstations comp up as platforms for high-performance computing due to the availability of powerful workstations and fast communication networks.

    Look at the following example:


Implementation issues6

Implementation issues

  • Figure 8. Parallel implementation of heterogeneous HCH algorithms.


Implementation issues7

Implementation issues

3.3 Static, Dynamic or Adaptive

  • static: This category represents parallel heuristics in which

    both the number of tasks of the application and the

    location of the work (tasks or data) are generated at

    compile time (static scheduling). The allocation of

    processors to tasks (or data) remains unchanged

    during the execution of the application regardless of

    the current state of the parallel machines. Most of

    the proposed parallel heuristics belong to this class.


The major disadvantage

The major disadvantage

When there are noticeable load or power differences between processors, a significant number of tasks are often idle waiting for other tasks to complete their work.


Implementation issues8

Implementation issues

  • dynamic: To improve the performance of parallel static

    heuristics, dynamic load balancing must be

    introduced. This class represents heuristics

    for which the number of tasks is fixed at

    compile-time, but the location of work (tasks,

    data) is determined and/or changed at run-

    time. For example, load-balancing requirement

    can be met by a dynamic redistribution of

    work between processors.


Disadvantage

Disadvantage

When the number of tasks exceeds the number of idle nodes, multiple tasks are assigned to the same node. Moreover, when there are more idle nodes than tasks , some of them will not be used.


Implementation issues9

Implementation issues

  • adaptive: Parallel adaptive programs are parallel

    computations with a dynamically changing set of

    tasks. Tasks may be created or killed as a function

    of the load state of the parallel machine. A task is

    created automatically when a node become idle.

    When a node becomes busy, the task is killed.


A grammar for extended hybrids

A grammar for extended hybrids

<hybrid – metaheuristic > <design – issues> <implementation issues>

<design – issues> < hierarchical >< flat >

< hierarchical > < LRH >| < LCH >| < HRH >| < HCH >

< LRH > LRH (< metaheuristic >(< metaheuristic >))

< LCH > LCH (< metaheuristic >(< metaheuristic >))

< HRH > HRH (< metaheuristic > + < metaheuristic >)

< HCH > HCH (< metaheuristic > )

< HCH > HCH (< metaheuristic >, < metaheuristic > )

< flat > ( < nature >, < optimization >, < function > )

< nature > homogeneous | heterogeneous

< optimization > global | partial


A grammar for extended hybrids1

A grammar for extended hybrids

< function > general | specialist

<implementation issues> sequential | parallel < scheduling>

< scheduling> static | dynamic | adaptive

< metaheuristic > LS | TS | SA | GA | ES | GP | NN |

< metaheuristic > GH | AC | SS | NM | CLP |<hybrid-metaheuristic>

Figure 9. A grammar for extended hybridization schemes


A grammar for extended hybrids2

A grammar for extended hybrids

Some hybridization examples:

  • HCH(HRH(GH+LCH(GA(LS)))) hierarchical scheme was used to solve set partitioning problem.

  • HRH(GH+LS+LCH(GA(GH+LS))) scheme was used for traveling salesman problem.

  • Three metaheuristics, GA, SA, and TS have been combined in a LCH(GA(HRH(SA+TS))) scheme to solve a scheduling problem.


Conclusion

Conclusion

  • A taxonomy for hybrid metaheuristics has been presented.

  • It considers solutions to design and implementation issues.

  • Treating the two problems orthogonally is beneficial because it allows one to study, understand, classify and evaluate the algorithms using a well defined set of criteria.


Conclusion1

Conclusion

  • Hybrid metaheuristics that hybridize population-based metaheuristics with local search heuristics have been proved to be very efficient for large size and hard optimization problem.

  • The HCH proposes natural way to efficiently implement algorithms on heterogeneous computer environment


References

References

  • E-G. Talbi. A Taxonomy of Hybrid Metaheuristics. Journal of Combinatorial Optimization, 1-45, 1999

  • V. Bachelet, P. Preux, and E-G. Talbi. Parallel Hybrid Meta-heuristics: Application to the Quadratic Assignment Problem. May, 1996

  • Olivier C. Martin, and Steve W. Otto. Combining Simulated Annealing with Local Search Heuristics. Annals of Operations Research, 1996

  • D.E. Brown, C. L. Huntley, and A. R. Spillane. A parallel genetic heuristic for the quadratic assignment problem. In Third Int. Conf. On Genetic Algorithms ICGA’ 89, San Mateo, USA, July 1989


The end

The End

  • Thank you !


  • Login