Topic 7 stigmergy swarm intelligence and ant algorithms
This presentation is the property of its rightful owner.
Sponsored Links
1 / 55

Topic 7: Stigmergy, Swarm Intelligence and Ant Algorithms PowerPoint PPT Presentation


  • 86 Views
  • Uploaded on
  • Presentation posted in: General

Topic 7: Stigmergy, Swarm Intelligence and Ant Algorithms. stigmergy swarm intelligence ant algorithms AntNet: routing AntSystem: TSP. Some natural Swarm Intelligence systems. Ants find the shortest path to food go on raiding parties to gather it carry it cooperatively

Download Presentation

Topic 7: Stigmergy, Swarm Intelligence and Ant Algorithms

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Topic 7 stigmergy swarm intelligence and ant algorithms

Topic 7: Stigmergy, Swarm Intelligence and Ant Algorithms

stigmergy

swarm intelligence

ant algorithms

AntNet: routing

AntSystem: TSP


Some natural swarm intelligence systems

Some natural Swarm Intelligence systems

  • Ants

    • find the shortest path to food

    • go on raiding parties to gather it

    • carry it cooperatively

    • make cemeteries

    • sort their brood by size

    • etc.

  • Termites

    • build nests with complex features like

      • fortified chambers

      • spiral air vents

      • fungus gardens

      • etc.

  • Bees

    • gather pollen with high efficiency, exploiting the nearest richest food source first.

  • Geese

    • coordinate takeoff and landing

    • flight patterns

    • etc.

  • Fish / Birds /…

    • swarms

  • Human societies

    • economies ?


Swarm intelligence

Swarm Intelligence

  • “Swarm Intelligence (SI) is the property of a system whereby the collective behaviors of (unsophisticated) agents interacting locally with their environment cause coherent functional global patterns to emerge.”

  • characteristics of a swarm:

    • distributed, no central control or data source

    • no (explicit) model of the environment

    • perception of environment, i.e. sensing

    • ability to change environment

    • problem solving is emergent behaviour


How do social insects coordinate their behaviour

How Do Social Insects Coordinate Their Behaviour?

  • communication is necessary

  • two types of communication

    • direct

      • antennation, trophallaxis (food or liquid exchange),mandibular contact, visual contact, chemical contact, etc.

    • indirect

      • two individuals interact indirectly when one of them modifies the environment and the other responds to the new environment at a later time

        called stigmergy


Stigmergy

Stigmergy

  • “La coordination des taches, la regulation des constructions ne dependent pas directement des oeuvriers, mais des constructions elles-memes. L’ouvrier ne dirige pas son travail, il est guidé par lui. C’est à cette stimulation d’un type particulier que nous donnons le nom du STIGMERGIE (stigma, piqure; ergon, travail, oeuvre = oeuvre stimulante).” Grassé P. P., 1959

  • [“The coordination of tasks and the regulation of constructions does not depend directly on the workers, but on the constructions themselves. The worker does not direct his work, but is guided by it. It is to this special form of stimulation that we give the name STIGMERGY (stigma, sting; ergon, work, product of labour = stimulating product of labour).”]


Example of stigmergy trail following and ants foraging behaviour

Food

Nest

Example of Stigmergy: Trail Following and Ants Foraging Behaviour

  • while walking, ants and termites

    • may deposit a pheromone on the ground

    • follow with high probability pheromone trails they sense on the ground


Example of stigmergy ant cemeteries in nature

Example of Stigmergy:Ant cemeteries in nature

  • simple behaviour rules for ants

    • wander around

    • if you find a dead antpick it up with probability inversely proportionalto the number of other dead ants nearby

    • if you are carrying a dead antput it downwith probability directly proportionalto the number of other dead ants nearby


Stigmergy1

Stigmergy

Stimulation of workers

by the performance

they have achieved

Grassé P. P., 1959


Stigmergy2

Stigmergy

Communication through marks in the environment …

  • Marks serve as a shared memory for the agents

  • Promotes loose coupling

  • Robust

    Stigmergy ~ action + sign:

  • Agents put marks in the environment (inform other agents about issues of interest)

  • other agents perceive these marks (influence their behavior)

  • manipulate marks

  • other agents perceive the marks

  • ….etc.

    Marks can be

  • static (environment does not manipulate marks over time)

    • e.g., flags

  • dynamic (environment manipulates marks over time)

    • e.g., pheromones, gradient fields


Ants foraging behaviour example the double bridge experiment

Ants Foraging BehaviourExample: The Double Bridge Experiment

15 cm

A

Nest

Food

B

Simple bridge % of ant passages on

the two branches

Goss et al., 1989, Deneubourg et al., 1990


Food foraging in nature

Food foraging in nature


Asymmetric binary bridge experiment

Asymmetric Binary Bridge Experiment


Some results

r = 1

Some Results

r = 2

r is the length ratio among

the two bridges

r = 2

Short branch added later


Artificial stigmergy

“Artificial” Stigmergy

Indirect communicationmediated by

modifications of environmental states

which are only locally accessible by

the communicating agents

Dorigo & Di Caro, 1999

  • Characteristics of artificial stigmergy:

    • Indirect communication

    • Local accessibility


What are ant algorithms

What Are Ant Algorithms?

  • Ant algorithms are multi-agent systems that exploit artificial stigmergy as a means for coordinating artificial ants for the solution of computational problems

  • examples

    • AntNet: network routing

    • AntSystem: TSP


Antnet an ant algorithm for routing in packet switched networks e g internet

AntNet: An Ant Algorithm for Routing in Packet-Switched Networks (e.g. Internet)

  • the routing problem

    • build routing table at each node

    • costs are dynamic

    • adaptive routing is hard


Antnet data structure

AntNet data structure

  • routing table

    • probabilities of choosing each neighbor nodes for each possible final destination

  • trips vector

    • contains statistics about ants’ trip times from current node k to each destination node d (means and variances)

P(k,n,d)


Artificial ants in a network

Probabilistic rule to

choose the path

Artificial Ants in a Network

Pheromone trail

Memory

depositing

?

Source

Destination


Simple antnet algorithm

Simple AntNet Algorithm

  • ants are launched regularlyfrom each node to randomly chosen destination

  • ants build their paths probabilistically with probability function of

    • artificial pheromone values

    • heuristic values

  • ants memorize visited nodes and incurred costs

  • once destination is reached, ants deterministically retrace their path backwards, updating pheromone trails that is a function of the quality of the solution they generated

    • refresh + evaporation !!


  • Antnet routing agents

    AntNet Routing - Agents

    • Two kinds of Agents

      • Forward Ant

        • explores the network and collects information

        • when reaches the destination, changes into backward ant

      • Backward Ant

        • goes back in the same path as forward ant

        • update routing tables for all the nodes in the path


    Antnet f ants and b ants

    (N1,T1)

    (N2,T2)

    (N3,T3)

    (N4, T4)

    (N1,T1)

    (N2,T2)

    (N3,T3)

    (N2,T2)

    (N1,T1)

    (N1,T1)

    Forward Ant

    Forward Ant

    Forward Ant

    Forward Ant

    AntNet: F-Ants and B-Ants

    • F-ants

      • collect implicit and explicit information on paths and traffic load

        • implicit through arrival rate at destination

        • explicit by storing experienced trip times

      • share queues with data packets

    • B-ants

      • backpropagate fast

      • use higher priority queues

    2

    1

    4

    3

    Backward Ant

    Backward Ant

    Backward Ant

    Backward Ant


    Using pheromone and memory to choose the next node

    Using Pheromone and Memoryto Choose the Next Node

    Memory of

    visited nodes

    tijd

    j

    i

    tikd

    tird

    k

    r


    Ants probabilistic transition rule

    Ants’ Probabilistic Transition Rule

    • tijd is the amount of pheromone trail on edge (i,j,d)

    • Jik is the set of feasible nodes ant k positioned on

      node i can move to


    Using pheromone and heuristic to choose the next node

    Using Pheromone and Heuristicto Choose the Next Node

    Memory of

    visited nodes

    a heuristic evaluation of link (i,j,d)

    which introduces problem specific information

    stored in pheromone table

    tijd;ijd

    j

    i

    tikd;ikd

    tird;ird

    k

    r


    Using pheromone and heuristic to choose the next node1

    Using Pheromone and Heuristicto Choose the Next Node

    Memory of

    visited nodes

    a heuristic evaluation of link (i,j,d)

    which introduces problem specific information:

    for AntNet: proportional to the inverse of link (i,j) queue length

    stored in pheromone table

    tijd;ijd

    j

    i

    tikd;ikd

    tird;ird

    k

    r


    Topic 7 stigmergy swarm intelligence and ant algorithms

    • B-ants update routing tables:

      • if B-ant has source node d and goes from node n to k

      • Increase the probability of the channel that backward ant comes from

        P(k,n,d) = P(k,n,d) + R.(1 – P(k,n,d))

      • Decrease the probability of the other channels

        P(k,i,d) = P(k,i,d) - R.(1 – P(k,i,d))for all i != n

        R (0 < R <= 1), function of

        - T: time experienced by F-ant

        - m: avg time for same destination memorized in trips table

        - sigma: std. deviation for same destination memorized in trips table

    P(k,n,d)


    Antnet evaluation

    AntNet evaluation

    • extensive tests and experiments

      • American NSF net

      • Japanese NTT net

      • artificial networks

        • simple graphs

        • grid networks

    • in short: good results

      • similar throughput compared to other routing algorithms

      • remarkably smaller avg. packet delay


    Ant colony optimization aco

    Ant Colony Optimization (ACO)

    • TSP has been a popular problem for the ACO models.

      - several reasons why TSP is chosen …

    • Key concepts:

      • Positive feedback

        • build a solution using local solutions, by keeping good solutions in memory.

      • Negative feedback

        • want to avoid premature convergence, evaporate the pheromone.

      • Time scale

        • number of runs are also critical.


    Traveling salesman problem tsp

    Traveling Salesman Problem (TSP)


    Ant optimization of tsp

    Ant Optimization of TSP

    • seed all paths with initial pheromone

    • ants make a tour of all cities

      • ant probabilistically selects next city to visit considering distance and pheromone strength on each path

  • ants deposit pheromone on their path with an intensity inversely proportional to the length of their tour

  • pheromone decays with each time step

  • repeat steps 2 - 4 until some threshold is met


  • A simple tsp example

    []

    []

    []

    []

    []

    1

    2

    3

    4

    5

    A simple TSP example

    A

    B

    C

    D

    E

    dAB =100;dBC = 60…;dDE =150


    Iteration 1

    [A]

    [E]

    [C]

    [B]

    [D]

    1

    5

    3

    2

    4

    Iteration 1

    A

    B

    C

    D

    E


    How to build next sub solution

    [A]

    [A]

    [A]

    [A]

    [A,D]

    1

    1

    1

    1

    1

    How to build next sub-solution?

    A

    B

    C

    D

    E


    Iteration 2

    [C,B]

    [E,A]

    [A,D]

    [B,C]

    [D,E]

    3

    5

    1

    2

    4

    Iteration 2

    A

    B

    C

    D

    E


    Iteration 3

    Iteration 3

    [D,E,A]

    [E,A,B]

    [C,B,E]

    [B,C,D]

    [A,D,C]

    4

    5

    3

    2

    1

    A

    B

    C

    D

    E


    Iteration 4

    [D,E,A,B]

    [B,C,D,A]

    [E,A,B,C]

    [A,D,C,E]

    [C,B,E,D]

    4

    2

    5

    1

    3

    Iteration 4

    A

    B

    C

    D

    E


    Iteration 5

    [A,D,C,E,B]

    [C,B,E,D,A]

    [D,E,A,B,C]

    [B,C,D,A,E]

    [E,A,B,C,D]

    1

    3

    4

    2

    5

    Iteration 5

    A

    B

    C

    D

    E


    Path and pheromone evaluation

    [A,D,C,E,B]

    [E,A,B,C,D]

    [B,C,D,A,E]

    [C,B,E,D,A]

    [D,E,A,B,C]

    1

    5

    2

    3

    4

    Path and Pheromone Evaluation

    L1 =300

    L2 =450

    L3 =260

    L4 =280

    L5 =420


    Topic 7 stigmergy swarm intelligence and ant algorithms

    End of First Run

    Save Best Tour (Sequence and length)

    All ants die

    New ants are born

    stopping criteria

    stagnation

    max.runs


    Ant cycle for tsp

    Ant cycle for TSP

    • tabu list

    • random walk

    • priorities

    • backtracing and considering edge length

    5

    1

    4

    1

    1

    3

    2


    Tabu list

    Tabu List

    n : number of cities = one city tourm : number of antsk : ant indexs : member in tabu list (current city)

    for each iteration in the ant algorithm cycle:insert town in „visited node list“

    for (int k = 1 ; k <= m ; k++) {

    insert town of ant k in Tabuk(s) }


    Random walk priorities

    Random Walk & Priorities

    i,j : edge between nodes i,jnij : 1/distance(i,j)a : weight for markingb : weight for node „neighbourhoodness“

    allowedk : unvisited cities for ant k

    from ant routing table:

    probability of ant k for going from city i to city j


    Backtracing

    Backtracing

    i,j : edge between nodes i,jtij (t): marking at time tr : evaporation rate (t,t+n)

    Q/Lk : const/tour-length of ant k

    With Tabu List, ant routing table:

    marking_delta = sum of markings of all ants k who passed (i,j)

    marking after tour = evaporation_rate * marking_old + marking_delta


    Ant system ant cycle dorigo 1 1991

    Ant System (Ant Cycle) Dorigo [1] 1991

    t = 0; NC = 0; τij(t)=c for ∆τij=0

    Place the m ants on the n nodes

    Initialize

    Update tabuk(s)

    Tabu list management

    Choose the city j to move to. Use probability

    Move k-th ant to town j. Insert town j in tabuk(s)

    Compute the length Lk of every ant

    Update the shortest tour found

    =

    For every edge (i,j)

    Compute

    For k:=1 to m do

    Yes

    Yes

    NC<NCmax

    && not stagn.

    Set t = t + n; NC=NC+1; ∆τij=0

    No

    End


    10 cities problem cca0

    10-Cities-Problem CCA0

    AntSystem marking distribution at beginning and after 100 cycles


    Oliver30 problem

    Oliver30 Problem

    cycles: 342length : 420

    a = 1b = 5r = 0.5


    Topic 7 stigmergy swarm intelligence and ant algorithms

    Advantages:

    • positive feedback accounts for rapid discovery of good solutions

    • distributed computation avoids premature convergence

    • the greedy heuristic helps find acceptable solution in the early solution in the early stages of the search process

    • the collective interaction of a population of agents

      Disadvantages:

    • slower convergence than other heuristics

    • performed poorly for TSP problems larger than 75 cities


    Improvements to as ant colony system acs

    Improvements to AS: Ant Colony System (ACS)

    • ACS (pheromone update)

    • update pheromone trail while building the solution

    • ants eat pheromone on the trail

    • local search added before pheromone update


    Research summary

    Research Summary

    • Ant System (AS) [Dorigo, Maniezzo & Colorni]

      • Original implementation of ant-inspired optimization

  • Ant Colony System (ACS) [Dorigo & Gambardella]

    • Ants probabilistically choose exploitation or biased exploration

  • Max-Min Ant System (MMAS) [Stutzle & Hoos]

    • Only ant with best tour deposits pheromone

    • Range of pheromone limited to given interval

    • Pheromone levels initialized to max levels


  • Research summary1

    Research Summary

    • refinements

      • Pheromone Trail Smoothing (PTS)

        • increases pheromone on all segments in proportion to the difference from the max value

  • Ranked Ants

    • n ants with best paths deposit pheromone based on rank

  • Elite Ants

    • best path per cycle receives additional pheromone

  • Modified 3-Opt

    • iteratively swap order of three-city sub-paths such that total path length decreases


  • Result comparison

    Result Comparison

    [Stutzle & Hoos, 1999]

    PTS = Pheromone Trail Smoothing

    MMAS = MaxMin AntSystem

    ACS = Ant Colony System

    AS = Ant System

    10,000 cycles per run


    General aco

    General ACO

    • a stochastic construction procedure

    • probabilistically build a solution

    • iteratively adding solution components to partial solutions

      • heuristic information

      • pheromone trail

    • reinforcement

    • modify the problem representation at each iteration

    • ants work concurrently and independently

    • collective interaction via indirect communication leads to good solutions


    Topic 7 stigmergy swarm intelligence and ant algorithms

    Conclusion

    • general ACO work well on static problems like TSP, but hard to beat specialist algorithms

      but most of all …

    • ants are “dynamic” optimizers

    • inherently capable of dealing with dynamism

      Other Application Domains for Ants

    • manufacturing control

    • peer-2-peer

    • active networking

      • similar principles

        • artificial stigmergy / feedback through pheromones

        • information decay (pheromone evaporation)

        • probabilistic choice in path following


  • Login