Multi-swarm Problem Solving in Networks Tony White email: tony@sce.carleton.ca

1. Multi-swarm Problem Solving in NetworksTony Whiteemail: tony@sce.carleton.ca

2. Overview • Introduction • What is it and why is it interesting? • Problem solving and stigmergy. • Swarm problem solving • Swarm agent architecture • Connection-oriented routing • Simple diagnosis • Extensions and Future

3. What is Swarm Intelligence? • “Swarm Intelligence is a property of systems of non-intelligent robots exhibiting collectively intelligent behavior.” [Beni, 89] • Characteristics of a swarm: • distributed, no central control or data source; • no (explicit) model of the environment; • perception of environment, i.e. sensing; • ability to change environment.

4. What is Swarm Intelligence (cont.)? • Swarm systems are examples of behavior-based systems exhibiting: • multiple lower level competencies; • situated in environment; • limited time to act; • autonomous with no explicit control provided; • problem solving is emergent behavior; • strong emphasis on reaction and adaptation;

5. Motivations • Robust nature of animal problem-solving • simple creatures exhibit complex behavior; • behavior modified by dynamic environment. • Emergent behavior observed in: • bacteria • ants • bees • ...

6. Emergent Problem Solving • For Lasius Niger ants, [Franks, 89] observed: • regulation of 1 degree celcius range; • forming bridges; • raiding specific areas for food; • building and protecting nest; • sorting brood and food items; • cooperating in carrying large items; • emigration of a colony; • finding shortest route from nest to food source; • preferentially exploiting the richest food source available.

7. Stigmergy • Indirect communication via interaction with environment [Grassé, 59] • Sematonic [Wilson, 75] stigmergy • action of agent directly related to problem solving and affects behavior of other agents. • Sign-based stigmergy • action of agent affects environment not directly related to problem solving activity.

8. Ant Colony • Ants are behaviorally unsophisticated; collectively perform complex tasks. • Ants have highly developed sophisticated sign-based stigmergy • communicate using pheromones; • trails are laid that can be followed by other ants.

9. Pheromone Trails • Species lay pheromone trails travelling from nest, to nest or possibly in both directions. • pheromones evaporate. • pheromones accumulate with multiple ants using path. Food source Nest

10. E E 30 ants 30 ants D D D d=0.5 C C H C H B B B A A A Pheromone Trails continued E T = 1 T = 0 10 ants 20 ants 15 ants 15 ants d=1.0 H d=0.5 d=1.0 10 ants 20 ants 15 ants 15 ants 30 ants 30 ants

11. Swarm operation My Agent • Swarm agents: • arrive at a node, • sense environment, • undertake local activity, • modify environment, • use sensory input to make migration decision

12. Swarm Agent Architecture A=(E,R,C,MDF,m) • Agents have a uniform architecture consisting of five components: • emitters (E), • receptors (R), • chemistry (C), • a migration decision function (MDF), • memory (m)

13. Multi-swarm Architecture Planning Management Control

14. My Agent Emitters • Generators of chemical messages (E) Chemical modification of environment

15. My Agent Receptors • Sensors of chemical messages (R) from local environment Detector for local environment

16. Chemicals • Chemicals are digitally-encoded using a {1, 0, #} alphabet. • The # symbol unifies with 1 or 0. • Chemicals have two attributes: • encoding • concentration • Chemicals participate in reactions.

17. My Agent Chemistry • The set of chemical reactions (C) that can operate on sensed and locally stored chemicals. Chemical interactions

18. Examples of chemical reactions Catalytic breakdown of 011 Exothermic reactions Endothermic reactions

19. Migration Decision Function • The MDF is used to determine the next node to visit in the network pijk (t) = Pp[Tijkp(t) ]-akp[C(i,j)]-b/ Nk(i,j,t) Nk(i,j,t) = Sj in A(i)Pp[Tijkp(t) ]- akp[C(i,j)]-b akp, b are control parameters Nk(i,j,t) is a normalization term A(i) is the set of available egress links

20. Multi-Swarm scenario • Distributed Network Management employing delegation [Yemini, 91] • Three interacting swarms: • connection finding, • connection monitoring, • connection fault diagnosis D A E B C

21. Routing Problem • Idea • Ants dropping different pheromones used to compute “shortest” path from source to destination(s); • more flexible adaptation to failures and network congestion; • use only local knowledge for routing and avoid costly communication of state to all network nodes.

22. Why Routing? • Conventional routing often relies on: • global state available at all nodes; • centralized control; • fixed “shortest path” (Dijkstra) algorithms; • limited ability to deal with congestion or failure. • Ideally, would like to have network adapt routing patterns to take advantage of free resources and move existing traffic if possible.

23. Routing Research • Three approaches so far investigated: • [White et al, 96+] • [Schoonderwoerd et al, 97] (*) • [Di Caro and Dorigo, 97] • Differences • Link cost metric constant in * • Point to point traffic only in * • AS parameter settings constant in *

24. Routing agents • Agent types: • explorer • used for route determination • allocator • allocates resources in network when route emerged • deallocator • deallocates resources in network at end of call

25. Point-2-Point Connections • For explorer agents: • At each node, they choose path with probability proportional to f(ce, pe); • explorers visit edges once only (achieved through use of tabu list); • when destination reached, ants return along the path explored laying down pheromone trail; • when explorers return a decision is made regarding path emergence

26. Path Emergence • At source node (“nest”): • store paths for previous m explorer agents; • when p% follow same path allocator agent is sent to allocate bandwidth in network; • explorer agents continue to look for new (possibly better) paths. • Applies for one or many pt-2-pt connections: • ants use different, non-reacting pheromones.

27. 1. Initialize set t:= 0 For every edge (i,j) set an initial value Tij(t) for trail intensity. Place m ants on the source node. [Generate new explorers at freq. ef] ] 2. Set s:= 1 { tabu list index) for k:= 1 to m do Place starting town of the kth ant in tabuk(s). 3. Repeat until dest’n reached: Set s := s + 1 for k:=1 to m do Choose the node j to move to with probability pijk (t) Move the kth ant to node j. Update explorer route cost: rk = rk + c(i,j) if (rk > rmax) kill explorerk Insert town j in tabuk(s). At destination go to 4. 4. While s > 1 traverse edge (i,j) T(i,j) = T(i,j) + pe s := s - 1 5. At source node do: if (pathe = pathBuffer * d) create and send allocator if t > Tmax create and send allocator Explorer agent algorithm Evaporation occurs concurrently with exploration

28. Point-2-Multipoint Connections • For j destinations, consider as j pt-2-pt connections with: • same pheromone, i.e. all explorers communicate; • j allocator agents only allocate bandwidth once; • allocator send decision made when % of all j explorer ants agree on spanning tree.

29. Routing Function Transition probability (mdf): pijk (t) = [Tijk(t) ]-a[C(i,j)]-b/ Nk(i,j,t) Nk (i,j,t) = Sj in (S-Tabu(k)) [Tijk(t) ]-a[C(i,j)]-b a, bare control parameters that determine the sensitivity of the algorithm to link cost and pheromone. C(i,j) a function that depends upon the type of traffic, the length and utilization of the link.

30. Allocator agents • Allocator agents can fail: • bandwidth already allocated by time allocator is sent; • allocator agent backtracks to source rolling back resource allocation and decreases pheromone levels; • decision to re-send allocator made at a later time (a backoff period is observed); • explorer ants continue to search for routes.

31. Connection monitoring agents • Connection’s quality of service monitored. • Changes in QoS cause Connection monitoring agents to be sent into network, laying down q-chemical. D 2 A E B C

32. Connection diagnosis agents • Examples of netlets • Senseq-chemicalconcentrations Mdf D 2 A E B C

33. Experimental Parameters (fixed) • Number of ants to create = 20 • Frequency of creation = every 10 cycles • Amount of pheromone dropped = 10 • pheromone evaporation rate = 0.9 • Sample window size = 50 • Emergence criterion = 0.9

34. Routing Results • Shortest paths emerged quickly • Mixed pt-2-pt, pt-2-mpt routes emerged • Routing responds to changes in environment: • node failure; • link failure; • link cost changes

35. Parameter Sensitivity • Bad solutions and stagnation • For high values of a the algorithm enters stagnation behavior very quickly without finding very good solutions. • Bad solutions and no stagnation • a too low, insufficient importance associated with trail. • Good solutions • a , b in the central area (1,1), (1,2), (1,5), (0.5, 5)

36. Routing Results • Link cost functions: C(i,j) quadratic constant Five functions studied experimentally: • constant • linear • linear threshold • quadratic • server (1/1-u) At high occupancy (> 50%) server appeared to give the best results. At low occupancy (<25%) function relatively unimportant. server linear threshold

37. Diagnosis • Diagnosis agent is ‘hill climbing’ in space of q-chemical. • Very quickly finds high q-chemical concentrations and initiates diagnostic activity. • An example of distributed diagnosis through constructive chemical interference.

38. Self Adaptation • pheromone and cost sensitivities should vary during search: • avoid premature convergence; • speed up search considerably. • Explorers encode sensitivity values: • fitness of encoding is cost of route; • new agents are created with and use genetically-manipulated values for route finding.

39. Extensions to current m-Swarm system • Multi-class routing • Higher priority traffic causes pheromone levels of lower priority traffic to decay; • automated re-routing of traffic performed. • Swarms learn to avoid regions of network providing low quality connections.

40. Other (potential) applications of m-Swarm systems • Behavior-based Network Management; • explorer agents allocate routes; • parent agents monitor “health” of explorers; • congestion agents identify global network congestion; • congestion agents signal re-planning of network; • congestion agent pheromones react with routing agent pheromones in order to cause re-routing after network re-planned.

41. Futuristic? • Active networks are being researched at: • M.I.T. • U. Penn. • CMU • Georgia Tech. • Management by delegation [Yemini, 91] considered an essential design criterion for next generation network management systems.