Self organizing agents for grid load balancing
This presentation is the property of its rightful owner.
Sponsored Links
1 / 16

Self-Organizing Agents for Grid Load Balancing PowerPoint PPT Presentation


  • 53 Views
  • Uploaded on
  • Presentation posted in: General

Self-Organizing Agents for Grid Load Balancing. Junwei Cao, Ph.D. Research Scientist Center for Space Research Massachusetts Institute of Technology Cambridge, MA 02139 Phone(617)253-8160 Fax(617)253-7014 [email protected] http://www.mit.edu/~caoj. Acknowledgements.

Download Presentation

Self-Organizing Agents for Grid Load Balancing

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Self organizing agents for grid load balancing

Self-Organizing Agents for Grid Load Balancing

Junwei Cao, Ph.D.

Research Scientist

Center for Space Research

Massachusetts Institute of Technology

Cambridge, MA 02139

Phone(617)253-8160

Fax(617)253-7014

[email protected]

http://www.mit.edu/~caoj


Acknowledgements

Acknowledgements

  • This work was carried out when the author was with C&C Research Laboratories, NEC Europe Ltd., Sankt Augustin, Germany. The COSY development was partly funded by the European GEMSS project (EC/IST FP5 Project No. IST-2001-37153).

  • This work originated from the author’s PhD research at the High Performance Systems Group, Department of Computer Science, University of Warwick, Coventry, UK. The author would like to express his gratitude to group members, including Prof. Graham R. Nudd, Dr. Darren J. Kerbyson, Dr. Stephen A. Jarvis, and Mr. Daniel P. Spooner, for their previous contribution.

  • The author appreciates MIT/LIGO supports to attend the SC2004/GRID2004 conference very much.

GRID2004, Pittsburgh, USA


Two tier resource management in grid computing environments

ARMS

Agent

COSY

User

User

ARMS

Agent

ARMS

Agent

COSY

ARMS

Agent

ARMS

Agent

COSY

COSY

COSY

COSY

Two-tier Resource Management in Grid Computing Environments

GRID2004, Pittsburgh, USA


Cosy resource scheduling and load balancing for clusters

Processor 1

Processor 2

Processor 3

Processor 4

Processor 5

Processor 6

Processor 7

Processor 8

2n-1

COSY: Resource Scheduling and Load Balancing for Clusters

GRID2004, Pittsburgh, USA


Arms agent based resource management for grid computing

ARMS: Agent-Based Resource Management for Grid Computing

GRID2004, Pittsburgh, USA


Grid load balancing user driven vs self organization

Grid Load Balancing: User-Driven vs. Self-Organization

  • An ARMS system can achieve grid load balancing as a result of trying to meet QoS requirements specified explicitly by users

  • Self-organization is investigated in this work for agents to perform load balancing automatically for batch queuing jobs that are not explicitly associated with execution deadlines.

GRID2004, Pittsburgh, USA


Ant like self organization local rules lead to global changes

Ant-like Self-Organization: Local Rules Lead to Global Changes

  • An ant wanders from one agent to another randomly and tries to remember the identity of an agent that is most overloaded;

  • After a certain number of steps (m), the ant changes the mode to search a most underloaded agent, though still wandering randomly.

  • After the same m steps, the ant stops for one step to suggest the current two remembered agents (considered to be most overloaded and underloaded, respectively) to balance their workload.

  • After load balancing is performed, the ant is initialized again and starts a new loop from 1.

GRID2004, Pittsburgh, USA


Ant like self organization for agent based grid load balancing

Ant-like Self-Organization for Agent-Based Grid Load Balancing

GRID2004, Pittsburgh, USA


Performance evaluation a modeling and simulation approach

Performance Evaluation: a Modeling and Simulation Approach

GRID2004, Pittsburgh, USA


Performance metrics balancing speed s vs system efficiency e

Performance Metrics: Balancing Speed (s) vs. System Efficiency (e)

  • Average Workload

  • Load Balancing Level

GRID2004, Pittsburgh, USA


Performance impact of the number of ants on load balancing

Performance Impact of the Number of Ants on Load Balancing

  • With the number of ants increased, load balancing speed is improved.

  • With 200 ants involved, high system efficiency can be achieved as well as reasonable load balancing level and speed.

GRID2004, Pittsburgh, USA


Making tradeoff between balancing speed and system efficiency

Making Tradeoff between Balancing Speed and System Efficiency

n=50; step=300 n=100; step=300

n=20; step=1 n=20; step=300

n=1000; step=300 n=2000; step=300

n=200; step=300 n=500; step=300

GRID2004, Pittsburgh, USA


Performance impact of the number of ant wandering steps

Performance Impact of the Number of Ant Wandering Steps

  • When workload is seriously unbalanced among agents, more load balancing processes help improve performance instead of wandering a lot.

  • When the system has achieved a reasonable load balancing level, it seems wandering more to look for more overloaded or underloaded agents in a larger scope becomes more important.

GRID2004, Pittsburgh, USA


Performance impact of the ant wandering style

Performance Impact of the Ant Wandering Style

  • The optimization guides all of the ants in one direction and a very good load balancing is only achieved in a local area of the agent grid.

  • An additional mechanism is introduced to enable ants “jumping” randomly once a while before stucking in a local area.

GRID2004, Pittsburgh, USA


Ant jumping for global effectiveness of load balancing

Ant Jumping for Global Effectiveness of Load Balancing

style=optimal; step=300 style=jumping; step=300

GRID2004, Pittsburgh, USA


References and related links on agent based grid computing

References and Related Links on Agent-Based Grid Computing

  • Grid Load Balancing Using Intelligent Agents. Future Generation Computer Systems, Special Issue on Intelligent Grid Environment: Principles and Applications, 2005. (to appear)

  • ARMSim: a Modeling and Simulation Environment for Agent-based Grid Computing. SIMULATION, Special Issue on Modeling and Simulation Applications in Cluster and Grid Computing, 80(4-5), 221-229, 2004.

  • Queue Scheduling and Advance Reservations with COSY. IPDPS 2004, Santa Fe, New Mexico, USA.

  • GridFlow: Workflow Management for Grid Computing. CCGrid 2003, Tokyo, Japan, 198-205.

  • ARMS: an Agent-based Resource Management System for Grid Computing. Scientific Programming, Special Issue on Grid Computing, 10(2), 135-148, 2002.

  • Performance Evaluation of an Agent-Based Resource Management Infrastructure for Grid Computing. CCGrid 2001, Brisbane, Australia, 311-318.

  • http://www.mit.edu/~caoj

  • http://www.ccrl-nece.de/gemss

  • http://www.dcs.warwick.ac.uk/~hpsg

  • http://www.agilecomputing.org

GRID2004, Pittsburgh, USA


  • Login