1 / 23

Agent-Based Grid Load Balancing Using Performance-Driven Task Scheduling

Agent-Based Grid Load Balancing Using Performance-Driven Task Scheduling. Junwei Cao (C&C Research Labs, NEC Europe Ltd., Germany) D. P. Spooner, S. A. Jarvis and G. R. Nudd (Dept. Computer Science, Univ. of Warwick, UK) S. Saini ( NASA Ames Research Center, USA). Outline. Overview

omana
Download Presentation

Agent-Based Grid Load Balancing Using Performance-Driven Task Scheduling

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Agent-Based Grid Load Balancing Using Performance-Driven Task Scheduling Junwei Cao (C&C Research Labs, NEC Europe Ltd., Germany) D. P. Spooner, S. A. Jarvis and G. R. Nudd (Dept. Computer Science, Univ. of Warwick, UK) S. Saini (NASA Ames Research Center, USA) IPDPS 2003, Nice, France

  2. Outline • Overview • Performance Prediction • Performance-Driven Task Scheduling • Agent-Based Grid Load Balancing • Experimental Results • Conclusions IPDPS 2003, Nice, France

  3. PACE Application Tools Global Grid Management (GGM): Agent-Based Load Balancing PACE Performance Evaluation Engine Local Grid Management (LGM): Performance-Driven Task Scheduling PACE Resource Tools Overview Grid Users Grid Resources IPDPS 2003, Nice, France

  4. Source Code Analysis Object Editor Object Library HMCL Compiler PSL Compiler CPU Network (MPI, PVM) Cache (L1, L2) The PACE Toolkit Application Tools Evaluation Engine Resource Tools IPDPS 2003, Nice, France

  5. 2n-1 LGM - FIFO Algorithm Processor 1 Processor 2 Processor 3 Processor 4 Processor 5 Processor 6 Processor 7 Processor 8 IPDPS 2003, Nice, France

  6. LGM - Genetic Algorithm • Heuristic • Evolutionary • Near-optimal: • Makespan • Idletime • Deadlines IPDPS 2003, Nice, France

  7. LGM – System Implementation Requests Results Service Communication Module Task Management Task Execution Resource Monitoring Performance-Drvien Task Scheduling PACE Evaluation Engine IPDPS 2003, Nice, France

  8. A A A A GGM – Agent Methodology • Agent structure • Communication layer • Decision-making layer • Local management layer • Agent hierarchy • Service advertisement • Service discovery • Agent Capability Tables A User IPDPS 2003, Nice, France

  9. M GGM - Optimization Strategies • Advertisement • Data-push & data-pull • Periodic & event-driven • Discovery • Local services • Services in ACTs • Lower or upper agents • Optimisation • Modeling • Simulation A User A A A A IPDPS 2003, Nice, France

  10. M A A A A A L L L L L GGM – System Implementation • Service information • PACE resc models • Makespan • Request information • Exec scripts • PACE app model • Deadline • Matchmaking • Estimation (FIFO) • Deadline User IPDPS 2003, Nice, France

  11. Load Balancing Metrics • Total makespan • Average advance time of task execution completions (required deadline - actual task completion time) • Average processor utilisation rate (busy time / total makespan) • Load balancing level (1 - mean square deviation of processor utilisation rates / average processor utilisation rate) • Total number of network packages for both advertisement and discovery IPDPS 2003, Nice, France

  12. S1 (SGIOrigin2000, 16) S2 (SGIOrigin2000, 16) S5 (SunUltra5, 16) S4 (SunUltra10, 16) S3 (SunUltra10, 16) S12 (SunSPARCstation2, 16) S10 (SunUltra1, 16) S8 (SunUltra1, 16) S6 (SunUltra5, 16) S11 (SunSPARCstation2, 16) S7 (SunUltra5, 16) S9 (SunUltra1, 16) Experiment Design Tasks: sweep3d fft improc closure jacobi memsort cpi IPDPS 2003, Nice, France

  13. FIFO FIFO FIFO FIFO FIFO Experiment 1 IPDPS 2003, Nice, France

  14. FIFO FIFO FIFO FIFO FIFO Experiment 1 IPDPS 2003, Nice, France

  15. GA GA GA GA GA Experiment 2 IPDPS 2003, Nice, France

  16. GA GA GA GA GA Experiment 3 IPDPS 2003, Nice, France

  17. Task Execution Both GA and agents contribute towards the improvement in task executions. IPDPS 2003, Nice, France

  18. Resource Utilisation Less powerful S11 & S12 benefit mainly from the GA. More powerful S1 & S2 benefit mainly from agents. IPDPS 2003, Nice, France

  19. Load Balancing The GA contributes more to local grid load balancing. Agents contribute more to global grid load balancing. IPDPS 2003, Nice, France

  20. Total Makespan The centralised pure data-pull can always achieve the best results Distributed agents with the hierarchical model can also achieve reasonably good results IPDPS 2003, Nice, France

  21. Network Package The network overhead for the pure data-pull strategy to achieve better results is very high. Distributed agent-based service advertisement and discovery can scale well. IPDPS 2003, Nice, France

  22. Conclusions • Application performance prediction can be utilized for grid QoS support and resource scheduling. • An multi-agent paradigm provides a clear high-level abstraction of grid management system. • Distributed service advertisement and discovery strategies can be utilized to achieve grid load balancing. IPDPS 2003, Nice, France

  23. Related Works • Medical Simulation Services via the Grid G. Lonsdale, NEC Europe Ltd. Friday, Industrial Track II, IPDPS 2003 • Performance Prediction and its use in Parallel and Distributed Computing Systems S. A. Jarvis, University of Warwick Saturday, PMEO Workshop , IPDPS 2003 • GridFlow: Workflow Management for Grid Computing To appear in CCGrid 2003, Tokyo, Japan IPDPS 2003, Nice, France

More Related