1 / 29

A Resource Allocation Approach for Supporting Time-Critical Applications in Grid Environments

A Resource Allocation Approach for Supporting Time-Critical Applications in Grid Environments. Qian Zhu and Gagan Agrawal Department of Computer Science and Engineering The Ohio State University. IPDPS 2009 Conference. May 28 th , 2009 Rome, Italy. IPDPS 2009. Context.

akamu
Download Presentation

A Resource Allocation Approach for Supporting Time-Critical Applications in Grid Environments

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Resource Allocation Approach for Supporting Time-Critical Applications in Grid Environments Qian Zhu and Gagan Agrawal Department of Computer Science and Engineering The Ohio State University IPDPS 2009 Conference May 28th, 2009 Rome, Italy IPDPS 2009

  2. Context • Ongoing research on supporting time-critical adaptive applications • Fixed time, flexible computations • Maximize a QoS/Benefit function • Previous work • Middleware design • Self-adaptation algorithm (ICAC 2008)

  3. Motivating Application: Real-timeVolume Rendering (VR) • Flexibility: image quality, image size… • Time constraints 3 IPDPS 2009

  4. Motivating Application: Great Lake Nowcasting and Forecasting (GLFS) • Flexibility • Grid resolution • Internal time step • External time step • Time Constraints IPDPS 2009

  5. Summary of Application Needs • Time-Critical Event Handling • Intense computation and communication • Time and resource constraints • Application-specific flexibility • benefit function • VR application • GLFS application • Grid Resources IPDPS 2009

  6. Overview of Our Research • To Optimize the Benefit Function within the Time Constraint • Parameter Adaptation • VR application: error tolerance, image size • GLFS application: internal/external time step • Resource Allocation • Heterogeneous and dynamic resources IPDPS 2009

  7. Outline Motivation and Introduction Resource Allocation Approach Approach Overview Efficiency Value Scheduling Algorithm Experimental Evaluation Related Work Conclusion 7 IPDPS 2009

  8. Experimental Study: Real-time Volume Rendering • The CPU/memory usage increases as ErrorTolerance value decreases or the ImageSize value increases. • The change in the value of ErrorTolerance has a more significant impact, compared to the ImageSize parameter. IPDPS 2009

  9. Experimental Study: Great Lake Nowcasting and Forecasting • The CPU usage changes as the values of ExternalTimeStep and InternalTimeStep vary. • The memory usage remains roughly the same. IPDPS 2009

  10. Problem Description • Heterogeneous and Dynamic Resources • Different CPU, Memory, and/or Bandwidth Usage • Different service components • Different values of adjustable service parameters within the same service component • Schedule the Service Components to Maximize the Benefit Function Within the Time Constraint IPDPS 2009

  11. Outline Motivation and Introduction Resource Allocation Approach Approach Overview Efficiency Value Scheduling Algorithm Experimental Evaluation Related Work Conclusion 11 IPDPS 2009

  12. Error tolerance Image size Wavelet coefficient Application Model Data Packet S1 S2 S3 S4 S5 S6 • Each service component is deployed on a single node • Multiple processing round IPDPS 2009 CAC 2008

  13. Resource Allocation Approach Overview • Allocate Heterogeneous Resources to Services to Maximize the Benefit Within the Time Constraint • Unique characteristics of resource usage • Extra resource usage by varying the values of adaptive parameters • Normal Execution Phase • Train rules for Efficiency Value estimation • Assign service priority • Event Handling Phase • Apply the learned rules to infer Efficiency Value • Priority-based scheduling IPDPS 2009

  14. Efficiency Value • To capture the suitability of executing the Service on the Processing Node • Definition • Benefit contribution , where • Adaptation overhead , where • Node status • Weighted sum of standard deviation of the workload and resource variance every 30 seconds IPDPS 2009

  15. Efficiency value estimation Fuzzy logic rules how efficient is for supporting parameter adaptation of for overall benefit optimization Efficiency Value – Cont’d • Calculating Efficiency Value standard deviation of workload and resource variance IPDPS 2009

  16. Efficiency Value -- Example Figure: Example of Efficiency Value Calculation: (a) Computed Values (b) Normalized Benefit with Different Allocations • Assigning to and to yields the maximum benefit • Our definition of efficiency value captures the suitability of different nodes for different services 16 IPDPS 2009

  17. Scheduling Algorithm • Greedy Scheduling • Service priority based • Benefit Optimization and Meeting the Time Deadline • Adjust and communication time of computation time of IPDPS 2009

  18. Outline Motivation and Introduction Resource Allocation Approach Approach Overview Efficiency Value Scheduling Algorithm Experimental Evaluation Related Work Conclusion 18 IPDPS 2009

  19. Experiments Setup • Algorithms Compared • GrADS (UCSD) • Optimal • Metrics • Normalized benefit • Success-rate • Simulated Grid Environments • HighReHetero, ModReHetero, and LowReHetero IPDPS 2009

  20. Experiment1: Effectiveness of Our Learning Approach • MSE converges within 20mins, 35mins and 1hour for a 5-hour run IPDPS 2009

  21. Experiment2: Normalized Benefit Comparison (VolumeRendering) Figure 10: Normalized Benefit Comparison of Our Approach with GrADS and Optimal: Highly Heterogeneous Environment * Our algorithm achieves an average of 87% normalized benefit comparing to the Optimal and it is 32% higher than GrADS. IPDPS 2009

  22. Experiment2: Success-Rate Comparison (VolumeRendering) Figure 10: Success-Rate Comparison of Our Approach with GrADS and Optimal: Highly Heterogeneous Environment * Our algorithm achieves 90% to 100% success-rate comparing to the Optimal. While GrADS can achieve 80% to 90%. IPDPS 2009

  23. Experiment2: Overhead Comparison (a) (b) Figure 14: Resource Allocation Overhead Comparison of Our Approach with GrADS: (a) Volume Rendering Application (b) GLFS Application * The overhead caused by our algorithm is within 10% and 7% of that of the GrADS for VR and GLFS applications. IPDPS 2009

  24. Experiment 3: Scalability Figure 15: Scalability of Different Resource Allocation Approaches • An average slowdown of 9%, 7%, and 3%, respectively, in the three grid environments • Scheduling 160 service components is 26.4 seconds IPDPS 2009

  25. Outline Motivation and Introduction Resource Allocation Approach Approach Overview Efficiency Value Scheduling Algorithm Experimental Evaluation Related Work Conclusion 25 IPDPS 2009

  26. Related Work Resource Allocation in Grid Computing Iosup et al. (SC2007) Xu et al. (ICAC2007) Huang et al. (SC2007) Tesauro et al. (ICAC2006) Real-Time Scheduling Survey (Real Time Systems, 2004) Q-RAM (RTSS1998) Gopalan et al. (MMCN2002) 26 IPDPS 2009

  27. Outline Motivation and Introduction Resource Allocation Approach Approach Overview Efficiency Value Scheduling Algorithm Experimental Evaluation Related Work Conclusion 27 IPDPS 2009

  28. Conclusion • Capture How Effectively of Processing a Service on a Node • Efficiency value estimation • Greedy scheduling • Evaluate Our Resource Allocation Approach using Two Adaptive Applications • 32% more benefit comparing to GrADS • Within 10% overhead comparing to GrADS • Our approach is scalable IPDPS 2009

  29. Thank you! IPDPS 2009

More Related