1 / 28

Transparent Adaptive Resource Management for Middleware Systems

Transparent Adaptive Resource Management for Middleware Systems. Jaiganesh Balasubramanian Ossama Othman Dr. Douglas C. Schmidt {jai, ossama, schmidt}@dre.vanderbilt.edu. Department of Electrical Engineering and Computer Science Vanderbilt University, Nashville. Distributed Systems.

bena
Download Presentation

Transparent Adaptive Resource Management for Middleware Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Transparent Adaptive Resource Management forMiddleware Systems Jaiganesh Balasubramanian Ossama Othman Dr. Douglas C. Schmidt {jai, ossama, schmidt}@dre.vanderbilt.edu Department of Electrical Engineering and Computer Science Vanderbilt University, Nashville

  2. Distributed Systems • Typical issues with distributed systems • Heterogeneous environments • Concurrency • Large bursts of client requests • 24/7 availability • Stringent QoS requirements • Examples of distributed systems • E-commerce • Online trading systems • Mission critical systems

  3. Motivation • Development and maintenance of QoS-enabled distributed systems • Non-trivial • Requires expertise that application developers often lack • Solution: Middleware (e.g. CORBA) • Can shield distributed system developers from the complexities involved with developing distributed applications • Can facilitate manipulation of QoS requirements and management of resources

  4. Load Balancing • Load balancing can improve the efficiency and scalability of a distributed system • Load balancing service can be implemented in the following layers • Network layer • OS layer • Middleware layer • Why choose middleware-layer load balancing? • Can take into account distributed system state • Can take into account the system run-time behavior • Application level control over load balancing policies • Can take into account request content

  5. Common Scenario • Multiple clients making request invocations • Potentially non-deterministic • Duration called a session • Members • Multiple instances of the same object implementation • Object groups • Collections of members among which loads will be distributed equitably • Logically a single object • Load balancer • Transparently distributes requests to members within an object group

  6. TAO Load Balancer (Cygnus) • Cygnus makes it easier to develop distributed applications by providing • application transparency, • scalability, • flexibility, • adaptability and • interoperability. • Has the following built-in strategies: • RoundRobin • Random • LeastLoaded • LoadMinimum EVENT SERVICE SECURITY TRANSACTIONS AV STREAMS DYNAMIC/STATIC SCHEDULING LOAD BALANCING RT-CORBA 1.0

  7. Cygnus Load Balancing Strategies • Client binding granularity Per-session, client permanently forwarded to a replica Per-request, requests forwarded on client’s behalf On-demand, client can be rebound to another replica when necessary

  8. Cygnus Load Balancing Architectures • Load balancing architecture comprised of a combination of client binding granularity and balancing policy • Given the strategies just described, there are six possible architectures • Three common architectures • Non-adaptive per-session • Adaptive per-request • Adaptive on-demand

  9. Cygnus Load Balancer Components • Load Monitor • Provides load feedback • Load Analyzer • Determines location and member load conditions • Member Locator • Binds client to appropriate object group member • Load Alert • Facilitates load control (load shedding) • Load Manager • Mediates all interactions between load balancer components

  10. Adaptive Load Balancing Interactions

  11. Load Balancing Strategies (1/2) • Adaptive Load Balancing Strategies • Use run-time system information, example host load information to make load balancing decisions. • Adaptive strategies integrated in Cygnus are • LeastLoaded, which allows locations to receive loads until a threshold value is reached. Once a threshold is reached , subsequent requests are transferred to the location with the lowest load. • LoadMinimum, which calculates the average load at all locations and if the load at a particular location is greater than the average load and greater than the least loaded location by a certain threshold, then the load at the particular location is transferred to the least loaded location.

  12. Load Balancing Strategies (2/2) • Non-adaptive load balancing strategies • Makes load balancing decisions based on some static information. • Do not use run-time system state information while making load balancing decisions. • Non-adaptive load balancing strategies integrated into Cygnus are RoundRobin and Random. • RoundRobin strategy keeps a list of locations and iterates through that list to select the next location that will receive requests from the client session. • Random strategy keeps a list of locations and selects a random member to receive the requests from the next client session.

  13. Load Metrics • Cygnus supports the following load metrics • Request-per-second, which calculates the average number of requests arriving per second at each server. • CPU run queue length, which returns the load as the average number of processes waiting in the OS run time queue over a configurable time period in seconds, normalized over the number of processors. • CPU utilization, which returns the load as the CPU usage percentage.

  14. Evaluating load balancing strategies • Load balancing strategies can be of the type adaptive or non-adaptive • Load balancing strategies can be employed various run time parameters and load metrics. • Determining the appropriate load balancing strategies for different classes of distributed applications is hard without the guidance of comprehensive performance evaluation models, systematic benchmarks and empirical results.

  15. Introducing LBPerf • LBPerf is an open source benchmarking tool suite for evaluating middleware load balancing strategies. • Extensible C++ middleware framework. • Supports range of adaptive and non-adaptive load balancing strategies. • Tune different configurations of middleware load balancing , including choosing different load balancing strategies and run-time parameters associated with those strategies. • Evaluate strategies using different metrics like throughput, latency, CPU utilization.

  16. Workload Characterization (1/2) • Empirical evaluations not useful, unless the workload is representative of the context in which the load balancer is deployed. • Workload model could be of the type: • Closed analytical network models • Simulation models • Executable models • Our benchmarking experiments are based on the executable workload models.

  17. Workload Characterization (2/2) • Executable models can be classified as : • Resource type, where workloads are characterized by the type of resource being consumed. • Service type, where the workloads can be characterized by the type of service being performed on behalf of the clients. • Session type, where the workloads are characterized by the type of requests initiated by a client and serviced by a server in the context of a session. • Our benchmarking experiments focused on generating session type workloads.

  18. Experiment • Single threaded clients generating CPU intensive requests on the servers • Experiments repeated for different strategies including RoundRobin, Random, LoadMinimum and LeastLoaded. • Measure throughput, which is the number of requests processed per second by the server and the CPU utilization, which is the CPU usage percentage.

  19. Run-time configurations • Load reporting interval, which is the time interval between successive load information updates from the server to the load balancer. • Reject threshold value of the LeastLoaded strategy, which is the load at which the servers will stop receiving extra loads. • Critical threshold value of the LeastLoaded strategy, which is the load at which the servers will start shedding loads. • Migration threshold value of the LoadMinimum strategy, which is the difference between the most loaded machine and the least loaded machine to trigger a migration of load from the most loaded machine to the least loaded machine. • Dampening value, which is the fraction of the newly reported load that will be considered for fresh load balancing decisions.

  20. Results Analysis • Our results indicate that: • Adaptive load balancing strategies perform better than non-adaptive load balancing strategies in the presence of non-uniform loads. • LeastLoaded strategy is better than the other three strategies under such loading conditions • Reducing the number of client migrations is necessary for achieving maximum performance. In other words, client session migrations are not always effective. • Need to maintain system utilization to enable more predictive behavior of the strategies.

  21. Future Work • Extend LBPerf to evaluate other types of workloads. • Use the empirical results as a learning process to understand the nuances of the different run-time behaviors of these adaptive load balancing strategies. • Use observations from the learning process to train an operational phase by developing self-adaptive load balancing strategies that can dynamically tune the run-time parameters according to the load being experienced. • Define stateful load balancing model • Decentralized load balancing • Reduced network overhead • Improved reliability • Integrating load balancer service into the CCM model.

  22. Concluding Remarks • Cygnus load balancing architecture and model is • Flexible • Extensible load balancing strategies • Allows both non-adaptive and adaptive (including self-adaptive) strategies to be used • Freedom to implement in a variety of ways • Centralized • Decentralized / Federated / Cooperative • Hierarchical • Generic • Supports multiple object groups • Not application-specific • Familiar • Uses many of the same group management concepts found in existing fault tolerance models

  23. References • Ossama Othman, Jaiganesh Balasubramanian, Douglas C. Schmidt “ The Design of an Adaptive Load Balancing and Monitoring Service” http://www.dre.vanderbilt.edu/~jai/IWSAS_2003.pdf • Jaiganesh Balasubramanian, Ossama Othman, Douglas C. Schmidt “Evaluating the performance of middleware load balancing strategies” http://www.dre.vanderbilt.edu/~jai/EDOC_2004.pdf • Download Cygnus, LBPerf and TAO from http://deuce.doc.wustl.edu/Download.html

More Related