IBM Initiatives in Autonomic Computing and Policy

1. IBM Initiatives in Autonomic Computing and Policy Alan Ganek Vice President, Autonomic Computing

2. Today’s Complex Infrastructure Business Data UNIX Mainframe PCs Web Servers SSL Appliances Security & Directory Servers Database Servers Routers Switches Application Servers UNIX PCs Firewall Servers Caching Appliances DNS Servers File/Print Servers UI Data LAN Servers

3. IT Challenges Missing or Loss of critical data is immeasurable Up to 40% of today’s outages are unscheduled stoppages Complex, heterogeneous environments Poorly documented legacy applications make it painful to diagnose and resolve complex cross-product problems 25-50% of IT resourcesare spent on problem determination and resolution The skills needed to do manual cross-product problem determination are scarce and expensive Outages of mission-critical systems cost quite a bit Outages & unscheduled work leads to saturation on backup systems & power systems

4. Autonomic Vision “Intelligent” open systems that: • Manage complexity • Know themselves • Continuously tune themselves • Adapt to unpredictable conditions • Prevent and recover from failures • Provide a safe environment Providing customer value • Increased return on IT investment • Improved resiliency and quality of service • Accelerated time to value Focus on business, not infrastructure

5. Autonomic Computing Attributes Self-managing systems that deliver: Increased Responsiveness Adapt to dynamically changing environments Business Resiliency Discover, diagnose,and act to prevent disruptions Operational Efficiency Tune resources and balance workloads to maximize use of IT resources Secure Information and Resources Anticipate, detect, identify, and protect against attacks

6. Levels of Autonomic Maturity IT Focus Business Focus Element Management On Demand Management Centralized Management • Autonomic, self managed environment • Dynamic provisioning • Automated best practices • Proactive and Predictive • Automated provisioning • Manual processes • Reactive and tactically aligned • Resource intensive Autonomic Adaptive 5 Evolutionary approach; Revolutionary outcome 4 Predictive 3 Managed 2 Basic Basic 1

7. Deliver core infrastructure technologies that provide for an open framework for the industry • Deliver products with built-in autonomic capabilities • Create and leverage open standards for autonomic computing The Big Picture of Autonomic Computing Technology • Define a base reference architecture model which creates a common vernacular for autonomic computing Business policy Workload Management Resource Provisioning Autonomic core capabilities Solution Install Policy Autonomic Computing Architecture Problem Determination Admin Console Products delivering autonomic features Open Standards

8. Effectors Sensors Resource Manager Managed Element Autonomic Computing Architecture Overview Sensors Effectors Analyze Plan Monitor Execute Knowledge Autonomic Manager Action Data Manageability Interface

9. Autonomic control loops: next step evolution BUSINESS SLA POLICY USER RESPONSE TIME AVAIL. RESOURCE Globalenvironmentview and knowledge Autonomic features Local view

10. Sensors Effectors Analyze Plan Monitor Execute Knowledge Autonomic Manager Policy and Autonomic Managers Policy Policy • Policy: a set of considerations designed to guide decisions on courses of actions (Calo, et al.) • In autonomic (self-managing) systems, policies reflect human judgement. • In an autonomic systems, policies are considerations, stored as knowledge, that guide the actions of autonomic managers • Policies can tell an autonomic manager what to achieve (goals) or how to achieve it (actions) Policy Policies allow administrators to configure autonomic systems.

11. Employee Scheduling Employee Scheduling Data and Transaction Servers Appliance Servers Web Application Servers Internet Business Partners Employee Benefits • Workflow -centric management model • Resource allocation based on goal achievement • Service class goal exceptions trigger adjustment algorithms • Predictive algorithms avoid nonproductive adjustments • Business importance determines problem prioritization • Business importantance determines winners and losers Policy in Practice: eWLM Policy Based Business Priority based goals • 90% less than 2 seconds, critical to the business • Avg response time 3 seconds, important, but not critical • Defined by user group and/or business process Dynamic, learned topology controls • No static definitions of servers/application environments

12. Policy in an IT Infrastructure Access control policies Quality of service policies Security policies Utilization policies Business resiliency & data retention policies

13. (Some) Key issues in policy for autonomic computing • Business driven: how autonomic manages can manage against high-level business policies. • Reduce the complexity by managing toward business objectives • Canonical specification: autonomic systems are heterogeneous. • Multiple policy specifications leads to overly complex, fragile systems. • Orchestration: how policies resolve conflicting systems needs. • Transformation, validation, user centered design, and many, many more…

14. Anatomy of an AC Policy: a canonical policy language • Four common concepts identified: • Scope • Specifications to identify what is or is not subject to the intent. • Precondition • Specifications to express when a policy is to be applied or is active. • Business Value • Specifications to express utility functions to make economic trade offs • Decision • Specifications to describe observable behavior or objective. • Designed by adopting concepts from prevalent policy languages • eWLM, IETF/DMTF standard, WS-Policy, WSLA

15. Scope Precondition Business Value Decision Scope Precondition Business Value Decision Scope Precondition Business Value Decision AC Policies and WS-Policy WS-Policy defines a container for assertions. WS-Policy Document AC policies are domain assertions <wsp:policy> AC Policies are assertions contained in WS-Policy documents </wsp:policy>

16. Defining and Executing Policies Observations: • Policies are used by an autonomic manager to control a managed resource. • Therefore, policies must relate to the resource’s sensors and effectors. • At definition time, policies should be associated with a resource type • At run-time, autonomic managers apply the policies appropriate for the types of resource under management The next two charts illustrate these points.

17. Scope: Condition: Business value: Decision: S E Autonomic Manager A P E M K The policy is a function of the MR’s externalized sensors and effectors. A 4-tuple is a type of “K”. The managed resource’s descriptor enumerates a resource’s sensors and effectors Sensor Effector Managed Resource Defining Policies Policy 4-tuple 4-tuple List of Sensors List of Effectors Managed Resource’s descriptor

18. How autonomic managers work with policy Sensor Effector Monitors for events. Deliver to “K””. Autonomic Manager “A” gets data from “K”. Analyze Plan Retrieves state and consults K as needed for policy evaluation. Execute Monitor CBE Knowledge Evaluates the policy conditions and business value CBE Sensor Effector Managed Resource Performs the command as indicated by the policy decision

19. Load Balancer an example – policy-driven self-optimizing solution:IBM Server Allocation for WebSphere High priority Database Server WebSphere Transactional Grid Stock Trading Mid priority Account Manager Low priority Forecaster Advice Application • Multiple WebSphere transactional applications • Multiple Service Level objectives • Dynamic and automated application provisioning Application Provisioning Parallel Services IBM Server Allocation for WebSphere WebSphere Application Server v5

20. Mustang: Objective and Approach • Objective • Maintain service level objectives (e.g. response time) in the face of varying workload • Approach • Configuration management for rapid resource addition/removal • Online capacity planning to estimate resource needs • Policy enabled controller

21. Optimizer Configuration Transition Engine Condition Evaluator Server Agent Notification Data Provider WSLA DB2 8.1 SPEC driver Policy Server pool Policy WAS 5.1 WAS 5.1 WAS 5.1 WAS 5.1 trade driver WAS 5.1 WAS 5.1 DB2 8.1 WAS 5.1 WAS 5.1 WAS 5.1 batch driver Mustang: Policy enabled self-optimizing controller Sensor Effector Autonomic Manager Analyze Plan Monitor Execute Knowledge Retrieve State Command Managed Resource Sensor respTime, thruput Effector Start/Stop Server

22. During daytime (6am -6pm) Trade has priority and is assigned high min #nodes (not shown: SPEC assigned min #nodes = 1) Sample policy on min nodes: determined by time-of-day

23. More sample Mustang time-of-day policies During nighttime (6pm –6am) SPEC has priority and is assigned high min #nodes (not shown: Trade assigned min #nodes=1)

24. { SPEC has 1 WAS nodes SPEC has 5 WAS nodes } Trade has 5 WAS nodes Trade has 1 WAS nodes DAYTIME NIGHTTIME WAS Server deployment transition: night d day

25. Unity system and policy issues • Unity is an autonomic system integration project • Focus issues: • Interactions between autonomic managers • Conflict resolution • Resource allocation • Self-configuration (dynamic discovery and runtime binding) • Self-healing • Policy focus: • Extending policy language and framework to support utility functions • Utility functions enable: • Principled policy decomposition • Conflict resolution • Continual optimization

26. Resource Arbiter Sentinel Application Manager Application Manager Database Registry Policy Repository Database Server Server Demand Router Router Server Storage Unity ArchitectureAC system composed of interacting autonomic components An Autonomic System Application Environment Application Environment Demand . . .

27. Resource Arbiter Application Manager Application Manager Policy Repository Server Server Server Server Server Resource Allocation in Unity (utility-based approach) UI/Editor The utility functions provide a principled basis for resolving the resource contention, and for converting business-value policies into action. The policy repository holds and distributes policies that specify a business value for the behaviors of each application. Scenario: an overall solution contains a number of applications, and resources that must be allocated among the applications. The application managers use these policies to tell the resource arbiter the likely effects of changing allocations. The arbiter allocates the available resources in a way that maximizes the expected business value of the overall solution. Demand Demand . . .

28. S S E E RM RM Unified Policy Management • Standardized approach to enable integration of policy-based solutions • Independent of any product infrastructure • Rich syntax for the definition of policies • Persistence and distribution of policy definitions • Allow policy guidance to be stated in a variety of forms (Results, Actions, and Goals) Decision Federator Decision Federator APIs Policy Editing Tools Decision Federator APIs S E AutonomicManager Decision Maker Policy Definition (Policy Grammar) A P E M K Decision Maker APIs Scope: database Preconditions: Log file size > 100,000 entries Business value: high Decision (Action): incremental backup Managed Element Managed Element

29. Research Challenges Policy: Set of guidelines or directives provided to autonomic element to influence its behavior. • Human Interface • Authoring and understanding policies • Avoiding or ameliorating specification errors • Developing a universal representation and grammar • Many different application domains, disciplines • Many different flavors of policy • Covers service agreements as well ? • Algorithms that operate upon policies (and agreements ?) • Automated derivation of actions (e.g. planning, optimization) • Automated derivation of lower-level policies from high-level policies (e.g. maximize profit from this set of service contracts) • Conflict Resolution • Both design time and runtime • Need to establish protocols, interfaces, algorithms

30. The Journey Has Started • Products, technologies and services are available today • IBM is working with academia, business partners and standards organizations to develop open standards for self-managing systems • Broad IT industry participation is needed – this is an industry-wide initiative • Policy is a critical element and one of the most challenging aspects of AC • Innovation & Collaboration are a must!! Aggressive research is essential!! Freeing people to focus on their business instead of their infrastructure

31. Questions? • Useful info • ganek@us.ibm.com • URLs • www.ibm.com/autonomic • www.research.ibm.com/autonomic • www.alphaworks.ibm.com/autonomic