An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers

1. An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers TS10 – Service Applications Thomas Sandholmsandholm@pdc.kth.se Olle Mulmo Peter Gardfjäll Erik Elmroth Lennart Johnsson

2. ? Key Question How do we share national Grid compute resources in a fair, secure, open, and scalable way

3. Outline • Requirements on Software Qualities • Open Grid Services Architecture • SweGrid National Grid Testbed • SweGrid Accounting System • Results • Lessons Learned • Q & A

4. SecureOperation FLEXIBILITY INTER-OPERABILITY  Maximum Utilization Fair Distribution Scalable Efficiency Fairness vs. Utilization • Trade-off: Fair resource distribution and optimal resource utilization • Soft real-time quota enforcement • User preferences • Resource policies • Allocation authority policies

5. SecureOperation FLEXIBILITY INTER-OPERABILITY  Maximum Utilization Fair Distribution Scalable Efficiency Security vs. Scalability • Integrity & Privacy • Single Sign-on/Impersonation • DoS/Replay Attack prevention • Privilege Delegation • Message Level vs. Transport Level • Policy Driven Authorization: PDP, PAP, PIP, PEP Scale: • National Grid • No single point of failure but coordinated allocationenforcement

6. Openness & Interoperability • Systems Integration Platform • Scheduler/Workload Manager Agnostic • Programming Language/Model Agnostic • Portable (100% pure Java) • XML Based Standards: XPath, XQuery, XSLT, GGF-UR, XML-Signature, XML-Encryption, XACML • Web/Grid Services Standards: SOAP, WSDL, WS-Security, OGSA, GGF-UR, GSI, GSSAPI, OGSI/WSRF

7. Outline • Requirements on Software Qualities • Open Grid Services Architecture • SweGrid National Grid Testbed • SweGrid Accounting System • Results • Lessons Learned • Q & A

8. Open Grid Services Architecture • Global Grid Forum Standardization Initiative • Architecture extending SOA and WSA to dynamically share stateful resources across organizational boundaries (=realizing the Grid vision) • “… defining, within a service-oriented architecture, a set of core capabilities and behaviors that address key concerns in Grid systems.” OGSAv1 • Assumes state modeling according to OGSI/WSRF • Core Infrastructure offering Inspection, Discovery, Lifetime Management, Notifications, Fault Handling • WS-Resource = stateful resource and associated Web service. • Provide context for message exchange • Addresses Grid security requirements such as Delegation and Single Sign-On

9. Role of Accounting in OGSA Accounting foundational service to: • Job Execution • Make sure that only jobs with sufficient quota can be executed on the compute resource • Decide queue priority based on available funds and usage history • QoS/SLA Management • Negotiate pricing based on resource usage • Optimizing Utilization • SLA Attainment/Policing • Security • Auditing • Access Control • PEP/PIP

10. Outline • Requirements on Software Qualities • Open Grid Services Architecture • SweGrid National Grid Testbed • SweGrid Accounting System • Results • Lessons Learned • Q & A

11. SweGrid • SweGrid connects 600 compute nodes (Intel P4) across 6 Swedish HPC centers interconnected by 10Gbs GigaSunet network • 400 HPC users at all centers (some overlapping) • Inaugurated March 2004 • ~50 currently active researchers • Up to 10k jobs per month per site

12. SweGrid Continued • Resource quotas allocated by Swedish NationalAllocations Committee (SNAC) after peer-review of promising research projects with high computational demands (c.f. NRAC) • Initially homogeneous hardware but heterogeneous scheduling, security, and accounting environment (policies, tools, data, processes, etc) • Wanted: Uniform resource quota use & allocation

13. Outline • Requirements on Software Qualities • Open Grid Services Architecture • SweGrid National Grid Testbed • SweGrid Accounting System • Results • Lessons Learned • Q & A

14. SweGrid Accounting System (SGAS) Key Design Points • Decentralized accounting solution based on standard, open protocols in compliance with the proposed OGSA • 3-party (user, resource, allocation authority) policy customization • Non-intrusive to local site accounting systems • All components governed by a scalable cross-organizational authorization framework

15. SGAS Component Overview PAT AddUser WSDL WSDL Query SubmitJob Bank LUTS Reserve/Release PublishUR WSDL WSDL JARM User Broker Resource Manager Resource Scheduler Policy Administration ToolLogging and Usage Tracking ServiceJob Account Reservation Manager

16. SGAS Security Design XML-Signature XML-Encryption PAT External Authorization Service PDP PIP Membership/CommunityService Bank PAP PDP LUTS PIP PAP WS-SecureConversation PEP JARM PDP PIP Site Policy Manager User Broker Resource Manager PKI Resource Scheduler Kerberos Policy Administration PointPolicy Decision Point Policy Information Point Policy Enforcement Point Credential Delegation

17. Outline • Requirements on Software Qualities • Open Grid Services Architecture • SweGrid National Grid Testbed • SweGrid Accounting System • Results • Lessons Learned • Q & A

18. Overdraft XACML Policy <Condition FunctionId= "urn:oasis:names:tc:xacml:1.0:function:integer-less-than-or-equal"> <Apply FunctionId= "urn:oasis:names:tc:xacml:1.0:function:integer-one-and-only"> <EnvironmentAttributeDesignator AttributeId= "sgas:overdraw:percent:requested" DataType= "http://www.w3.org/2001/XMLSchema#integer"/> </Apply> <AttributeValue DataType= "http://www.w3.org/2001/XMLSchema#integer"> 175 </AttributeValue> </Condition>

19. Overdraft Fuzzy Logic Policy R1: overdraft is low allocation left is muchallow reservation R2: overdraft is high allocation left is littledisallow reservation R3: allocation proximity is soon overdraft is high allocation left is muchallow reservation R4: allocation proximity is soon overdraft is low allocation left is littleallow reservation

20. Super Computing 2004 Demonstration

21. Outline • Requirements on Software Qualities • Open Grid Services Architecture • SweGrid National Grid Testbed • SweGrid Accounting System • Results • Lessons Learned • Q & A

22. Conclusions • Document centric communication in conjunction with semi-structured native XML databases is a very flexible combination • Batch charging and eager prepare reservation needed for scalability • Timestamp based allocations distributed in a staggered monthly flow result in the best trade-off between fairness and utilization • Generic PEP/PDP/PIP/PAP model useful for encapsulating and evolving authorization code • OGSI/WSRF state management ideal for controlling fine grained service state such as account quotas, reservations and policies in a standard way

23. Future Work • With large-scale flexibility and configurability comes complexity and it becomes hard to optimize high-level goals and to realize detailed user QoS requirements – development of an SLA Management framework and user/resource goal driven optimizing agents (WS-Agreement, ContractNet) • Initial focus has been on scientific community resource sharing - support economic brokering and for-profit banks • Multi jobs may overload the bank - SAML assertions (c.f. cheques) as a multi-allocation payment and reservation method

24. Learn more… http://www.sgas.se http://www.swegrid.se

25. Outline • Requirements on Software Qualities • Open Grid Services Architecture • SweGrid National Grid Testbed • SweGrid Accounting System • Results • Lessons Learned • Q & A