1 / 17

Single Point of Access to Resources of HPC-Europa

Single Point of Access to Resources of HPC-Europa. Krzysztof Kurowski, Jarek Nabrzyski, Ariel Oleksiak, Dawid Szejnfeld Poznań Supercomputing and Networking Center Cracow Grid Workshop, 14 December 2004. Outline. HPC-Europa JRA2: Single Point of Access Assumptions & Requirements

overton
Download Presentation

Single Point of Access to Resources of HPC-Europa

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Single Point of Access to Resources of HPC-Europa Krzysztof Kurowski, Jarek Nabrzyski, Ariel Oleksiak, Dawid Szejnfeld Poznań Supercomputing and Networking Center Cracow Grid Workshop, 14 December 2004

  2. Outline • HPC-Europa • JRA2: Single Point of Access • Assumptions & Requirements • Architecture • Components of the SPA • Job Submission Interface • Conclusion

  3. HPC-Europa • HPC-Europa - Pan-European Research infrastructure on High Performance Computing for the Science of 21st century • Goal: to provide advanced computational services in an integrated way to the European research community • Budget: ~20 mln EURO • 14 Partners across Europe • Project activities • Transnational Access Programme • Networking Activities • Joint Research Activities (JRA1, JRA2)

  4. JRA2: Single Point of Access • Motivation • To provide a uniform access to all centers resources, transparently and regardless of user physical location • Main objectives • ease of use, • improve availability, • global optimization of resources utilization • To achieve these goals • JRA2 builds a HPC-Europa portal providing uniform, flexible and intuitive user access to Grid resources from anywhere • Develops and/or adapts tools and services needed to establish full operational Grid environment • Will develop a meta-broker responsible for users’ access to all consortium’s resources

  5. User-oriented Uniform, transparent access Easy to use, intuitive interface Increased resource availability and access to all centers Provider-oriented Keep centers autonomy To enable it to use its own chosenmiddleware To allow local policies Not to weaken center’s security Optimize global resource usage Assumptions & Requirements

  6. Accounting & Charging Model Center C • Allocation Unit (AU) • 1TFlop sustained for 1 hour • Kept for each resource provider for every single HPC system • Used for 2 purposes: • charging the EC for the use of HPC facilities • limiting the maximum amount of resources a user (or group of users) may utilize • The common pool of AUs concept • If a center contributes n AUs to the global infrastructure, then its users get access to n AUs from a common pool nC AUs nC AUs Common pool of AUs nA AUs nB AUs nA AUs nB AUs Center A Center B

  7. Typical Grid Architecture VO Grid Portal / Grid Resource Broker HPC center HPC center HPC center HPC center

  8. HPC-Europa SPA VO Grid Portal / Grid Resource Broker HPC center Grid Middleware A HPC center HPC center HPC center Grid Middleware C Grid Middleware B Grid Middleware D

  9. HPC Centers Middleware

  10. Architecture of the SPA (1st stage) Authorization Service Accounting System SPA Portlets Job submission Data mngmt Resource Information Workflow Accounting Sofware Repository User Manager GridSphere Framework Information Service Service Plugins Multi-Grid Services JOSH Plugin GRMS Plugin eNanos Plugin UNICORE Plugin GRIA Plugin Globus MDS Plugin … Software Repository Data Management System

  11. Architecture of the SPA Authorization Service SPA Accounting System Portlets Job submission Data mngmt Resource Information Workflow Accounting Sofware Repository User Manager GridSphere Framework Information Service Meta-broker Service Plugins Multi-Grid Services Software Repository JOSH Plugin GRMS Plugin eNanos Plugin UNICORE Plugin GRIA Plugin Globus MDS Plugin … Data Management System

  12. GridSphere • GridSphere:The advanced open-source portlet-based portal framework (www.gridsphere.org) • Features: • portlet-based (supporting JSR168 standard) • customization mechanism for a wide range of end users • a flexible easy to use interface for users • a model to create pluggable and dynamic application support for portal developers • pluggable access to Grid services using the Portlet Services concept • uniform access to various Grid systems using the Grid Portlets model • set of useful core and Grid portlets • open source • Chosen as the HPC-Europa portal on the basis of portals evaluation made

  13. Main Services • Meta-broker • Discovers and choose resources, and submits job to HPC centers • Replica management system • Goal of the HPC-Europa NA3 • Accounting System • Historical information about resource usage (including AUs) • Information System • Central information indexing service • Software Repository • Logical names, meta-data, binaries compiled for various systems, needed environment (preinstalled libraries, environment variables, etc.) • Authorization Service • Authorize users’ request, check resource limits

  14. Multi-Grid Services • Extension of the GridSphere’s Grid Portlets concept by • Interface based on standards and functionality of several Grid systems • Dynamic loading of multiple service implementations (on the basis of a given HPC center) • Remote access • Definition of the common interface • For each functionality, e.g. job submission, resource information etc. • Based on standards where possible (e.g. JSDL) • Taking into account both gathered requirements and the functionality of Grid middleware deployed in HPC centers • Capability check • description of implemented capabilities in the form of the XML document • used to disable not available options (in portal) or to select sites which can accept given job description (meta-broker)

  15. Job submission operations submitJob (UNICORE, GRMS, JOSH, eNANOS, GRIA) cancelJob (UNICORE, GRMS, JOSH, eNANOS, GRIA) getJobInfo (UNICORE, GRMS, JOSH, eNANOS, GRIA) findResources (UNICORE, GRMS, eNANOS) submitJobToBestResource (GRMS, JOSH, eNANOS, GRIA) findBestResource (JOSH) Data elements descriptions Job Description XML schema Job Information XML schema Auxiliary operations getMiddlewareName getMiddlewareDesc getMiddlewareCaps Job states FINISHED, FAILED, PENDING, QUEUED, RUNNING, CANCELLED, SUSPENDED, WAITING Job Submission Interface (JSI)

  16. Job Description • Based on the GGF Job Submission Description Language (JSDL) specification • Subset of JSDL defines the common interface • The specification is still evolving

  17. Conclusion • Benefits of the SPA • Practical: • enables scientists to solve their real problems using HPC resources distributed across Europe in a uniform and easy way • Research: • interoperability of various grid technologies and middleware, • hierarchical model of Grid middleware • Open issues • Interoperability of different security models • Global policies concerning usage of AUs by HPC-Europa users

More Related