1 / 28

The VOCE Environment EGEE & SEEGRID Summer School

The VOCE Environment EGEE & SEEGRID Summer School. Lud ěk Matyska CESNET. Outline. The Virtual Organization Concept The middleware components Access control and authentication VOCE in detail. Virtual Organization Concept. Grid – heterogeneous set of resources and users

odin
Download Presentation

The VOCE Environment EGEE & SEEGRID Summer School

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The VOCE EnvironmentEGEE & SEEGRID Summer School Luděk Matyska CESNET

  2. Outline • The Virtual Organization Concept • The middleware components • Access control and authentication • VOCE in detail EGEE & SEEGRID Summer School, July 3rd, 2006

  3. Virtual Organization Concept • Grid – heterogeneous set of resources and users • Different requirements and expectations • Some internal organization needed • Virtual Organization • Set of resources • Computing elements • Storage elements • Set of users • Services • Policies • Who is allowed to join • What the resources should offer/guarantee EGEE & SEEGRID Summer School, July 3rd, 2006

  4. VOs in EGEE • EGEE accessible through VOs only • Original idea – Application specific VOs • Complex approval process • Heavy weight setup process • Difficult for not too well organized communities • VO for Central Europe (VOCE) • Regional VO • Application neutral • Setup by CE partners • No direct end users’ involvement required • Available for all CE users EGEE & SEEGRID Summer School, July 3rd, 2006

  5. Middleware Overview • Middleware as a glue • Basic components • User interface • Resource Broker • Computing Elements • Storage Elements • Information and Monitoring Services EGEE & SEEGRID Summer School, July 3rd, 2006

  6. User Interface • The primary input point into a Grid • Batch processing as a primary way of use Grid • Support for job submission • Input sandbox preparation • The executable (name or binary) • (Small) input files • Job environment • Job description • JDL (Job Description Language) • Assigns global job identifier (JobID) to each job EGEE & SEEGRID Summer School, July 3rd, 2006

  7. UI in VOCE • Primary: skurut4.cesnet.cz • Access through the gsissh • Authentication through certificate • Each user does have an account • The only place with a user specific account on Grid • More users interfaces can be added • Specific setup • Load sharing • The job submission may be a heavy load task EGEE & SEEGRID Summer School, July 3rd, 2006

  8. Resource Broker • Key component of the Workload Management System (WMS) • Assigns jobs to resources • The matchmaking process • Combines user requirements (JDL) with advertised properties of computing elements and • Takes into account data placement • Needs actual information from the Grid infrastructure • Supports automatic resubmission • If allowed by the user for a particular job EGEE & SEEGRID Summer School, July 3rd, 2006

  9. Resource Broker in VOCE • Primary: skurut3.cesnet.cz • Users do not directly interact with RB EGEE & SEEGRID Summer School, July 3rd, 2006

  10. Computing Elements • The infrastructure components • Provided by resource owners • An abstraction of a Grid resource • Could be a single machine or a cluster (even a Grid) • Computing element is represented by one or more queues • Resource Broker submits jobs to CE queues • WMS keeps track of jobs • Condor-G based • Can be shared between VOs • Actual work/job processing done by Worker Nodes EGEE & SEEGRID Summer School, July 3rd, 2006

  11. Computing Elements in VOCE • CE institutions provide individual resources • List • ce.grid.tuke.sk • ce.polgrid.pl • grid109.kfki.hu • ares02.cyf-kr.edu.pl • zeus02.cyf-kr.edu.pl • skurut17.cesnet.cz • ce.egee.man.poznan.pl • 999 CPUs available • Easy to add new resources if needed • Plan to have most CE resources also on VOCE EGEE & SEEGRID Summer School, July 3rd, 2006

  12. Storage Elements • EGEE Grid started within the DataGrid project • Data and their storage is becoming more important then simple number crunching • Storage Element as a primary data deposit • An abstraction • May be a file server, an ftp deposit or a complex hierarchical storage system • Particular access method • Large capacity • Long term/permanent storage • However, beware of individual SE policies EGEE & SEEGRID Summer School, July 3rd, 2006

  13. Storage Elements in VOCE • Provided by the CE institutions • List • ce.grid.tuke.sk • se.polgrid.pl • grid100.kfki.hu • ares03.cyf-kr.edu.pl • zeus03.cyf-kr.edu.pl • skurut18.cesnet.cz • se1.egee.man.poznan.pl • 13TB available • Easy to add more EGEE & SEEGRID Summer School, July 3rd, 2006

  14. Job Tracking • Jobs could be lost – a tracking system is a necessity • Logging and Bookkeeping service • Collects events produced by middleware components • Process them into a Job State • Provides information about a job state to end users • Address of a job appropriate LB server is part of the JobID • LB assigned during the job submission • When JobID is generated • LB in VOCE: skurut3.cesnet.cz • However, more LBs can be easily added EGEE & SEEGRID Summer School, July 3rd, 2006

  15. Authentication • Grid must recognize individual end users • However, it is impossible to register individually each end user at each CE and SE • Uses PKI (Public Key Infrastructure) • Based on asymmetric cryptography • End users’ public keys signed by Certification Authority (CA) – the Certificate • Set of accepted (trusted) CAs • Each CE/SE can decide not to accept some CAs • VOs can define sets of accepted CAs • Each user with a valid certificate from accepted CA can use resources EGEE & SEEGRID Summer School, July 3rd, 2006

  16. MyProxy • Users need private key plus certificate to access resources • Usually long validity (one year) • Copying private key to each UI/RB/CE opens door to key compromise (theft) • Proxy certificates • Short term restricted key/certificate (usually 8 hours) • Used instead of full certificate to access Grid • MyProxy service • Deposit of long term certificates/keys • Users access MyProxy Server (with login/password) and create the proxy certificate • Renewal service • The Grid must be able to renew proxy certificates • Use of MyProxy plus valid proxy certificate EGEE & SEEGRID Summer School, July 3rd, 2006

  17. VOCE Access • VOCE accepts EGEE accepted CAs • Each country has at least one accepted CA • Registration process rather strict • High level of trust • Resources can accept also other CAs • On-line CA (federative registration) • t-CA (training) • MyProxy at skurut4.cesnet.cz • Proxy certificate creating the first step to access a Grid EGEE & SEEGRID Summer School, July 3rd, 2006

  18. VOCE Registration • Users • Must register at the http://voce-register.farm.particle.cz/ • Must possess a valid certificate from accepted CA • Must agree to the VOCE policies • Must belong to CE institution • No other requirements • Resources • A negotiating process • Easy if EGEE certified resource EGEE & SEEGRID Summer School, July 3rd, 2006

  19. VOCE Goals • To provide complete and self-sustained Grid infrastructure based on the EGEE/gLite middleware • VOCE spans the whole Central Europe • Primary services operated and maintained by CESNET • To provide application neutral environment for CE users • Even small groups can access EGEE production Grid • No need to invest into specific VO before testing and using the EGEE/gLite Grid environment • Available for production runs, not only a training infrastructure EGEE & SEEGRID Summer School, July 3rd, 2006

  20. Regional Principle • VOCE is a regional VO • The first regional VO in EGEE • Currently this model accepted for all regions • VOCE officially registered in the list of EGEE VOs • Fully production environment • Quick first experience with Grid computing for Grid newcomers • Smooth transition to production use • Support for preparation of specific VOs • When users get more organized with more experience EGEE & SEEGRID Summer School, July 3rd, 2006

  21. Specific Activities • NA3: Training infrastructure • Fully production environment • What users will see during training they will use afterwards • Needs to solve the authentication/access for users without a valid long term certificate • NA4: Application support • VOCE is application neutral • Support for generic set of applications • Extensible EGEE & SEEGRID Summer School, July 3rd, 2006

  22. Specific Activities • SA1: Operations • Research and development of lightweight setup of VOs and associated services • Support for new application-specific VOs • Together with end users and their communitites EGEE & SEEGRID Summer School, July 3rd, 2006

  23. CE specific enhancements P-GRADE portal • Portal access to the grid infrastructure • Multi-grid support • Charon • Command line interface for job control • Other components could be addedd EGEE & SEEGRID Summer School, July 3rd, 2006

  24. Charon System • Uniform and modular approach for complex comfortable computational jobs control • submission, monitoring, output retrieval • Supported Programs • Computational chemistry • Turbomole, Gaussian • Molecular modeling • Amber • Conversion and analysis • Visualization • Povray, raster3D, molscript • Extensible by new programs EGEE & SEEGRID Summer School, July 3rd, 2006

  25. VOCE support • Extensive documentation • VOCE portal at http://egee.cesnet.cz/en/voce/ • VOCE help desk/request tracking • Direct through e-mail to voce@cesnet.cz • VOCE dedicated queue in the CESNET RT system • Also CE ROC helpdesk and GGUS could be used • People • CESNET team • Includes gLite developers • Charon • SZTAKI • P-GRADE portal • Resource administrators EGEE & SEEGRID Summer School, July 3rd, 2006

  26. VOCE Use • (Almost) exponential increase in number of users • Primary for “testing the Grid” • Provides full production level Grid environment • Users are not required to install middleware on their systems/machines not to maintain it • Reliability and sustainability challenges • Reasonable good results • Plans for Compute challenges • Test setup of a new VO including • Use the accumulated VOCE resources for a serious scientific work EGEE & SEEGRID Summer School, July 3rd, 2006

  27. Summary • Fast and easy access to the EGEE Grid • Easy access to newcomers • Around 130 users registered • Support production and experimental runs • Environment where CE scientists can meet and collaborate • Extensible with CE specific components • Open for new applications • Service available for all CE potential users • A catch-all model currently accepted EGEE-wide EGEE & SEEGRID Summer School, July 3rd, 2006

  28. Questions? Thank you EGEE & SEEGRID Summer School, July 3rd, 2006

More Related