html5
1 / 19

CERN-INTAS project No. 03-52-4297 Subtask 6.2:

CERN-INTAS project No. 03-52-4297 Subtask 6.2: OGSA/Globus evaluation for data intensive applications by A. Demichev (SINP-MSU) ‏. Objectives & executed work. The subtask objectives : An evaluation of new OGSA/WS(RF)-based Grid technology considered as a candidate for further stages of LCG

selah
Download Presentation

CERN-INTAS project No. 03-52-4297 Subtask 6.2:

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CERN-INTAS project No. 03-52-4297 Subtask 6.2: OGSA/Globus evaluation for data intensive applications by A. Demichev (SINP-MSU)‏

  2. Objectives & executed work The subtask objectives : An evaluation of new OGSA/WS(RF)-based Grid technology considered as a candidate for further stages of LCG Tested/evaluated Toolkits & MW components potentially useful for gLite OMII Globus Toolkit 4 (GT4)‏ gLite components The subtask participants: M. Lamanna (CERN IT)‏ V. Pose (JINR)‏ A. Demichev (SINP-MSU)‏ L. Shamardin (SINP-MSU)

  3. Objectives & executed work (2)‏ In collaboration with ARDA team (CERN)‏ KIAM RAS (Moscow)‏ JINR (Dubna)‏ JRA1 team (CERN)‏ OMII & GT4 teams Russian participants visited CERN for works related to the subtask (~ 6 PM)‏

  4. OMII testing/evaluation (while GT4 was being prepared for release...) Testing/evaluation of OMII (www.omii.ac.ru) MW was interesting for JRA1 from the point of view of potential use of some its components for EGEE MW The goal of the evaluation was to realize: how well the components of the OMII MW meets the needs of a large grid environment with ten thousands of users, tens of big virtual organizations, hundreds of resource centres, ten thousands of jobs, Peta bytes of data etc.

  5. OMII testing/evaluation (2)‏ Description of the functionality provided by the MW Installation and configuration of the MW on different platforms Testing of performance, concurrency, reliability, usability and management of the services Job Services Data Services Security Adding an application Adding a new service Performance and concurrency of dummy services Interoperability with gLite Workload Management System (WMS)‏ Quality of OMII Support and Documentation

  6. OMII: general features OMII has several interesting features Web services architecture Account management Resource consumption accounting Conversation mechanism and authorization module PBAC Process Based Access Control Easy and compact installation (but restricted to the certain OS) The provided documentation clearly covers the main topics, but at some points (e.g. budget management, service creation) it was not clear and comprehensive enough

  7. OMII: Some results on performance, concurrency, reliability and usability Job Service remained stable with up to 20 concurrent clients a maximal job submission rate of about 6 jobs/min for a 2.4GHz P4 CPU server node stability: in case of sequential submission of multiple jobs by a user an error occurred near the 180-th job submission Data Service reliability: all 1000 upload + download cycles of 10MB big files were successful concurrency: all upload + download cycles of 1..100KB big files were successful with up to 5 concurrent clients Dummy Services response time for a non-PBAC dummy service - about 0.1sec with PBAC about 0.7sec Concurrency - 100 parallel clients executed successfully Adding simple applications is easy A simple new service has been easily deployed

  8. OMII: Evaluation results OMII is oriented towards web services rather than Grid architecture There are no Workload Management System & Resource Broker, Information System, Logging and Bookkeeping, Replica Catalogs and Replica Management System OMII does not support GSI and proxy credentials Virtual Organizations management

  9. OMII: Evaluation results (2)‏ No interoperability with Globus Toolkit and the gLite WMS Management and administrative techniques are intended to servicing individual servers and small projects but not large VOs or extended resource infrastructure More details on OMII evaluation: http://lit.jinr.ru/Reports/annual-report05/omni-55.pdf http://theory.sinp.msu.ru/~demichev/omii/OMII_evaluation-EGEE3.ppt

  10. OMII's plans... OMII Technical Roadmap (mid. 2005):“We will collaborate with EGEE (in particular CERN) to ensure that uses of the EGEE software (in particular gLite) will be able to have solutions that interoperate with OMII hosted components.”

  11. Globus Toolkit 4 (GT4) Testing GT4.0.0 was installed and properly configured on a number of platforms (in total on 7 machines): Fedora Core 3 – using the Fedora Core 3 Binary Installer Redhat 9 – using theRedHat 9 Binary Installer + Patches from http://www.globus.org/toolkit/downloads/4.0.0/ SLC3 (Scientific Linux Cern Release 3.0.4) - using the Source Instsaller SL4 (Scientific Linux Release 4.0) - using the RPMs file from ftp://ftp.linux-ink.ru/pub/SL/40/projects/GRID+HEP

  12. GT4 studying, testing and evaluation WSRF implementations comparison: GT4-Java (Java WS Core of GT4); GT4-C (C WS Core of GT4); pyGridWare (Python WS Core); the Perl-based WSRF::Lite; Functional comparison: Significant commonalities are seen with regard to dispatching and SOAP processing techniques, though in each implementation they are different

  13. GT4 studying, testing and evaluation (2)‏ All the GT4 WSRF implementations Security: support three security protocols: TLS/SSL transport-level security protocol; Secure Conversation SecureMessage. WSRF::Lite supports only transport-level security. Persist WS-Resources in memory by default and come with modules that allow resources to be saved to disk, providing the ability to survive server failure at the cost of some performance. Lifetime Management includes resource creation and resource destruction.

  14. GT4 studying, testing and evaluation (3)‏ WS-Notification. GT4-Java and pyGridWare do not implement WS-Brokered Notification and have some other limitations. GT4-C does not implement producer-side notification. WSRF::Lite does not support any Notification specifications. Authorization. GT4-Java and pyGridWare allows the service developer to provide custom authorization. GT4-C implements three built-in mechanisms. WSRF::Lite client passes security information to the WS-Resource through environment variables, leaving the WS-Resource implementation to implement its own authorization.

  15. GT4 studying, testing and evaluation (4)‏ Interoperability. Being consistent with the WS-Interoperability, the five implementations achieve a base level of interoperability with regard to XML, HTTP, SOAP, and WSDL. But the implementations, nevertheless, are not fully interoperable. Namespace incompatibilities are the key concern, as well as several technical details (HTTP headers, application-specific portions of messages) which lies outside the WSRF/WSN specifications). By now: OASIS has accepted the WSRF 1.2 (WS-Resource, WS-ResourceProperties, WS-ResourceLifetime, WS-ServiceGroup, WS-BaseFaults) as an OASIS standard.

  16. GT4 Testing Estimation of the overhead expenses of GT4 Java container overhead expenses of GT4 Java container (i.e. first part which is common for invocation of all services) and client application. Estimation of average times for common WSRF operations average times for common WSRF operations (in GT4 Java container) on server side and client side and investigation the influence of the number of created resources on the request time. Estimation of GT4 Java container scalability and robustness GT4 Java container scalability = maximum possible number of parallel invocations before server failure). GridFTP and RFT testing Further details and numerical results of the testing are available at http://theory.sinp.msu.ru/~demichev/GT4evaluation.doc

  17. gLite – GT4 WSS compatipility Studying of the gLite – GT4 WorkSpace Service (WSS) compatibility was carried out by special request from JRA1. WSS should allow to enforce VO politics in a way which does not contradict to local sites politics. The goal was to verify compatibility of WSS with gLite and evaluate the service Succeeded in combining gLite+WSS At the time of the study (Fall of 2005) WSS proved to be too immature and unstable to be considered as real candidate for gLite component.

  18. Comparative studying: GT4 - gLite In each main area gLite has services, which are absent in GT4, but are substantial: Security. Virtual organization support (VOMS gLite); Execution Management - Workload Management System; Data Management - support of storage resource internal organization: SE, SRM, file catalogues R-GMA (gLite) vs. MDS4: essentially different approaches: MDS4 - hierarchical R-GMA - relational

  19. Glite Testing The subtask team participated in testing gLite components: Fireman (file + replica) and Metadata catalogshttp://lcg.web.cern.ch/lcg/PEB/arda/public_docs/CHEP04/CaseStudies/Fireman1.doc IO-serverhttp://lcg.web.cern.ch/lcg/PEB/arda/public_docs/CHEP04/CaseStudies/gLiteIO%20-%20testing-v2.pdf Workload Management Systemhttp://lcg.web.cern.ch/lcg/activities/arda/public_docs/2005/Q1/ARDAatTaiwan-apr05.ppt AMGA Metadata Service(V.Pose's report at GRID'2006 on 28.06)‏ The work was carried out in ARDA framework (partially supported by the INTAS-CERN project) and in close collaboration with JRA1 Team

More Related