1 / 26

Cyber Science Infrastructure and NAREGI Grid middleware

Cyber Science Infrastructure and NAREGI Grid middleware. Kento Aida National Institute of Informatics. Outline. overview of Cyber Science Infrastructure (CSI) grid infrastructure in CSI NAREGI Grid Middleware deployment and user support summary. Cyber-Science Infrastructure (CSI).

yvonne
Download Presentation

Cyber Science Infrastructure and NAREGI Grid middleware

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Cyber Science Infrastructure and NAREGI Grid middleware Kento Aida National Institute of Informatics Kento Aida, National Institute of Informatics

  2. Outline • overview of Cyber Science Infrastructure (CSI) • grid infrastructure in CSI • NAREGI Grid Middleware • deployment and user support • summary Kento Aida, National Institute of Informatics

  3. Cyber-Science Infrastructure (CSI) • new information infrastructure to boost today’s advanced scientific research • integrated information resources and systems • high-performance computing/storage resources • software • databases and digital contents • “human” and research processes themselves • services • networking : SINET3 • federated ID management : UPKI • grid : NAREGI Kento Aida, National Institute of Informatics

  4. SINET3: Lambda-based Academic Networking Backbone Hokkaido-U ★ ● ★ Tohoku-U Kyoto-U ☆ ★ ★ ★ Tokyo-U Kyushu-U ★ NII Nagoya-U ★ Osaka-U (Titech, Waseda-U, KEK, etc.) CSI (cont’d) NII-REO (Repository of Electronic Journals and Online Publications GeNii (Global Environment for Networked Intellectual Information) Virtual Labs Live Collaborations NAREGI: National Research Grid Initiative UPKI: National Research PKI Infrastructure International Infrastructural Collaboration Industry/Societal Feedback Kento Aida, National Institute of Informatics

  5. Network (SINET3) http://www.sinet.ad.jp/ Kento Aida, National Institute of Informatics

  6. SINET3 International Connectivity http://www.sinet.ad.jp/ Kento Aida, National Institute of Informatics

  7. UPKI Sign, Encrypt. NII Pub CA Other Pub CA Open Domain PKI Webサーバ S/MIME Webサーバ S/MIME Webサーバ S/MIME Webサーバ S/MIME Web Srv. S/MIME Web Srv. S/MIME Auth, Sign, Encrypt. Auth, Sign, Encrypt. Campus PKI B Univ.CA A Univ.CA 学内用 学内用 学内用 学内用 EE EE Grid Computing Grid PKI A Univ.NAREGI CA B Univ.NAREGI CA Proxy EE Proxy EE Proxy EE Proxy EE Proxy EE Proxy EE Server, Super Computer Server, Super Computer Student,Faculty Student,Faculty Kento Aida, National Institute of Informatics

  8. Grid Kento Aida, National Institute of Informatics

  9. Around Tokyo Computing Resources (April2006.) >= 40TFLOPS Hokkaido Univ. Japan Atomic Energy Agency (Tokai) 10TFLOPS - 40TFLOPS Japan Atomic Energy Agency (Naka) Tohoku Univ. (Information Synergy Center) Japan Atomic Energy Agency (Oarai) 1TFLOPS - 10TFLOPS Tohoku Univ. (Institute for Materials Research) < 1TFLOPS High Energy Accelerator Research Organization(KEK) National Research Institute for Earth Science and Disaster Prevention Tohoku Univ. (Institute of Fluid Science) Tsukuba Univ.(Academic Computing & Communications Center, Center for Computational Science) JST (Institute for Advanced Biosciences, Keio Univ. ) JST (Tsukuba Univ.) Japan Aerospace Exploration Agency (Kakuta) National Institute for Materials Science Japan Advanced Institute of Science and Technology JST(National Institute of Advanced Industrial Science and Technology) National Institute for Fusion Science Tokyo Univ.(Institute for Solid State Physics) JST (Institute for Solid State Physics, Tokyo Univ.) Kyoto Univ.(Academic Center for Computing and Media Studies) Tokyo Univ.(Information Technology Center ) Hiroshima Univ. Yamaguchi Univ. JST (Tokyo Univ.) Chiba Univ. Kyushu Univ. RIKEN National Astronomical Observatory of Japan Tokyo Metropolitan University National Institute of Informatics Kobe Univ. Japan Aerospace Exploration Agency (Sagamihara) Oosaka Univ. Tokyo Univ. (The Institute of Medical Science) Kyoto Univ. (Institute for Chemical Research,Research Institute for Sustainable Humanosphere) Japan Aerospace Exploration Agency (Choufu) The Instituteof Statistical Mathematics Japan Atomic Energy Agency( Kansai Photon Science Institute) Nagoya Univ. JST (The University of Electro-Communications) Tokyo Institute of Technology (GSIC) Institute for Molecular Science Japan Agency for Marine-Earth Science and Technology JST(Tokyo Institute of Technology) National Institute of Genetics Kento Aida, National Institute of Informatics RIKEN(Yokohama) Yokohama City University

  10. WSRF(NAREGI implementation + Globus 4) NII Research Organizations IMS etc NAREGI Grid Middleware Grid-Enabled Nano-Applications ACS Grid PSE Grid Visualization Grid Programming Libraries - GridRPC - GridMPI Grid Workflow Tool Data Grid GridRPC CIM GridFTP Super Scheduler Information Service MPI 1.0/2.0 GFS JSDL OGSA-EMS OGSA-DAI WSRF OGSA- RUS XKMS Grid VM X.509 High-Performance & Secure Grid Networking, Certification VOMS SuperSINET Computing Resources Kento Aida, National Institute of Informatics

  11. Job Execution Scenario Application requirement definition CA/RA Work-flow MPIsource … Co-allocation Job a: Sign-on b: Deployment c: Edit PSE WFT 8: Visualization Portal User cert. 1: Submission VOMS Information Service 2: Resource discovery SuperScheduler Proxy cert. (User DN, VO) MyProxy 3: Reservation (Co-Allocation) 9: Accounting GridVM GridVM GridVM 5: IMPI starts 6: MPI job starts 4: Reservation LocalScheduler LocalScheduler LocalScheduler Network monitor 7: MPI init. 2: Monitoring FMOJob IMPIServer RISMJob GridMPI Site ρ Site α Site μ DataGrid Grid File System Kento Aida, National Institute of Informatics

  12. User-Level Grid Tools & PSE • Grid PSE • support for compilation and deployment • execution support • Grid Workflow Tool • workflow language • GUI Kento Aida, National Institute of Informatics

  13. Workflow based Grid FMO Simulations of Proteins fragment data monomer calculation densityexchange dimer calculation total energy calculation njs_png2005 njs_png2005 njs_png2006 njs_png2006 njs_png2007 njs_png2007 njs_png2008 njs_png2008 visuali-zation njs_png2009 njs_png2009 njs_png2002 njs_png2010 njs_png2002 njs_png2010 input data njs_png2003 njs_png2011 njs_png2003 njs_png2011 njs_png2004 njs_png2002 njs_png2012 njs_png2004 njs_png2002 njs_png2012 NII Resources njs_png2057 njs_png2057 Data component dpcd054 dpcd054 dpcd055 dpcd055 dpcd056 dpcd056 dpcd052 dpcd052 IMS Resources dpcd057 dpcd057 dpcd053 dpcd056 dpcd053 dpcd056 source: Prof. Aoyagi (Kyushu Univ.) Kento Aida, National Institute of Informatics

  14. Grid Middleware Grid Middleware Adaptation of Nano-science Applications to Grid Environment GridMPI NII IMS (Super-Sinet) Electronic Structurein Solutions FMO RISM Solvent Distribution Analysis Electronic StructureAnalysis Data Transformationbetween Different Meshes MPICH-G2, Globus FMO RISM Fragment Molecular Orbital method Reference Interaction Site Model source: Prof. Aoyagi (Kyushu Univ.) Kento Aida, National Institute of Informatics

  15. Cluster A: Cluster B: YAMPII IMPI YAMPII IMPI server Programming • GridRPC • RPC on the grid • API standardization by OGF • Ninf-G • a reference implementation of GridRPC API • GridMPI • MPI library on the grid • MPI communication between parallel systems on the grid Numerical Library IDL FILE IDL Compiler Client 4. connect back 3. invoke Executable generate 2. interface reply Remote Executable GRAM 1. interface request fork Interface Information LDIF File MDS retrieve Client side Server side Kento Aida, National Institute of Informatics

  16. NAREGI Data Grid Environment Grid Workflow Job 1 Job 2 Job n Data Grid Components Data 1 Data 2 Data n Import data into workflow Data Access Management Place & register data on the Grid Job 1 Job 2 Metadata Construction Assign metadata to data Grid-wide DB Querying Meta- data Meta- data Job n Meta- data Data Resource Management Data 1 Data 2 Data n Grid-wide File System Store data into distributed file nodes Kento Aida, National Institute of Informatics

  17. VO-APL1 SS IS VO-RO1 SS IS GridVM GridVM GridVM GridVM IS IS IS IS IS IS GridVM GridVM VO Service Decoupling VOs and Resource Providers RO3 Client Client Client VO-APL2 ResearchOrg (RO)1 VOMS VOMS SS IS RO2 VOs & Users VO-RO2 VOMS VOMS IS SS Resource Providers IS IS • Policy • VO-R01 • Policy • VO-R01 • VO-APL1 • VO-APL2 • Policy • VO-R02 • Policy • VO-R01 • VO-APL1 • Policy • VO-R01 • VO-APL1 • VO-APL2 • Policy • VO-R02 • VO-APL2 a.RO2 b.RO2 n.RO2 A.RO1 B.RO1 N.RO1 Grid Center@RO1 Grid Center@RO2 Kento Aida, National Institute of Informatics

  18. NAREGI Portal Computing Resource NAREGI Client Job SRM Client Gfarm Client SRM Client Gfarm Client gLite Client NAREGI Metadata Server gLite Client DPM (SRM Server) LCG Utility LFC (Metadata Server) GridFTP Server Gfarm Server Storage Storage NAREGI-EGEE data Sharing (GIN) • NAREGI and EGEE gLite clients can access to both data resources (e.g., bi-directional file copy) using SRM interface. • GridFTP is used as its underlying file transfer protocol. • File catalog (metadata) exchange is planned. Kento Aida, National Institute of Informatics

  19. gLite-WMS gLite-BDII GIN-BDII NAREGI-IS gLite-UI NAREGI Portal gliteCE NAREGI-SS NAREGI-GAHP NAREGI-SC NAREGI Client Lib Interop-SC EGEE user NAREGI user lcgCE Computing Resource WS GRAM NAREGI GridVM PreWS-GRAM NAREGI-EGEE Job Submission (GIN) • Architecture • Demo • NAREGI  EGEE: using NAREGI Workflow • EGEE  NAREGI: using glite WMS commands Kento Aida, National Institute of Informatics

  20. FY2003 FY2004 FY2005 FY2006 FY2007 Development and Integration of αVer. Middleware Roadmap of NAREGI Grid Middleware FY2008 OGSA/WSRF-based R&D Framework UNICORE-based R&D Framework Utilization of NAREGI-Wide Area Testbed Utilization of NAREGI NII-IMS Testbed Prototyping NAREGI Middleware Components Apply Component Technologies to Nano Apps and Evaluation β Ver. Release Version 1.0 Release Evaluation of αVer. In NII-IMS Testbed Deployment of β Ver. Evaluation of β Ver. By IMS and other Collaborating Institutes Development and Integration of βVer. Middleware Evaluation on NAREGI Wide-area Testbed α Ver. (Internal) Development of OGSA-based Middleware Verification & Evaluation Of Ver. 1 Kento Aida, National Institute of Informatics

  21. NAREGI Deployment • High Energy Accelerator Research Organization (KEK) • HEP application • National Astronomical Observatory of Japan • virtual observatory • Institute for Molecular Science • nano-science application • Osaka University • computing service, certificate authority • Tokyo Institute of Technology • computing service on TSUBAME Kento Aida, National Institute of Informatics

  22. Grid Operation Center UPKI NII CP/CPS GOC NAREGI help desk developers training program CA SINET NOC admin. SINET admin. VO VO admin. users users university/laboratory Kento Aida, National Institute of Informatics

  23. Grid CA (NAREGI CA) • operation of Grid CA for CSI participants • issuing internationally valid grid certificates • ApGrid PMA • cooperation with computing centers (planed) • Local Registration Authority (LRA) at computing centers in universities/laboratories • cooperation with UPKI (planed) • issuing grid certificates from UPKI certificates Kento Aida, National Institute of Informatics

  24. Issuing User Certificate (planed) GOC-CA NII GOC GOC-RA (2)request license ID LRA internet • (3) • receive license ID • apply certificate • (3) • receive license ID • apply certificate (2)request license ID (2) examine DN • (1) • exam by document • exam by F2F • (3) • receive license ID • apply certificate (1) apply with DN LRA • (1) • exam by document • exam by F2F LRA UPKI cert. Univ. A(operating UPKI) Univ. B Univ. C Kento Aida, National Institute of Informatics

  25. Summary • CSI is a new information infrastructure to boost today’s advanced scientific research in Japan. • networking, federated ID management, grid • NAREGI grid middleware realizes to built a virtual single computing environment on geographically distributed and storage resources. • NAREGI middleware ver.1 will be released in May 2008. • next step • The NAREGI deployment phase is starting in 2008. • NII plans to run GOC. Kento Aida, National Institute of Informatics

  26. Kento Aida, National Institute of Informatics

More Related