Status and prospective in japan
1 / 46

Status and Prospective - PowerPoint PPT Presentation

  • Updated On :

KEK High Energy Accelerator Research Organization. Status and Prospective in Japan. Atsushi Manabe Computing Research Center (CRC) KEK. Outline. Grid projects (budget) in Japan academic region. HEP Data Grid activities in Japan. KEK Needs in KEK Strategy of KEK CRC Pacifi-Grid

Related searches for Status and Prospective

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Status and Prospective ' - Roberta

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Status and prospective in japan l.jpg


High Energy Accelerator Research Organization

Status and Prospective in Japan

Atsushi Manabe

Computing Research Center (CRC)


Int'l HEP Data Grid Work Shop in Daegu

Outline l.jpg

  • Grid projects (budget) in Japan academic region.

  • HEP Data Grid activities in Japan.

    • KEK

      • Needs in KEK

      • Strategy of KEK CRC

      • Pacifi-Grid

      • Gfarm project

      • Grid/HPSS test bed on SupserSINET.

    • ICEPP: ATLAS Japan Regional Center

Int'l HEP Data Grid Work Shop in Daegu

Grid projects budget in japan academic area only l.jpg
Grid Projects(budget) in Japan(academic area only)

  • MEXT(Ministry of Education, sports …. and technology) projects

    • SuperSINET; Gigabit backbone for academic researchers in Japan

    • MEXTInformatics Project

      • A05: Application Grid projects for science ~ US$ 6M /5 y

        Astronomy (Virtual Observatory), io-Infomatics/Genome (BioGrid)……

    • Networking supercomputers: ITBL ~US$ 100M /5y

    • National Research Grid Initiative ~US$ 300M/5y focused in Nano-tech application


  • METI (Ministry of Economy, Trade and Industry) project

    • AIST GTRC(Grid Technology Research Center) ~US$ 140M

Int'l HEP Data Grid Work Shop in Daegu

Gigabit network l.jpg
Gigabit network

  • Super SINET

    • 10Gbps MEXT DWDM nationwide network

    • About 40 universities and labs

    • US$ 60M/yr

    • 5 application areas: HEP, Nanotech, Bioinformatics, Astronomy, Grid Computing

  • Tsukuba-WAN

    • 330Gbps DWDM ring in the Tsukuba area

    • Tens of national labs

    • 6 supercomputers in TOP100

    • Testbed for Grid infrastructure and applications

Int'l HEP Data Grid Work Shop in Daegu

Slide5 l.jpg

Tohoku U


NII Chiba

IP 10 Gbps

WDM path



IP router

Nagoya U


Nagoya Hub


Osaka Hub

Tokyo Hub


Osaka U



Kyoto U





U Tokyo









Network Topology of SuperSINET

Int'l HEP Data Grid Work Shop in Daegu

Supersinet backbone l.jpg


SuperSINET Backbone



Int'l HEP Data Grid Work Shop in Daegu

Network infrastructure l.jpg
Network Infrastructure





Japan (NII) -NY : 2.4G x 2 Jan. 2003

Japan – US: 622M x 2 (TRANSPAC)

Korea – US: 45 M

Korea – Japan: 2.4G Jan. 2003

Korea – Europe: 2 M (TEIN)

China(IHEP) – Japan(KEK): 128 kbps (HEP)

China – US: 10 M

Taiwan – Japan: 155 M

Taiwan – US: 622 M (Dec. 2002)

Int'l HEP Data Grid Work Shop in Daegu

Mext grid projects l.jpg
MEXT Grid Projects

  • MEXT “Informatics” Project : A05 “Grid” Area

    • Lead by Shinji Shimozo(Osaka U.) & Satoshi Matsuoka(Titech)

    • 11 groups of universities and labs

    • Research and development on application, middleware, and infrastructure

    • Support the Virtual Observatory(astronomy) and the BIO-Grid(bio-Infomatics/Genome) projects

  • Tokyo Institute of Technology “Titech Campus Grid” (US$ 2M)

    • Lead by Satoshi Matsuoka(Titech)

    • Total of 800 CPU PC clusters at 13 locations in a 30km range connected by Super TITANET(1-4Gbps) backbone

    • Pilot project for virtual parallel computer infrastructure for high end computational e-Science

Int'l HEP Data Grid Work Shop in Daegu

Slide9 l.jpg

  • ITBL(IT-based Laboratory)

    • To Connect Super-Computers in Government Labs: NAL, RIKEN, NIED, NIMS, JST, JAERI

    • Project period: 2001-2005 (3-stage project) with total of US$ 105M funding

    • Software development project.

    • Applications: mechanical simulation, computational biology, material science, environment, earthquake engineering

    • Step 1: Supercomputer centers of government lab arenetworked via SuperSINET

    • Step 2: “Virtual Research Environment”: Grid-enabling laboratory applications

    • Step 3: Sharing information among researchers from widely distributed disciplines and institutions

Int'l HEP Data Grid Work Shop in Daegu

Naregi national research grid initiative l.jpg
NAREGI (National Research Grid Initiative)

  • NII (The National Institute of Informatics) under MEXT project.

  • Making Center of Grid Research & Development

    • Development of Grid Lower and Middleware(Grid scheduler,VM, RPC, MPI, GUI tools…)

    • Networking, Security and User management(Routing, robust TCP/IP protocol, Traffic management)

    • Applications for Nano-science and technology.Joint with Institute for Molecular Science.


Int'l HEP Data Grid Work Shop in Daegu

Grid technology research center l.jpg
Grid Technology Research Center

  • Part of AIST under METI (Ministry of Economy, Trade and Industry)

  • GTRC(Grid Technology Research Center)

    • Established in Jan. 2002

    • Director: Satoshi Sekiguchi

  • US$140M for building, equipment, R&D and operations

    • AIST super server; PC cluster2004-2Q 11TFLOPS/ AMD Operon x 2116CPUs

  • To be central institute for Grid research in Japan

    • Not just R&D, but serve as central CA, ApGrid’s principal site

    • Collaboration with institutions and other Grid projects of both domestic and international scale

    • Organizing GRID Consortium Japan


  • Int'l HEP Data Grid Work Shop in Daegu

    Grid related activity at kek l.jpg

    Grid related Activity at KEK

    Int'l HEP Data Grid Work Shop in Daegu

    Major projects in kek l.jpg
    Major projects in KEK

    • KEK sponsors projects which use accelerators and also projects in related fields

      • KEKB accelerator

        • BELLE experiment

      • Proton synchrotron

        • nuclear and elementary particle physics

        • medical

      • Synchrotron Radiation Facility

        • bio, material science and so on

      • Also many theorists in related fields

    • Support International HEP collaboration

      • ATLAS Japan

    Int'l HEP Data Grid Work Shop in Daegu

    Kek future projects l.jpg
    KEK Future projects

    • J-PARC

      • will be built in Tokai div. of Japan Atomic Energy Research Institute (JAERI) where ~50km north from KEK start the operation in JFY2005

      • 50 GeV high-intensity proton synchrotron

      • nuclear physics and elementary particle physics

    • Super B factory

      • x30 Luminosity(require x10 CPU power). Under discussion.

    • Linear Collider

      • discussion is still under way

    Int'l HEP Data Grid Work Shop in Daegu

    Computing facilities at kek l.jpg
    Computing Facilities at KEK

    • Computing Research Center offers group oriented computing services

      • Exclusive computing resource for main groups

        • Networking, storage and computing resource

    • User group owns their own computer systems

      • PC farms

    Int'l HEP Data Grid Work Shop in Daegu

    Main computing facilities l.jpg
    Main computing facilities

    • KEKCC computer system

      • for experiments at PS, linear collider R&D, theory, ATLAS and so on

        • IBM SP3, 120TB HPSS storage

    • KEKB computer system for Belle

      • Sun WSs and PC farms. 500TB storage.

    • KEKSC: Super-computer system for mainly lattice QCD G.

      • Hitachi SR-8000 100 nodes; almost 1TFLOPTS

    Int'l HEP Data Grid Work Shop in Daegu

    Our needs in grid era l.jpg
    Our needs in Grid Era

    • ATLAS Regional Center (Grid is premise)

      • Being set up at ICEPP (Int’l Center for Particle Physics, U. Tokyo)

      • Still KEK have large portion of ATLAS Japan members and they will make data analysis using computing resource at KEK. We will support them.

    • And

      • BELLE; international collaboration and require more and more CPU and storage.

      • J-PARC; joint project of other Inst. regionally separated.

    Int'l HEP Data Grid Work Shop in Daegu

    Belle l.jpg

    • BELLE is the major project at KEK

    • They have many collaborators world wide

    • They always needs additional CPU power because acc. luminosity improving and improving.

    • They seek a solution of their problems related computing issue using GRID.

    • We interviewed the BELLE computing coordinator and got the wish list shown in the following slides

    Int'l HEP Data Grid Work Shop in Daegu

    Wish lists from belle l.jpg
    Wish lists from BELLE

    • What BELLE wants:

      • A way to integrate CPU powers and way to distribute data among collaborators’ Univ.s and Inst.s for their data production.

      • Automatic way to submit analysis jobs in the distributed environment.

        • once analysis codes have placed somewhere, process for all of experimental data and Monte Carlo should be done automatically and simply users gain the result

      • Automatic Monte Carlo production

      • Automatic version control for libraries, constants and others on all of the computer system in BELLE

      • Automatic update on reconstructed data or Monte Carlo if libraries or constants are once updated

    Int'l HEP Data Grid Work Shop in Daegu

    Wish lists from belle 2 l.jpg
    Wish lists from BELLE 2

    • It is not easy to achieve their requests technically

      • simplest solution will be hire more Post-Docs or find more grad-students, however

    • We have decomposed their requests

      • should be done in application level (BELLE)


      • can be solved by GRID

    Int'l HEP Data Grid Work Shop in Daegu

    What they need related grid l.jpg
    What they need related GRID

    • An efficient way to distribute jobs and way to manage/share/distribute data for the jobs.

      • effective data catalogue

        • physical to logical mapping

        • Seamless and unique file tree

        • query

          • file names, contents and others

        • works on the Internet but secure

      • efficient and easy to distribute data to remote sites

      • easy to use

        • web interface and GUI also

    Int'l HEP Data Grid Work Shop in Daegu

    Continued l.jpg

    • mass storage management system

      • not only tapes but also large amount of disks

      • robust and fast transfer

      • easy integration with other component

    • We already have them partially on LAN, but we need to expand them to the Internet. We hope GRID will solve them.

    Int'l HEP Data Grid Work Shop in Daegu

    Kek crc strategy towards grid l.jpg
    KEK CRC strategy towards GRID

    Int'l HEP Data Grid Work Shop in Daegu

    Done or on going l.jpg
    Done or On going.

    • Install GT2(Globus toolkit ver.2) and applications (LSF) to two CRC Systems (KEKCC & KEKB) ) and will open to some limited users. (remote Job submission)

    • Build temporal CA with single Virtual Organization(VO).

    • Build a inter-institute wide Grid test bed on SuperSINET and make a performance test. (talk later)

    • Trial use of SRB with Belle data in a test system.

    • Start to touch LCG tools by ATLAS Japan people.

    • watch technical trends carefully.

    Int'l HEP Data Grid Work Shop in Daegu

    Slide25 l.jpg

    • SRB is available today and looks useful even in HEP ( )

      • developed at San Diego Super Computer Center

      • seamless solution to HPSS

        • good for the KEKCC (central computer) users

      • Data catalogue and replication

      • SLAC is also testing

    Int'l HEP Data Grid Work Shop in Daegu

    Until early 2004 l.jpg
    Until early 2004

    • Deploy GT3, evaluate LCG tools and other something new technology.

    • Build own VO at each sites and integrate them.

    • File sharing by SRB (or ?) and seamless job submission between KEKCC and KEKB

    • Pacifi-Grid test environment; Pan-Pacific collaboration (see later)

      • KNU (Korea), Univ. of Melbourne (Australia), SLAC and KEK will test SRB

    • Start to develop higher level software (experiment side) on the top of Globus and SRB.

      • Message passing with GSI authentication ?

    Int'l HEP Data Grid Work Shop in Daegu

    Slide27 l.jpg

    • KEKB and KEKCC (5yeas rental )system replacement (2005-4Q) .

      • GRID will become concrete until this stage or die

      • Final specification have to be decided

        • Decision of middle ware (Grid or else?)

        • CA

        • Security

        • human resource

          • trained engineer on site

    Int'l HEP Data Grid Work Shop in Daegu

    Other topics discussed internally l.jpg
    Other topics discussed internally

    • CA

      • nation wide, institution wide or else? KEK certify all Japanese HEP user?

      • Rules for certification of users.

    • PKI

      • secure enough? Safety private key management (IC card?)

      • management costs?

      • seamless integration with local authentication

    • Network

      • How we can overcome latency?

      • better connectivity to the Internet, routing and throughput

    • middleware

      • Globus, Avaki or something else

      • SRB, LCG or other data replication solution

    • Storage

      • SAN

      • StorageTank (IBM)

    • security vs usability

    Int'l HEP Data Grid Work Shop in Daegu

    Pacifi grid proposal on open pacific grid collaboration l.jpg

    Pacifi-Grid:Proposal on Open Pacific Grid collaboration

    Takashi Sasaki (KEK)

    Int'l HEP Data Grid Work Shop in Daegu

    Charter l.jpg

    • Give research institutions and IT vendors opportunity of testing Grid technologies under inter-state environment.

      • Emphasis on Data Grid

      • Open to everybody who wants to join

      • Grid is a newly emerging technology and more experiences are necessary before deploying at the large scale production systems

        • Test on high latency network

        • robustness tests

        • interoperability tests among sites and vendors

        • security

    Int'l HEP Data Grid Work Shop in Daegu

    Collaboration under discussion l.jpg
    Collaboration (under discussion)

    • Korea

    • U of Melbourne, Australia

    • SLAC, USA

    • JAPAN

      • ICEPP (Univ. of Tokyo)

      • KEK

    • Some venders

      • Under discussing with Fujitsu, IBM Japan and so on

    Int'l HEP Data Grid Work Shop in Daegu

    Gfarm r d l.jpg
    Gfarm R&D

    Development R&DGrid middleware of Cluster File System with job execution.

    Int'l HEP Data Grid Work Shop in Daegu

    R d of gfarm grid data farm l.jpg
    R&D of Gfarm (Grid data farm)

    • Gfarm

      • Development by CS scientist of AIST, Titech and KEK since 2000

        • AIST :Advanced Inst. for Science and Technology

      • Architecture

        • PC farm with large local disk/node as a GRID fabric

        • Large data file is divided into fragments and stored in the disks

        • Data I/O by parallel file system

        • Affinity scheduling of process and storage for data residence

    Int'l HEP Data Grid Work Shop in Daegu

    Data access locality l.jpg

    user job

    user job

















    high speed






    Data Access Locality

    Exploit the data access locality as much as possible

    Int'l HEP Data Grid Work Shop in Daegu

    Gfarm l.jpg


    • Authentication:Globus GSI and/orShared Private Key

    • Grid Data Farm : Beta 3 release is now available.

    • Parallel I/O: exploit the data access locality, store and access files by “fragments”

    • Parallel Job: program runs on thenodes where the file fragmentsreside: “owner computes”

    • Programs and file fragments,job history, data checksum,file replication are managed withMetaDB

    • Fragments are replicated forbackup and load balancing

    • User sees the fragmented files asa single Gfarm URL

    • Provide system call hooks foropen(), close(), read(), write() etc

    Int'l HEP Data Grid Work Shop in Daegu

    Fads goofy in atlas l.jpg
    FADS/Goofy in ATLAS

    • Used in sub-detector software developments and physics validations

    • Detector modules also run in ATLAS mainstream framework (Athena)

    • Testbed for ROOT I/O

    • Bandwidth Challenge inSC2002

    • Generated 106 fully simulatedhigss  4 events in 2 dayswith ~ 400 CPU

    Int'l HEP Data Grid Work Shop in Daegu

    Gfarm achievement l.jpg
    Gfarm achievement

    • “File affinity scheduling” provides parallel processing capability for both I/O-bound and CPU-intensive jobs.

    • FADS/Goofy was successfully implemented on Gfarm architecture. In the process, ROOT I/O module works with Gfarm parallel file system using the system call hook, without a changes to the ROOT package.

    • ROOT I/O files has been successfully replicated at 2.286 Gbps using the SC2002 Bandwidth Challenge testbed with 12 nodes ~ 190Mbps/node.

    • Please refer to CHEP03 reports by Y.Morita and O.Tatebe

    Int'l HEP Data Grid Work Shop in Daegu

    Status of atlas japan rc icepp u of tokyo l.jpg
    Status of ATLAS Japan RC (ICEPP U. of Tokyo)

    • R&D is ongoing

      • Test-buy of PC farm and storage

        • 1U P3, P4 servers ( >150nodes and more) , LTO (10TB) with CASTOR and >20TB disk servers.

      • Nordu GRID was tested first

        • Data transfer tests between U. Tokyo and KEK was done

      • ATLAS data challenges

      • LCG 0 and 1

    Int'l HEP Data Grid Work Shop in Daegu

    Lcg 0 deployment status l.jpg
    LCG-0 Deployment Status

    These sites deployed the LCG-0 pilot system and will be the first sites to deploy LCG-1

    Int'l HEP Data Grid Work Shop in Daegu

    Lcg resource commitments 1q04 l.jpg
    LCG Resource Commitments – 1Q04

    Int'l HEP Data Grid Work Shop in Daegu

    Status of atlas japan rc 2 l.jpg
    Status of ATLAS Japan RC 2

    • Budget requests have been submitted to the funding agency

      • still no decision has been made

        • and no decision will be made within this FY

          • All of national universities and some national laboratories including KEK will be non-nationalized (not privatized, but will be a kind of independent governmental agencies)

    Int'l HEP Data Grid Work Shop in Daegu

    Summary of kek grid activity l.jpg
    Summary ofKEK Grid activity

    • KEK had a step forward for the Grid computing

    • People at ATLAS Japan RC are working on LCG

    • Realistic environment is necessary to test middleware and applications

      • We have started the discussion among Pan-Pacific sites to collaborate each other

      • We want to exchange knowledge and experiences

    Int'l HEP Data Grid Work Shop in Daegu

    Slide46 l.jpg

    End of my presentation

    Thank you

    Int'l HEP Data Grid Work Shop in Daegu