Datagrid coordination meetings 4 march 2001 amsterdam 23 june 2001 rome 7 october 2001 rome
1 / 22

Datagrid Coordination Meetings 4 March 2001, Amsterdam 23 June 2001, Rome 7 October 2001, Rome - PowerPoint PPT Presentation

  • Uploaded on

Datagrid Coordination Meetings 4 March 2001, Amsterdam 23 June 2001, Rome 7 October 2001, Rome. Summary for HEP-CCC Meeting 16 November 2001, CERN L. Price, ANL. Goals of DG Coordination.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Datagrid Coordination Meetings 4 March 2001, Amsterdam 23 June 2001, Rome 7 October 2001, Rome' - melvyn

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Datagrid coordination meetings 4 march 2001 amsterdam 23 june 2001 rome 7 october 2001 rome

Datagrid Coordination Meetings4 March 2001, Amsterdam23 June 2001, Rome7 October 2001, Rome

Summary for HEP-CCC Meeting

16 November 2001, CERN

L. Price, ANL

L. E. Price, ANL, p. 1

Goals of dg coordination
Goals of DG Coordination

  • Define jointly the architecture of the data grid to avoid incompatible architectures from different projects.

  • Ensure that tools developed by all projects will interoperate to make as much as possible a seamless data grid system for LHC and our other scientific participants.

  • Partition major tasks among the projects in order to use scarce resources to best advantage and avoid having each project invent only the top priority 1/3 of what we ultimately need.

L. E. Price, ANL, p. 2


L. E. Price, ANL, p. 3

Benefits and costs of coordination

From Paul Avery

Benefits and Costs of Coordination

  • Benefits

    • Effective reach of projects

    • Common infrastructure

    • Larger effort improves visibility, attracts funding

  • Costs

    • More interactions

    • More meetings

    • More coordination

    • More reporting

    • More B.S.

L. E. Price, ANL, p. 4

Many flavors of joint projects

From Paul Avery

Many Flavors of Joint Projects

  • GriPhyN, PPDG and EU DataGrid are already joint projects

    • Interconnected management structures

    • Each serving needs of multiple constituencies

    • GriPhyN: CS, HEP/LHC, gravity waves, Digital astronomy

    • PPDG: CS, HEP (LHC + current expts), Nuc. Phys., networking

    • DataGrid: CS, HEP, earth sensing, biology, networking

L. E. Price, ANL, p. 5

Joint projects the reality

From Richard Mount

Joint Projects: The Reality

L. E. Price, ANL, p. 6

Discussion points
Discussion Points

Common testbeds are necessary as a means of strongly encouraging compatibility, as well as the usual practical benefits of testbeds, namely that any problem of the system in the real world will be revealed.

Support of shared code base. In addition to the overlapping goals of the datagrid projects, it was noted that the projects all rely on tools developed outside of the projects, including Globus, Condor, and others.

Many practical and legal issues are raised by the need to make program available to and usable by other parts of the overall datagrid community. Open source and other sharing concepts were discussed in some detail.

Desirability of working toward a common or at least compatible architecture. This is so partly because the LHC experiments, at least, are connected with all of the projects and will need to use parts of each project. It also permits separately developed modules to interoperate with less effort than otherwise.

L. E. Price, ANL, p. 7

Common architecture

From Ian Foster

Common Architecture

  • A unifying structure & approach

  • Standard protocols, APIs, policies

    • Reflects pragmatic choices corresponding to best available technology

    • Documented (and mostly implemented)

    • Certainly a work in progress

  • Requires

    • Some structure (“architect”? “arch board”?)

    • Procedures for collecting input from the user community and guiding development

L. E. Price, ANL, p. 8

Main coordination issues

From Fabrizio Gagliardi

Main coordination issues

  • Compatibility of DataGrid architecture with PPDG and GriPhyN

  • Synchronisation of deliverables among the different projects

  • Common test beds across Atlantic

  • Globus support and development in US and EU

L. E. Price, ANL, p. 9

Other points of discussion
Other Points of Discussion

Mechanisms for coordination.

Ian Foster and Carl Kesselman are already members of the EU DataGrid Architecture Task Force

Coordinating council concept, consisting of management representatives from the projects.

Technical steering group consisting of technical representatives of the projects. Synchronization of deliverables is also an issue, since some experiments are connected to multiple grid projects.

Dedicated facilities.

Dedicated bandwidth for tests, expecially across oceans.

Grid Operating Center will probably be needed.

L. E. Price, ANL, p. 10

Grid activities in japan

From Satoshi Sekiguchi

Ninf – ETL/TIT

Is a frontier developing “Network enabled servers”

Has been collaborating with NetSolve, UTK


Metacomputing – TACC/JAERI

MPI for vectors, STAMPI


A regional testbed across the Pacific Rim

Globus promotion 

Firewall compliance extension


ITBL – MEXT (ex-STA part)

Tsukuba-One (DWDM ring)


IPv6 promotion

North America





(123 Mbps)


South Korea



Latin America


Hong Kong










Exchange Point

Access Point

Current Status


Numerical Libraries



Ninf_call(FUNC, arg1, ...)

Grid activities in Japan

L. E. Price, ANL, p. 11

2nd meeting rome 23 june 2001
2nd Meeting: Rome 23 June 2001

  • Focus on organization

    • Relationship to CERN “LHC Grid Project”

    • Licensing and code sharing proposal

      • Ian Foster: Consortium for Open Grid Software

    • International Data Grid Coordination Organization

L. E. Price, ANL, p. 12

Consortium for open grid software
Consortium for Open Grid Software

From Ian Foster

L. E. Price, ANL, p. 13

Shared development of grid software a proposal

From Ian Foster

Shared Development of Grid Software:A Proposal

  • Long-term success of Grid concept requires establishment of a framework for shared development of Grid software, that

    • Addresses legal/social issues associated with many contributors and users

    • Addresses governance issues

    • Is connected with “standards” processes

    • Builds credibility for open architecture, open source Grid approach

  • COGS: Consortium for Open Grid Software

L. E. Price, ANL, p. 14

International data grid coordination organization
International Data Grid Coordination Organization

  • Extended discussion, resulting in following organization plan:

    • InterGrid Management Board

      • Start from the ad hoc group that has met twice

      • Consider small number of additions to be sure all players are represented

      • Board represents managements of other projects; does not manage the projects itself!

      • Meet 3 times per year, usually face to face

    • Joint Technical Board

      • Membership: 6 from North America; 6 from Europe; 2 Asia

      • Meet monthly by telephone or video conference

        • Face to face when needed

L. E. Price, ANL, p. 15

3rd meeting rome 7 oct 2001
3rd Meeting: Rome 7 Oct 2001

  • Update on LHC Grid Computing from Hoffmann (in person)

    • Agreement that LHC project will be part of HENP Intergrid Coordination group (new name)

  • Initial presentation of potential joint testbeds

    • Transatlantic testbeds or even better worldwide testbeds are considered to be the heart of the grid coordination process. Interoperability will be required for any success.

    • After initial coordination of grid architectures, testbeds will generally be done by groups from international physics experiments associated with the grid projects, so that practical benefit will be obtained.

  • Refinement of COGS plan

    • License draft generally agreed to with some comments

    • Governance and organization less clear—but considered not to be required

L. E. Price, ANL, p. 16

Getting going on joint testbeds
Getting going on joint testbeds

  • Tasks:

  • (a)identify and agree on the initial sites to be included in the pilot testbed, with 3 or 4 sites from each side of the Atlantic;

  • (b)agree with the collaborations on the target applications and an initial data challenge programme;

  • (c)define the resources, environment, software and services to be provided at each of the component sites;

  • (d)specify the Grid protocols, interfaces and standards necessary to ensure inter-working and a satisfactory application environment;

  • (e)prepare a plan for building and operating the testbed;

  • (f)prepare a plan for initial usage of the testbed by selected experiments.

L. E. Price, ANL, p. 17

Participating data grid projects
Participating Data Grid Projects

  • PPDG

  • GriPhyN

  • DataGrid

    • National Projects: Italy, France, UK, Holland,….

  • DataTAG

  • iVDGL

  • GridPP

  • LHC Grid Computing Project

  • Good liaison with experiments:

    • BaBaR, D0, ATLAS, CMS, ALICE especially

L. E. Price, ANL, p. 18

Licensing goals

From Ian Foster

Licensing: Goals

  • Allow anybody to use Grid software for any purpose

    • No constraints on intended purpose

    • Can include things you don’t like!

  • Allow anybody to contribute to Grid sofware

    • Get credit for their work

    • As few bureaucratic barriers as possible

  • Two goals are not necessarily compatible!

L. E. Price, ANL, p. 19

Globus toolkit as a vehicle

From Ian Foster

Globus Toolkit as a Vehicle

  • We offer the Globus Toolkit as a vehicle for those who want to use it

  • Believe that there are advantages:

    • Strong “brand” with considerable acceptance

    • Considerable code base already, of core code and contributed modules

  • Make commitment to community to support this process as best we can

L. E. Price, ANL, p. 20

First meeting of joint technical board 8 oct 2001
First Meeting of Joint Technical Board: 8 Oct 2001

  • Of membership of 14, 5 (+LEP) were together at Frascati, 6 were on a conference phone call, and 2 were not appointed yet.

  • General discussion of practical organization and goals of testbed projects. Beyond general agreement to “Keep it Simple”, there was not time to conclude the discussion.

  • Presentations of status, architecture plans, and near term milestones for each data grid project

  • Agreement to meet first Monday of each month at 7 am in California/4 pm in middle European time by phone or possibly video.

L. E. Price, ANL, p. 21


  • Datagrid projects have individual identities and responsibilities to funding agencies and cannot be managed as a single project

  • Neverthless close coordination is important for success

  • The adopted compromise is to organize so the technical leaders of the different projects talk frequently (JTB)

  • The managements also need to work together, but meetings are less frequent (IGMB)

  • A legal basis for joint software development is important and being worked on.

  • Common testbeds are very important.

L. E. Price, ANL, p. 22