Tony Doyle
This presentation is the property of its rightful owner.
Sponsored Links
1 / 38

Tony Doyle PowerPoint PPT Presentation


  • 91 Views
  • Uploaded on
  • Presentation posted in: General

Tony Doyle. GridPP Collaboration Meeting 9 May 2002. The Vision Thing… Grid Scale Integration Dissemination LHC Analyses Other Analyses. DataGrid LCG Interoperability Infrastructure Finanaces Summary. Outline. Inspiration??. GridPP Documents.

Download Presentation

Tony Doyle

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Tony doyle

Tony Doyle

GridPP Collaboration Meeting

9 May 2002


Outline

The Vision Thing…

Grid

Scale

Integration

Dissemination

LHC Analyses

Other Analyses

DataGrid

LCG

Interoperability

Infrastructure

Finanaces

Summary

Outline

Tony Doyle - University of Glasgow


Inspiration

Inspiration??

Tony Doyle - University of Glasgow


Gridpp documents

GridPP Documents

Tony Doyle - University of Glasgow


Gridpp vision

From Web to Grid - Building the next IT Revolution

Premise

The next IT revolution will be the Grid. The Grid is a practical solution to the data-intensive problems that must be overcome if the computing needs of many scientific communities and industry are to be fulfilled over the next decade.

Aim

The GridPP Collaboration aims to develop and deploy the largest-scale science Grid in the UK for use by the worldwide particle physics community.

GridPP Vision

Many Challenges..

Shared distributed

infrastructure

For all experiments

Tony Doyle - University of Glasgow


Gridpp objectives

1. SCALE: GridPP will deliver the Grid software (middleware) and hardware infrastructure to enable the testing of a prototype of the Grid for the LHC of significant scale.

2. INTEGRATION: The GridPP project is designed to integrate with the existing Particle Physics programme within the UK, thus enabling early deployment and full testing of Grid technology and efficient use of limited resources.

3. DISSEMINATION: The project will disseminate the GridPP deliverables in the multi-disciplinary e-science environment and will seek to build collaborations with emerging non-PPARC Grid activities both nationally and internationally.

4. UK PHYSICS ANALYSES (LHC): The main aim is to provide a computing environment for the UK Particle Physics Community capable of meeting the challenges posed by the unprecedented data requirements of the LHC experiments.

5. UK PHYSICS ANALYSES (OTHER): The process of creating and testing the computing environment for the LHC will naturally provide for the needs of the current generation of highly data intensive Particle Physics experiments: these will provide a live test environment for GridPP research and development.

6. DATAGRID: Grid technology is the framework used to develop this capability: key components will be developed as part of the EU DataGrid project and elsewhere.

7. LCG: The collaboration builds on the strong computing traditions of the UK at CERN. The CERN working groups will make a major contribution to the LCG research and development programme.

8. INTEROPERABILITY: The proposal is also integrated with developments from elsewhere in order to ensure the development of a common set of principles, protocols and standards that can support a wide range of applications.

9. INFRASTRUCTURE: Provision is made for facilities at CERN (Tier-0), RAL (Tier-1) and use of up to four Regional Centres (Tier-2).

10. OTHER FUNDING: These centres will provide a focus for dissemination to the academic and commercial sector and are expected to attract funds from elsewhere such that the full programme can be realised.

(…. WHAT WE SAID WE COULD DO IN THE PROPOSAL)

GridPP Objectives

Tony Doyle - University of Glasgow


Grid a single resource

Grid – A Single Resource

Glenn Patrick

Many millions

of events

Various conditions

Many samples

GRID

A unified approach

Peta Bytes of data storage

Distributed resources

Many 1000s of computers required

Worldwide collaboration

Heterogeneous operating systems

Tony Doyle - University of Glasgow


Grid what s been happening

OGSA

Grid - What’s been happening?

GRID

A unified approach

  • A lot…

  • GGF4, OGSA and support of IBM (and others)

    • [as opposed to .NET development framework and passports to access services]

    • Timescale? September 2002

  • W3C architecture for web services

    • Chose (gzipped) XML as opposed to other solutions for metadata descriptions… and web-based interfaces

  • linux

    • [as opposed to other platforms… lindows??]

  • C++ (experiments) and C, Java (middleware) APIs

    • [mono - Open Source implementation of the .NET Development Framework??]

  • Tony Doyle - University of Glasgow


    Tony doyle

    GridPP Context (Internally)

    Provide architecture and middleware

    Future LHC Experiments

    Running US Experiments

    Build Tier-A/prototype Tier-1 and Tier-2 centres in the UK and join worldwide effort to develop middleware for the experiments

    Use the Grid with simulated data

    Use the Grid with real data

    Tony Doyle - University of Glasgow


    Using grid middleware and datagrid tools

    Using Grid Middlewareand DataGrid Tools

    GRID

    A unified approach

    1. Authentication

    grid-proxy-init

    2. Job submission to DataGrid

    dg-job-submit

    3. Monitoring and control

    dg-job-status

    dg-job-cancel

    dg-job-get-output

    4. Data publication and replication

    globus-url-copy, GDMP

    5. Resource scheduling – use of CERN MSS

    JDL, sandboxes, storage elements

    Important to

    implement this

    for all experiments…

    Tony Doyle - University of Glasgow


    Edg testbed 1 status

    EDG TestBed 1 Status

    GRID

    A unified approach

    Web interface showing status of (~400) servers at testbed 1 sites

    GRID

    extend to all expts

    Tony Doyle - University of Glasgow


    Lhc computing at a glance

    LHC computing at a glance

    1. scale

    • The investment in LHC computing will be massive

      • LHC Review estimated 240MCHF (before LHC delay)

      • 80MCHF/y afterwards

    • These facilities will be distributed

      • Political as well as sociological and practical reasons

    Europe:

    267 institutes, 4603 users

    Elsewhere: 208 institutes, 1632 users

    Tony Doyle - University of Glasgow


    Gridpp funded staff at cern

    GridPP funded Staff at CERN

    7. LCG

    • IT/ADC

      • Ian Neilson from 16th April. Working on certificate authority then scaling up EDG testbed.

    • IT/API

      • Jacek Generowicz from 1st Feb. Working on GEANT4, in particular, tracking & visualisation

      • Maria Girone from 1st April. Settling in.

    • IT/FIO

      • John Hearns from 1st April. Configuration & installation management; relates to WP4 effort and LCFG.

      • Bill Tomlin from 16th April. State Management.

    • Little acorns..

    Tony Doyle - University of Glasgow


    Rtag status

    RTAG Status

    Tony Cass

    • 6 RTAGs created to date:

      • RTAG1 (Persistency Framework; status: completed)

      • RTAG2 (Managing LCG Software; status: running)

      • RTAG3 (Math Library Review; status: running)

      • RTAG4 (GRID Use Cases; status: starting)

      • RTAG5 (Mass Storage; status: running)

      • RTAG6 (Regional Centres; status: starting)

    • Two more in advanced state of preparation:

      • Simulation components

      • Data Definition Tools

    Tony Doyle - University of Glasgow


    Applications area has momentum

    Applications area has momentum

    Tony Cass

    • Torre Wenaus started as Area Manager in March.

    • Activities in the Applications area are now well organised. See http://cern.ch/lcg/peb/applications.

    • Weekly Applications area meetings initiated together with “Architects Forum”.

    • Active work for Software Process and Persistency. Work plan must be developed for SC2 approval.

    Tony Doyle - University of Glasgow


    Fabrics grid deployment

    Fabrics & Grid Deployment

    Tony Cass

    • LCG Level 1 Milestone: deploy a Global Grid Service within 1 year

      • sustained 24 X 7 service

      • including sites from three continents

        • identical or compatible Grid middleware and infrastructure

      • several times the capacity of the CERN facility

      • and as easy to use

    • Ongoing work at CERN to increase automation and streamline configuration, especially for migration to RedHat 7.2.

      • Aim to phase out old CERN solutions by mid-2003.

    Tony Doyle - University of Glasgow


    Lcg timeline

    Q1 Q2 Q3 Q4

    Q1 Q2 Q3 Q4

    Q1 Q2 Q3 Q4

    Q1 Q2 Q3 Q4

    2002

    2003

    2004

    2005

    LCG Timeline

    1. timescale

    Prototype of Hybrid Event Store (Persistency Framework)

    Hybrid Event Store available for general users

    applications

    Distributed production using grid services

    Full Persistency Framework

    Distributed end-user interactive analysis

    LHC Global Grid TDR

    grid

    “50% prototype” (LCG-3) available

    LCG-1 reliability and performance targets

    First Global Grid Service (LCG-1) available

    Tony Doyle - University of Glasgow


    Be a part of this

    LCG Development

    – Long Term Attachment at CERN

    This will enable Grid developments in the UK to be (more) fully integrated with long-term Grid development plans at CERN.

    The proposed mechanism is:

    1. submit a short one-page outline of current and proposed work, noting how this work can best be developed within a named team at CERN, by e-mail to the GridPP Project Leader (Tony Doyle) and GridPP CERN Liaison (Tony Cass).

    2. This case will be discussed at the following weekly GridPP PMB meeting and outcomes will be communicated as soon as possible by e-mail following that meeting.

    Notes

    1. The minimum period for LTA is 3 months. It is expected that a work programme will be typically for 6 months (or more).

    2. Prior DataGrid and LHC (or other) experiments' Grid work are normally expected.

    3. It is worthwhile reading

    http://cern.ch/lcg/peb/applications

    in order to get an idea of the areas covered, and the emphasis placed, by the LCG project on specific areas (building upon DataGrid and LHC experiments' developments).

    4. Please send all enquiries and proposals to:

    Tony Doyle <[email protected]> and

    Tony CASS <[email protected]>

    Be a part of this?

    Tony Doyle - University of Glasgow


    Summary of lcg presentation

    Summary of LCG presentation

    Tony Cass

    • Project got under way early this year

    • Launch workshop and early RTAGs give good input for high-level planning …

    • … to be presented to LHCC in July

    • New plan takes account of first beam in 2007

    • No serious problems foreseen in synchronising LCG plans with those of the experiments

    • Collaboration with the many Grid projects needs more work

    • Technical collaboration with the Regional Centres has to be established

    • Recruitment of special staff going well (but need to keep the recruitment momentum going)

    • Very serious problem with materials funding

    Tony Doyle - University of Glasgow


    Building upon success

    Building upon Success

    6. DataGrid

    • The most important criterion for establishing the status of this project was the European Commission review on March 1st 2002.

    • The review report of project IST-2000-25182 DATAGRID is available from PPARC.

    • The covering letter states “As a general conclusion, the reviewers found that the overall performance of the project is good and in some areas beyond expectations.”

    • The reviewers state “The deliverables due for the first review were in general of excellent quality, and all of them were available on time… All deliverables are approved. The project is doing well, exceeding expectations in some areas, and coping successfully with the challenges due to its size.”

    Tony Doyle - University of Glasgow


    Tony doyle

    6. DataGrid

    Tony Doyle - University of Glasgow


    Tony doyle

    Identifiable Progress...

    3. Dissemination

    t0

    t1

    Tony Doyle - University of Glasgow


    Uk tiera prototype tier 1 centre

    UK TierA/prototype Tier-1 Centre

    9. Infrastructure

    • Roles

    • Tier-A Centre for BaBar

    • EDG testbed(s)

    • LCG prototype Tier-1 Centre

    • prototype Tier-1 for LHC experiments (Data Challenges independent of LCG development…)

    • Interworking with other UK resources (JIF, JREI, eSC) = UK portal

    • existing LEP, DESY and non-accelerator experiments

    • Purchases

    • First year = Hardware Advisory Group (HAG1)

    • Balance between cpu, disk, and tape

    • Experts on specific technologies

    • Propose more HAGs (2 and 3)..

    • Needs to be successful in all roles... ..Report progress/problems

    Tony Doyle - University of Glasgow


    Rollout of the uk grid for pp

    Rollout of the UK Grid for PP

    9. Infrastructure

    • Operational stability of GridPP middleware = Testbed team

    • “the gang of four” … Andrew McNab, Steve Traylen, Dave Colling (other half) and Owen Moroney

    • Ensures the release of “Testbed” quality EDG software

      • documentation

      • lead for other system managers in terms of implementation

      • pre-defined software cycle releases (2 months..)

    • Subject of the Rollout Plan… “Planning for EDG Testbed software deployment and support at participating UK sites” (Pete Clarke, John Gordon)

    • See Andrew and John’s talks..

    • LCG is the proposed mechanism by which the EDG testbed at CERN becomes an LCG Grid Service. The evolution of the EDG testbed to the LCG Grid Service will take account of both EDG and US grid technology. Need to take account of this..

    Tony Doyle - University of Glasgow


    Longer term

    Longer Term..

    9. Infrastructure

    • LCG Grid Service

    • Takes account of EDG and US grid technology

    • A large-scale Grid resource, consistent with the LCG timeline, within the UK.

    • Scale in UK? 0.5 Pbytes and 2,000 distrib. CPUs = GridPP in Sept 2004

    • “50% prototype”?

    Tony Doyle - University of Glasgow


    Gridpp context externally

    GridPP Context (Externally)

    Neil Geddes

    9. Interoperability

    Tony Doyle - University of Glasgow


    Tony doyle

    GLUE

    9. Interoperability

    • How do we integrate with developments from elsewhere in order to ensure the development of a common set of principles, protocols and standards that can support a wide range of applications?

    • GGF…

    • Within the Particle Physics community, these ideas are currently encapsulated(??) in the Grid Laboratory Uniform Environment (GLUE).

    • Recommend this as a starting point for the wider deployment of Grids across the Atlantic. See http://www.hicb.org/glue/GLUE-v0.1.doc (Ruth Pordes et al.)

    Tony Doyle - University of Glasgow


    Tony doyle

    9. Interoperability

    Tony Doyle - University of Glasgow


    Tony doyle

    9. Interoperability

    = Minimal

    e-Bureaucracy

    Tony Doyle - University of Glasgow


    17m 3 year project

    £17m 3-Year Project

    Dave Britton

    10. Finances

    • Five components

      • Tier-1/A = Hardware + ITD Support Staff

      • DataGrid = DataGrid Posts + PPD Staff

      • Applications = Experiments Posts

      • Operations = Travel + Management + e Early Investment

      • CERN = LCG posts + Tier-0 + e LTA

    Tony Doyle - University of Glasgow


    0 projected staff effort sy

    0. Projected Staff Effort [SY]

    Status at GridPP3

    [email protected]

    WP1 Workload Management 0.5[IC]2.0 [IC]

    WP2 Data Management 1.5[Ggo]1.0 [Oxf] WP12

    WP3 Monitoring Services 5.0++[RAL, QMW]1.0 [HW]

    Security ++[RAL]1.0 [Oxf]

    WP4 Fabric Management 1.5[Edin., L’pool]

    WP5 Mass Storage 3.5++[RAL, L’pool]

    WP6 Integration Testbed 5.0++[RAL/M’cr/IC/Bristol]

    WP7 Network Services 2.0[UCL/M’cr]1.0 [UCL]

    WP8 Applications 17.0

    ATLAS/LHCb (Gaudi/Athena) 6.5[Oxf, Cam, RHUL, B’ham, RAL]

    CMS 3.0 [IC, Bristol, Brunel]

    CDF/D0 (SAM) 4.0 [IC, Ggo, Oxf, Lanc]

    BaBar 2.5 [IC, M’cr, Bristol]

    UKQCD 1.0 [Edin.]

    Total36.0++10.0 ->20.0++6.0

    Tony Doyle - University of Glasgow


    1 recruitment

    1. Recruitment

    • EDG Funded Posts (Middleware/Testbed)

      • All 5 in post + 1 additional

    • EDG Unfunded Posts (Middleware/Testbed)

      • 15 out of 15 in post

    • GridPP Posts (Applications + Tier1/A)

      • Allocated Dec 2001

      • ~14 out of 15 in post

    • CERN Posts

      • First Round = 105 Applicants, 12 Offers, 9 Accepted

      • 4 in Applications, 2 Data Management, 3 Systems

      • Second Round = 140 applicants, 9 Offers

      • Third Round ~ 70 Applicants

      • Aim ~ 28 posts

    Tony Doyle - University of Glasgow


    2 monitoring staff effort sm

    2. Monitoring Staff Effort [SM]

    Robin Middleton

    Tony Doyle - University of Glasgow


    3 progress towards deliverables

    3. Progress towards deliverables..

    Pete Clarke

    Tony Doyle - University of Glasgow


    1 next steps

    -1. Next steps..

    10. Finances

    • O(100k)

      • e CLRC support through to Sept 04

      • Other experiments – unfunded in peer review process

      • Tier-2 centres – unfunded initially

    • £2.3m

      • eDIKT (e-Data, Information and Knowledge Transformation) [SHEFC] Particle Physics = application area - assignment of two (of twelve) FTEs in initial planning. Discussions ongoing with EPCC.

    • O(€100m)

      • The first call for Framework 6 will be early next year.

      • Call out now for expressions of interest for new networks and integrated projects.

      • Draft document led by David Williams (CERN) “Enabling Grids and e-Science in Europe” plans to extend the current paradigms with CERN at its focus as the European e-Science Centre.

      • We believe this is the right approach.

      • Incorporates the UK’s e-Science agenda, adding a European dimension. It also recognises the central role of CERN and builds upon the recent successes of EDG.

      • Contact: Neil Geddes

    Tony Doyle - University of Glasgow


    Summary

    A vision is only useful if its shared

    Grid success is fundamental for PP

    Scale in UK? 0.5 Pbytes and 2,000 distrib. CPUs? = GridPP in Sept 2004

    Integration – ongoing..

    Dissemination – external and internal

    LHC Analyses – we’re getting there..

    Other Analyses – ditto - look forward to Roger’s talk

    DataGrid - - major investment = must be successful

    LCG – Grid as a Service

    Interoperability – sticky subject

    Infrastructure – Tier-A/1 in place, Tier-2’s to follow…

    Finances – (very well) under control

    Next steps on framework 6..

    CERN = EU’s e-science centre

    Co-operation required with other disciplines

    Monitoring mechanisms in place…

    Emphasis on deliverables

    End of “business” section

    Summary

    Tony Doyle - University of Glasgow


    Uml diagram

    UML Diagram

    GRID

    A unified approach

    True

    Vision??

    Tony Doyle - University of Glasgow


    Tony doyle

    4th Collab Meeting Agenda

    Wish I’d b(r)ought a network card.. Welcome!

    Tony Doyle - University of Glasgow


  • Login