slide1
Download
Skip this Video
Download Presentation
Tony Doyle

Loading in 2 Seconds...

play fullscreen
1 / 15

Tony Doyle - PowerPoint PPT Presentation


  • 153 Views
  • Uploaded on

GridPP: the UK\'s contribution to the international collaboration building a worldwide Grid, the LHC Computing Grid GridPP – is the system usable?. Tony Doyle. Summary. GridPP runs a major part of the EGEE/LCG Grid, which supports ~3000 users

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Tony Doyle' - decima


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

GridPP: the UK\'s contribution to the international collaboration building a worldwide Grid, the LHC Computing Grid GridPP – is the system usable?

Tony Doyle

summary
Summary
  • GridPP runs a major part of the EGEE/LCG Grid, which supports ~3000 users
  • The Grid is not (yet) as transparent as end-users want it to be
  • The underlying overall failure rate is ~10%
  • User (interface)s, middleware and operational procedures (need to) adapt
  • (see talks by Dave Britton and Stephen Burke for more info. on performance and operations [now])
  • Procedures to manage the underlying problems such that system is usable are highlighted

Usable Systems

virtual organisations
Virtual Organisations
  • Users are grouped into Virtual Organisations
    • Users/VO varies from 1 to 806 members (and growing..)
  • Broadly four classes of VO
    • LHC experiments
    • EGEE supported
    • Worldwide (mainly non-LHC particle physics)
    • Local/regional e.g. UK PhenoGrid
  • Sites can choose which VOs to support, subject to MOU/funding commitments
    • Most GridPP sites support ~20 VOs
    • GridPP nominally allocates 1% of resources to EGEE non-HEP VOs
    • GridPP currently contributes 30% of the EGEE CPU resources

Usable Systems

user view
User View?
  • Perspective matters
  • This talk is not
    • a usability survey
    • unbiased
    • representative
  • Straw poll
    • users overcame initial registration hurdles within ~two weeks
    • users adapt to Grid in (un-)coordinated ways
    • The Grid was sufficiently flexible for many analysis applications

Usable Systems

physics analysis

Analysis Object Data

Analysis Object Data

Analysis Object Data

AOD

Physics Analysis

ESD: Data or Monte Carlo

Event Tags

Collaboration

-wide

Tasks

Event Selection

Calibration Data

Analysis, Skims

INCREASINGDATAFLOW

Raw Data

Analysis

Groups

Physics

Objects

Physics

Objects

Physics

Objects

Individual

Physicists

Physics Analysis

Usable Systems

user evolution
User evolution

Number of UK Grid users (exc. Deployment Team)

Quarter: 05Q4 06Q2 06Q3

Value: 1342 1831 2777

Many EGEE VOs supported c.f. 3000 EGEE target

Number of active users (> 10 jobs per month)

Quarter: 05Q4 06Q1 06Q2

Value: 83 166 201

Fraction: 6.2% 11.0%

Viewpoint: growing fairly rapidly, but not as active as they could be? depends on the “active” definition

Usable Systems

know your users uk enabled vos
Know your users? UK-enabled VOs

806 atlas 763 dzero 577 cms 566 dteam 150 lhcb 131 alice 75 bio65 dteamsgm41 esr 31 ilc 27 atlassgm 27 alicesgm 21 cmsprg 18 atlasprg 17 fusn 15 zeus 13 dteamprg 13 cmssgm 11 hone 9 pheno 9 geant 7 babar 6 aliceprg 5 lhcbsgm 5 biosgm 3 babarsgm 2 zeussgm 2 t2k 2 geantsgm 2 cedar 1 phenosgm 1 minossgm 1 lhcbprg 1 ilcsgm 1 honesgm 1 cdf

Usable Systems

user interface

Scriptor

Job details

Logical

Folders

Job Monitoring

Job builder

Log window

User Interface
  • The GUI is relatively low-level (jobs, file collections)
  • Dynamic panels for higher level functions

Dockable windows

Screenshot of the Ganga GUI

Usable Systems

complex applications

ATLAS

  • GANGA software framework (jointly with LHCb)
  • data challenges
  • producing Monte Carlo data
  • 10 million CPU hours
  • per year
  • LHCb
  • DIRAC software to submit analysis jobs using Grid
  • 2006 analysis job completion efficiency improved to 91%
  • CMS
  • Monte Carlo production, data transfer, job submission
  • CMS transfers top a petabyte a month for the last three months
Complex Applications

Usable Systems

wlcg mou
Particle physicists collaborate, play roles and delegate

e.g. “prg” production group “sgm” software group managers

Underpinned by Memoranda of Understanding

Current MoU signatories:

China France Germany Italy India Japan Netherlands Pakistan Portugal Romania Taiwan UK USA

Pending signatures:

Australia Belgium Canada Czech Republic Nordic Poland Russia Spain Switzerland Ukraine

Negotiation w.r.t. resource and service level

WLCG MoU

Usable Systems

resource allocation
Resource allocation
  • Need to assign quotas and priorities to VOs and measure delivery
  • VOMS provides group/role information in the proxy
  • Tools to control quotas and priorities in site services being developed
    • So far only at whole-VO level
    • Maui batch scheduler is flexible, easy to map to groups/roles
    • Sites set the target shares
    • Can publish VO/group-specific values in GLUE schema, hence the RB can use them for scheduling
  • Accounting tool (APEL) measures CPU use at global level (UK task)
    • Storage accounting currently being added
    • GridPP monitors storage across UK
    • Privacy issues around user-level accounting, being solved by encryption

Usable Systems

user support
User Support
  • Becoming vital as the number of users grows
    • But modest effort available in the various projects
  • Global Grid User Support (GGUS) portal at Karlsruhe provides a central ticket interface
    • Problems are categorised
  • Tickets are classified by an on-duty Ticket Process Manager, and assigned to an appropriate support unit
    • UK (GridPP) contributes support effort
  • GGUS has a web-service interface to ticketing systems at each ROC
    • Other support units are local mailing lists
    • Mostly best-effort support, working hours only
  • Currently ~tens of tickets/week
    • Manageable, but may not scale much further
    • Some tickets slip through the net

Usable Systems

documentation training
Documentation & Training
  • Need documentation and training for both system managers and users
    • Mostly expert users up to now, but user community is expanding
    • Induction of new VOs is a particular problem – no peer support
    • EGEE is running User Fora for users to share experience
      • Next in Manchester in May ’07 (with OGF)
    • EGEE has a dedicated training activity run by NeSC/Edinburgh
  • Documentation is often a low priority, little dedicated effort
    • The rapid pace of change means that material requires constant review
  • Effort on documentation is now increasing
    • GridPP has appointed a documentation officer
      • GridPP web site, wiki
    • Installation manual for admins is good
      • There is also a wiki for admins to share experience
    • Focus is now on user documentation
      • New EGEE web site – coming soon

Usable Systems

alternative view
Alternative view?
  • The number of users in the Grid School for the Gifted is ~manageable now
  • The system may be too complex, requiring too much work by the “average user”?
  • Or the (virtual) help desk may not be enough?
  • Or the documentation may be misleading?
  • Or..
  • Having smart users helps (the current ones are)

Usable Systems

ad