ngs status n.
Skip this Video
Download Presentation
NGS Status

Loading in 2 Seconds...

play fullscreen
1 / 22

NGS Status - PowerPoint PPT Presentation

  • Uploaded on NGS Status. Stephen Pickles Technical Director, GOSC (with thanks to Andy Richards, NGS Co-ordinator) CERN, 17/1/2005. Outline. Users registrations, usage, helpdesk queries Services & Middleware what we offer today managing change

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'NGS Status' - megara

Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
ngs status

NGS Status

Stephen Pickles

Technical Director, GOSC

(with thanks to Andy Richards, NGS Co-ordinator)

CERN, 17/1/2005

  • Users
    • registrations, usage, helpdesk queries
  • Services & Middleware
    • what we offer today
    • managing change
  • Expansion
    • “joining” the NGS
gaining access
NGS core nodes

data nodes at RAL and Manchester

compute nodes at Oxford and Leeds

free at point of use

apply through NGS web site

accept terms and conditions of use

light-weight peer review

1-2 weeks

to do: project or VO-based application and registration

all access is through digital X.509 certificates

from UK e-Science CA

or recognized peer

National HPC services



Must apply separately to research councils

Digital certificate and

Conventional (username/ password) access supported

Gaining Access

UKGSC: Queue Totals For Month Prior To (10:11 AM, 26/11/2004)

  • Notes
  • Certification still dominant query type
  • General contains GGUS related queries from EGEE/GGUS helpdesk
  • (A separate project for this has now been created)
  • LCG savannah generated queries are increasing through this helpdesk
ngs core services

NGS Core Services

Globus, SRB, Oracle, OGSA-DAI, and others

ngs core services globus
NGS Core Services: Globus
  • Globus Toolkit version 2
    • GT 2.4.3 from VDT 1.2
  • Job submission (GRAM)
  • File transfer (GridFTP)
  • Shell (GSI-SSH)
  • Information Services (MDS/GIIS/GRIS)
    • Information providers from GLUE schema
    • Use BDII implementation of MDS2 (as does EGEE)
ngs core services srb
NGS Core Services: SRB
  • Storage Resource Broker from SDSC
  • Location transparent access to storage
  • Metadata catalog
  • Replica management
  • Clients on compute nodes
  • Servers on data nodes
  • Issues/to do:
    • licensing
    • MCAT replication and failover
ngs core services oracle
NGS Core Services: Oracle
  • Oracle 9i database
  • Only on data nodes
  • Populated by users/data providers
  • Infrastructure maintained by NGS database administrators
  • Accessed directly or via OGSA-DAI
ngs services ogsa dai
NGS Services: OGSA-DAI
  • Developed by UK e-Science projects OGSA-DAI and DAIT
  • OGSA-DQP (Distributed Query Processing)
  • Experimental service based on OGSI/GT3 on Manchester data node only
    • containment: 2 cluster nodes reserved for development and production
    • will consider WS-I and WSRF flavours when in final release
  • Uses Oracle underneath
  • User-provided Grid Data Services validated on test system, then transferred to production during scheduled maintenance
  • Early users from e-Social Science (ConvertGrid)
  • Established liaisons with OGSA-DAI team
ngs core services other
NGS Core Services: other

Operated by GOSC for NGS and UK e-Science programme

In production:

  • Certificate Authority
  • Information Services (MDS/GIIS)
  • MyProxy server
  • Integration tests and database
  • Cluster monitoring
  • LCG-VO

In testing:

  • VOMS
  • EDG Resource Broker

In development:

  • Accounting
    • using GGF Usage Record standard for interchange, software from MCS project and TeraGrid
  • Portal
managing middleware evolution









Other software sources

Software with proven capability & realistic deployment experience

Prototypes &




‘Gold’ services

Feedback & future requirements


Managing middleware evolution
  • Important to coordinate and integrate this with deployment and operations work in EGEE, LCG and similar projects.
  • Focus on deployment and operations, NOT development.
maintaining compatibility
Maintaining Compatibility
  • Operating a production grid means valuing robustness and reliability over fashion.
  • NGS cares about:
    • alignment/compatibility with leading international Grid efforts
    • special requirements of UK e-Science community
    • easy migration/upgrade paths
    • proven robustness/reliability
    • based on standards or standards-track specifications
  • NGS cannot support everything
  • Everyone wants service-oriented grids
    • but still settling out: WS-I, WS-I+, OGSI, WSRF, GT3, GT4, gLite
  • Caution over OGSI/WSRF has led to wide convergence on GT2 for production grids and hence some inter-Grid compatibility
    • but there are potentially divergent forces at work
  • Significant changes to NGS Minimum Software Stack will require approval by NGS Management Board on conservative time scales


Adding new resource providers


Resource providers join the NGS by

  • Defining level of service commitments through SLDs
  • Adopting NGS acceptable use and security policies
  • Run compatible middleware
    • as defined by NGS Minimum Software Stack
    • and verified by compliance test suite
  • Support monitoring and accounting

Two levels of membership

  • Affiliation
    • a.k.a. connect to NGS
  • Partnership

Affiliates commit to:

  • running NGS-compatible middleware
    • as defined in NGS Minimum Software Stack
    • this means users of affiliate’s resources can access these using same client tools they use to access NGS
  • a well-defined level of service and problem referral mechanisms
    • SLD approved by NGS Management Board and published on NGS web-site
  • providing technical, administrative, and security (CERT) contacts
  • providing an account and mapping for daily compliance tests (GITS++)
  • accepting UK e-Science certificates
  • maintaining baseline of logs to assist problem resolution

Partners commit to same as affiliates, plus:

  • making “significant” resources available to NGS users
    • creation of accounts/mappings
    • in future, VO support, pool accounts, etc
  • recognise additional CAs with which UK e-Science programme has reciprocal agreements
  • publish/provide additional information to support resource discovery, brokering
  • ability to compile code for computational resources

Bristol and Cardiff going through certification process now.

  • supported by “buddies” and NGS-Rollout list
  • useful feedback on viability of NGS Minimum Software Stack


  • NGS brand
    • certified NGS-compatible
  • better integrated support for local users who also access NGS facilities
  • assistance/advice in maintaining NGS-compatibility over time


  • higher brand quality
  • representation on NGS management board
    • either direct, or through regional or functional consortia

NGS brand must be valuable to make this work.

recent developments
Recent Developments
  • NGS newsletter
  • OMII helpdesk
  • Resource Broker (LCG) available for early adopters
  • TeraGrid
    • UK certificates accepted on TeraGrid
    • Looking at INCA monitoring with Jenny Schopf
  • Trial of accounting software from MCS project
    • using GGF Usage Record draft standard (as EGEE)
  • Meeting with Open Science Grid’s iGOC
  • Collecting user exemplars
    • Agreed form of acknowledgement
  • Workshop towards an NGS user induction course. Need:
    • Easier access (e.g. user interface machines and portals)
    • T-infrastructure


  • 4 core nodes operational
  • 150 Users registered (50 since 1 September ’04)
  • Grid enabled – Globus v2 (VDT distribution v1.2) at present
  • BDII information service (GLUE + MDS Schemas)
  • Data Services – SRB, Oracle, OGSA-DAI
  • Growing base of user applications
  • MyProxy and CA services
  • VO Management Software – LCG-VO
  • User support: Helpdesk


  • Other Middleware [gLite/OMII etc…]
  • NGS Portal
  • Resource Broker
  • SRB production service
  • Accounting
  • Continued expansion