NERC Data Grid

1. NERC Data Grid Helen Snaithand the NDG consortium…

2. What is NDG? • NDG provides the infrastructure which allows users to • Find data • Explore what is known about datasets (including information about the observing or simulating tools) • Access, manipulate and visualise data!

3. NDG Services • The “NDG Discovery Service” • A database of discovery information • A web-site, to provide a portal to that database • A set of web-services, allowing NDG consumers to exploit that database • The “NDG Vocabulary Service” • Databases of environmental thesauri and ontology tools to map between terms • Supports “machine assisted” browsing: eg, search for rainfall and find datasets with precipitation

4. What Can NERC Data Grid Do For You? • Depends on who you are! • Data User - eg scientist looking for data • Data Provider - eg data centre, large research group • Developer - providing access tools or using data sources

5. As a Data User? • Discovery leads directly to data • Simple search page or “google-like” toolbar gives direct access to data, not just the metadata • Comprehensive coverage • Seamless access • Hides data format differences and provides generic visualisation tools • Enhances interdisciplinary research

6. As a Data Provider? NDG Security • Provides a standards based metadata hierarchy • Exploit the NDG metadata structures to design a metadata system from the ground up, or use them as an interoperability tool • No need to run multiple security systems • Increases visibility and usage of datasets • Inclusion in NDG does not prevent inclusion in other data discovery or access initiatives • Minimise user support load

7. As a Developer? • Exploit NDG web services • Exploit NDG modular development • Exploit NDG metadata standards • NDG developments are non-proprietary • NDG standards compliance implies less future development

8. MetaData Framework • Discovery • ISO standard • MOLES - Metadata Objects for Links in Environmental Science • Detailed metadata for browsing • CSML – Climate Science Markup Language • For accessing “features” within data

9. CSML in Action

10. What Can’t the NDG Do? • NDG can’t generate the metadata! • Automated manipulation and searching relies on comprehensive metadata, which require quality data management to create and maintain • NDG does provide a clear framework for metadata requirements • NDG doesn’t provide information services:- it provides data services! • NDG does provide a framework within which it is possible to link data to derived information services:- research publications, reports, policy etc..

11. Competitors!How is NDG different from the rest? • e2edm, End-to-End Data Management: • Similar aims but using different technologies • NDG is more standards compliant • Thredds: • Provides only catalogue services • Thredds is significantly more mature than NDG • Suitable for data providers with hierarchical data / metadata • OPeNDAP: • A protocol for accessing data • Limited support for access control • Requires the user to know exactly what the data are

12. NDG Status • Approaching Beta • Discovery Gateway • Discovery Service • CSML V2 • MOLES • Security Infrastructure • Vocabulary Service • MOLES browser • Population (MOLES & CSML) by NDG partners • Implementing & testing gateways, browsers & interfaces ndg.nerc.ac.uk Trac wiki site