1 / 52

Connecting netcdf/CF to a semantic framework

Connecting netcdf/CF to a semantic framework. M. Benno Blumenthal International Research Institute for Climate and Society http://iridl.ldeo.columbia.edu/ontologies/. RDF/OWL and earth science metadata.

jolang
Download Presentation

Connecting netcdf/CF to a semantic framework

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Connecting netcdf/CF to a semantic framework M. Benno Blumenthal International Research Institute for Climate and Society http://iridl.ldeo.columbia.edu/ontologies/

  2. RDF/OWL and earth science metadata The standards underlying the Semantic Web -- Resource Description Framework (RDF) and Web Ontology Language (OWL), among others – show great promise in addressing some of the basic problems in earth science metadata. In particular they provide a single framework that allows us to describe datasets according to multiple standards, creating a more complete description than any single standard can support, and avoiding the difficult problem of creating a super-standard that can describe everything about everything.

  3. RDF is not a killer app • Resource Description Framework (RDF) is a framework to write down relationships in a reusable way, a semantic framework • A lot of CF is not written down in a usable way, e.g. relationships between different values of standard_names, or how data representations in CF correspond to concepts or other standards. • It is past time to fix that Some would prefer Java, some prefer english

  4. Different Representations of The CF Standard • English – gloriously vague and flexible • Java – complete implementation makes a great black-box, an API becomes yet-another-standard • RDF/OWL – can facilitate creating/enhancing both the English and Java versions (as well as other programming languages)

  5. CF metadata in a semantic framework • A literal level which explains which attributes are available to be attached to datasets/variables, • A more semantic level, which givesexplicit expression to concepts like Coordinate and Non-Coordinate variables, and how a Non-Coordinate Variable can be geo-located.

  6. Semantic interoperability Writing down the CF standard on a semantic level then allows interoperability with other standards, e.g. other ways of marking geolocated Non-Coordinate variables.

  7. additional issues • how to less-ambiguously tag metadata in netcdf files so that software can more easilydetermine which attribute belongs to which metadata standard • How to better register netcdf metadata standards in general • How to better register CF concepts so that interoperability (or indeed operability) can occur.

  8. Why RDF? Make implicit semantics explicit Web-based system for interoperating semantics Decontextualizes the information, facilitating reuse RDF/OWL is an emerging technology, so tools are being built that help solve the semantic problems in handling data

  9. Standard Metadata Standard Metadata Schema/Data Services Datasets Tools Users

  10. StandardMetadataSchema StandardMetadataSchema StandardMetadataSchema StandardMetadataSchema StandardMetadataSchema Datasets Datasets Datasets Datasets Datasets Tools Tools Tools Tools Tools Users Users Users Users Users Many Data Communities

  11. StandardMetadataSchema StandardMetadataSchema StandardMetadataSchema StandardMetadataSchema StandardMetadataSchema Datasets Datasets Datasets Datasets Datasets Tools Tools Tools Tools Tools Users Users Users Users Users Super Schema Standard metadata schema

  12. One take on semantic interoperability `When I use a word,' Humpty Dumpty said in rather a scornful tone, `it means just what I choose it to mean--neither more nor less.'`The question is,' said Alice, `whether you CAN make words mean so many different things.'`The question is,' said Humpty Dumpty, `which is to be master--that's all.' Through the Looking Glass (And What Alice Found There)Carroll, LewisPublished: 1871Type(s): Novels, Young Readers, FantasySource: Wikisource

  13. StandardMetadataSchema StandardMetadataSchema StandardMetadataSchema StandardMetadataSchema StandardMetadataSchema Datasets Datasets Datasets Datasets Datasets Tools Tools Tools Tools Tools Users Users Users Users Users Super Schema: direct Standard metadata schema/data service

  14. Flaws • A lot of work • Super Schema/Service is the Lowest-Common-Denominator, so you end up saying less-and-less about more-and-more. • Science keeps evolving, so that standards either fall behind or constantly change

  15. StandardMetadataSchema StandardMetadataSchema StandardMetadataSchema StandardMetadataSchema StandardMetadataSchema Datasets Datasets Datasets Datasets Datasets Tools Tools Tools Tools Tools Users Users Users Users Users RDF Standard Data Model Exchange Standard metadata schema RDF RDF RDF RDF RDF RDF

  16. RDF RDF RDF RDF RDF StandardMetadataSchema StandardMetadataSchema StandardMetadataSchema StandardMetadataSchem StandardMetadataSchema RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF Datasets Datasets Datasets Datasets Datasets Tools Tools Tools Tools Tools Users Users Users Users Users RDF Data Model Exchange Standard metadata schema RDF

  17. Why is this better? • Maps the original dataset metadata into a standard format that can be transported and manipulated • Still the same impedance mismatch when mapped to the least-common-denominator standard metadata, but • When a better standard comes along, the original complete-but-nonstandard metadata is already there to be remapped, and “late semantic binding” means everyone can use the new semantic mapping • Can use enhanced mappings between models that have common concepts beyond the least-common-denominator • EASIER – tools to enhance the mapping process, mappings build on other mappings

  18. queries queries queries RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF Architecture Virtual (derived) RDF

  19. RDF: framework for writing connections Triplets of • Subject • Property (or Predicate) • Object URI’s identify things, i.e. most of the above Namespaces are used as a convenient shorthand for the URI’s URI’s do not need to resolve

  20. Datatype Properties {WOA} dc:title “NOAA NODC WOA01” {WOA} dc:description “NOAA NODC WOA01: World Ocean Atlas 2001, an atlas of objectively analyzed fields of major ocean parameters at monthly, seasonal, and annual time scales. Resolution: 1x1; Longitude: global; Latitude: global; Depth: [0 m,5500 m]; Time: [Jan,Dec]; monthly”

  21. Object Properties {WOA} iridl:isContainerOf {Grid-1x1}, {Grid-1x1} iridl:isContainerOf {Monthly}

  22. WOA01 diagram

  23. Standard Properties {WOA} dcterm:hasPart {Grid-1x1}, {Grid-1x1} dcterm:hasPart {MONTHLY} Alternatively {WOA} iridl:isContainerOf {Grid-1x1}, {iridl:isContainerOf} rdfs:subPropertyOf {dcterm:hasPart}

  24. Data Structures in RDF Object properties provide a framework for explicitly writing down relationships between data objects/components, e.g. vague meaning of nesting is made explicit Properties also can be related, since they are objects too {SST} rdf:type {cfatt:non_coordinate_variable}, {SST} cfobj:standard_name {cf:sea_surface_temperature}, {SST} netcdf:hasDimension {longitude}

  25. Virtual Triples Use Conventions to connect concepts to established sets of concepts Generate additional “virtual” triples from the original set and semantics RDFS – some property/class semantics OWL – additional property/class semantics: more sophisticated (ontological) relationships SWRL – rules for constructing virtual triples

  26. Define terms • Attribute Ontology • Object Ontology • Term Ontology These are different ways RDF can be used

  27. Attribute Ontology • Subjects are the only type-object • Predicates are “attributes” • Objects are datatype • Isomorphic to simple data tables • Isomorphic to netcdf attributes of datasets • Some faceted browsers: predicate = facet e.g. longwell from MIT

  28. cf-att – CF transcribed

  29. cf-att with some attributes

  30. RDF helps decontextualizes

  31. Object Ontology • Objects are object-type • Isomorphic to “belongs to” • Isomorphic to multiple data tables connected by keys • Express the concept behind netcdf attributes which name variables • Concepts as objects can be cross-walked • Concepts as objects can be interrelated

  32. Example: controlled vocabulary {variable} cfatt:standard_name {“string”} Where string has to belong to a list of possibilities. {variable} cfobj:standard_name {stdnam} Where stdnam is an individual of the class cfobj:StandardName

  33. Example: controlled vocabulary Bi-direction crosswalk between the two is somewhat trivial, which means all my objects will have both cfatt:standard_name and cfobj:standard_name

  34. Example: controlled vocabulary If I am writing software to read/write netcdf files, I use the cfatt ontology and in particular cfatt:standard_name If I am making connections/cross-walks to other variable naming standards, I use cfobj:standard_name

  35. Some cf-obj classes

  36. Some cf-obj classes

  37. Term Ontology Concepts as individuals Simple Knowledge Organization System (SKOS) is a prime example standard_name as object would be such

  38. Nuanced tagging Concepts as objects can be interrelated: specific terms imply broader terms Object ends up being tagging with terms ranging from general to specific. • Search can then be nuanced • tagging can proceed in absence of perfect information • Partial information can be written down

  39. CF standard names .. I would add that standard names alone (in the cases where a standard name is sufficient) have the same kind of role as common concepts. The definitions of standard names allow some vagueness, though some are more precise than others, because their role is to indicate which things should validly be regarded as the same thing by visualisation and processing softwareJonathan Gregory 04/22/08 23:22:01 http://cf-pcmdi.llnl.gov/trac/ticket/24

  40. CF standard regions I don't think the regions can be exactly standardised, because part of the reason for having names is in order to be somewhat "vague". Just as a common standard name is given to quantities from different data sources when those quantities are regarded as comparable, the same standard region name would be given to data which represent the same region in a way which is regardedas comparable. For instance, different GCMs do not have exactly the same shape for the Atlantic Ocean, but Atlantic meridional overturning streamfunctions are calculated from each model, and these are regarded as comparable.Jonathan Gregory cf-metadata@cgd.ucar.edudate Wed, Aug 27, 2008 at 5:11 AM

  41. I.E. In other words, the broader the standard, the more vague. On the other hand, we can say something. In fact, we can say quite a lot about how these terms interrelate. And how they relate to less broad, less vague systems.

  42. What we can do easily Establish URI’s for the concepts in CF (standard_names, standard_regions, the attributes themselves) so that statements can be written about them in XML and RDF. Establish a machine-readable version of http://www.unidata.ucar.edu/software/netcdf/conventions.html so that we can write code to extract (decontextualize) metadata from netcdf files Start writing down the relationships between the concepts. Agree on a cf-att ontology, and work on cf-obj so that we can connect with other conventions. Set a convention for explicitly labeling netcdf attributes with their convention, i.e. namespace labels so that process of figuring out which convention covers which attribute is purely gramatical Set a convention for referring to a URI-identified concept in a netcdf file

  43. Search Interface • Items (datasets/maps) • Terms • Facets • Taxa

  44. Search Interface Semantic API {item} dc:title dc:description rss:link iridl:icon dcterm:isPartOf {item2} dcterm:isReplacedBy {item2} {item} trm:isDescribedBy {term} {term} a {facet} of {taxa} of {trm:Term}, {facet} a {trm:Facet}, {taxa} a {trm:Taxa}, {term} trm:directlyImplies {term2}

  45. Faceted Search w/Queries http://iridl.ldeo.columbia.edu/ontologies/query2.pl?...

  46. queries queries queries RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF Architecture Virtual (derived) RDF

  47. IRI RDF Architecture Data Servers MMI Ontologies JPL bibliography Start Point Standards Organizations RDF Crawler Location Canonicalizer RDFS Semantics Owl Semantics SWRL Rules SeRQL CONSTRUCT Time Canonicalizer Sesame Search Queries Search Interface

  48. Cast of Characters NC – netcdf data file format CF – Climate and Forecast metadata convention for netcdf SWEET - Semantic Web for Earth and Environmental Terminology (OWL Ontology) IRIDL – IRI Data Library

  49. CF attributes NC basic attributes IRIDL attributes/objects CF data objects CF Standard Names (RDF object) SWEET Ontologies (OWL) Location CF Standard Names As Terms IRIDL Terms SWEET as Terms Search Terms Gazetteer Terms

  50. Thoughts • Pure RDF framework seems currently viable for a moderate collection of data • Potential for making a lot of implicit data conventions explicit • Explicit conventions can improve interoperability • Simple RDF concepts can greatly impact searches

More Related