1 / 18

November 7th, 2012

EOL Data Services update: Legacy Data, EarthCube and the Earth Science Information Partners (ESIP). November 7th, 2012. Mike Daniels. Legacy Data. From the 2011 EOL Science Review Team Report.

orly
Download Presentation

November 7th, 2012

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. EOL Data Services update: Legacy Data, EarthCube and the Earth Science Information Partners (ESIP) November 7th, 2012 Mike Daniels

  2. Legacy Data

  3. From the 2011 EOL Science Review Team Report “The EOL effort to rescue and digitally archive historical field data that are presently stored in older paper, chart, and tape formats will be an extremely valuable information asset for research requiring long-term records.” “Continue progress toward and maintain a priority for making current and historical field data easily accessible”

  4. Climate Science 1.0: Observational data is very similar to model data (2003!) Data Lessons Learned Engineering Units Metadata QC Parameter Metadata QC Derived Parameter Metadata QC Obs data Integrated Parameter Climate Science 2.0: Observational data is very different from model data Value Model data Time

  5. Legacy Data Activities, 1st proposed in Spring 2010 (FY11 Annual Budget Review) • Data and Metadata Rescue • Project Documentation • Collect, digitize and webify • Improve documentation • Digitize Aircraft Videos • Media and Format Migration • (e.g. GENPro to NetCDF) • NCAR Iron Mountain Archives • Embarked on two pilot projects: • Sounding Legacy Project (CSU) • T-28 Facility Archive (SDSMT) 1974

  6. Protecting the legacy of current and future datasets: the EOL Data Audit • What is the data flow (from sensor to archive) of all EOL datasets, including individual instruments and HAIS? Who is responsible for key aspects of DM for these datasets? Do we have backup copies of recently acquired data? • Does each dataset have the necessary metadata in a recognized format? Are these metadata sufficient? • Are all EOL data secured in a common archive that is linked to other portals? • Are the data formats and access methods we use for these datasets today adequate for future communities?

  7. New Programmer/Technician Position One-time funds of .40 FTE allocated in FY13: C. Brooks Snyder, a student assistant in EOL, has accepted the position!

  8. EarthCube earthcube.ning.com

  9. EarthCube: What is it? Addresses data issues from point observations to peta-scale models Multi-disciplinary across the geo- bio- cyber- sciences and “from sun to the center of the earth” A Data and Knowledge Management system for the next decade and beyond Community, not NSF, driven Proposes to reduce efforts spent on data issues to free up time for research Strongly supported by the GEO Directorate

  10. What has happened thus far • “Charette” #1 November 2011, 106 whitepapers received which created a list of common E3 capabilities (Mike Daniels participated virtually) • EarthCube Stakeholder Alignment Survey (sent to EOL Scientists) • Early 2012: “Community Groups” formed to gather user requirements • “Concept Award Groups” formed to develop prototypes • Less formal “Special Interest Groups” address EarthCube-related issues not covered by the existing Community and Concept Groups. • Charette #2 June 2012 (KateYoung and Mike Daniels in attendance, other EOL staff participated virtually) • Defined roadmaps of next steps for moving groups forward • Vignette’s (use cases) from EOL submitted to EarthCube’s Workflow Community Group • E^3 moving toward domain-specific workshops and community events • Governance Group has plans to be in place by Spring 2013

  11. EarthCube Groups and Awards Working Groups – Form Requirements • Governance • Semantics and Ontology • Data Discovery, Access and Mining • Workflow Concept Awards – Develop Prototypes • Earth Systems Modeling • Interoperability • Brokering • Layered Architecture • Web Services Special Interest Groups • Early Career • Data integration for Biology and Earth Science • Physical Samples Cyberinfrastructure • Education and Workforce Development • Software Engineering Community of Practice • Website and Collaboration Environment

  12. Real-time data E3 workshop • Set out concrete example cases, showcasing how real-time data can be leveraged and addressed within EarthCube. • Bring together those in the real-time data community. • Focus on modeling efforts that require data quickly to aid in emergency management, forecasting, decision-making, risk assessment, deployment of resources, etc. • Determine how the needs of real-time data researchers, developers, data providers and stakeholders affect the overall EarthCube system architecture. • Discuss framework architectures and ideas relating to the distributed, online, highly interactive and time critical real-time operations. • Discussions of adaptive sensing networks, un-tethered sensing systems and distributed real-time observations occurring around the globe. • Explore concepts related to real-time quality control and monitoring. • PI: M. Daniels, Co-PIs: Dr. V. Chandrasekar (CSU), Dr. S. Graves (University of Alabama), Dr. B. Kerkez(University of Michigan), Dr. F. Vernon (UCSD)

  13. www.esipfed.org

  14. History and Purpose • Established to bring NASA groups together as an outcome of a National Academy report in 1998, UNIDATA was a Charter Member • Globe, NOAA, NSIDC accepted as members in 2004 but no NCAR(!) • I led an organization-wide effort for NCAR to become a member, we were accepted in January 2012 • ESIP Vision Pillars • Provides the Earth science informatics intellectual commons to drive innovation • Is the trusted community authority that supports the integration of science and data into mainstream use • Leads the development of the science data information profession • Achieves sustainability through diversification, global partnerships and partner recognition • Members are recognized as leaders in Informatics at AGU and other venues • Represents a forum to allow EOL and NCAR to develop and expand our data services tools and technologies

  15. Organic Growth of Topics (4) Formal, Chair is member of ESIP Assembly: e.g. Committee on Data Preservation and Stewardship StandingCommittee (6) Fixed Duration, Task Oriented: e.g. Working Group on Data Management Training Working Group (10) Very Informal, Monthly Telecons: e.g. Cluster on Climate Forecasting Conventions Cluster

  16. Annual Meetings: Winter Business and Summer Technical Semantic Web: Deborah McGuinness Comparison of EarthCube and ESIP Open Source Summit

  17. FUNding Friday! This year, all proposers were winners of a $5K award!

  18. Mike Daniels, EOL/CDSdaniels@ucar.edu Thank you! NCAREOL

More Related