Title

1. Title Improving NESDIS Data Management Planning Helen M. Wood Scott Hausman NOAA NESDIS

2. NESDIS DATA & INFORMATION GOAL • Provide a comprehensive and trusted set of products to serve users’ needs • Ensure data reaches new users to assure maximum benefit to the nation and scientific community • Strengthen support of NOAA’s data stewardship needs

3. Strengthening NESDIS Data Management Planning • Compliance with NOAA and Federal policies and best practices • Consistency across NESDIS Observing Systems, environmental data, and products • Integrate end-to-end data management into programs and projects

4. Overarching Principles • Data should be archived and accessible • Adequate resources for end-to-end management • Management activities should involve users • Interagency and international partnerships • Metadata are essential • Expert stewards required for management • Process to decide what data to archive • Archive must support discovery, access, and integration • Effective management requires a formal, ongoing planning process National Research Council Committee on Archiving and Accessing Environmental and Geospatial Data at NOAA, 2007

5. Challenges • Data management planning for NESDIS Observing Systems did not fully address data Archive elements • Planning for Archive readiness not well integrated with design/development/operations • Responsibilities for Archive elements distributed across NESDIS organizations • Detailed policy for NESDIS Observing Systems & Data Processing Systems planning needed

6. Key Definitions Archive An organization or facility of people and systems that has accepted the responsibility to preserve information. NCEI constitutes the official NOAA archive. NCEI maintains, processes, distributes, and provides long-term stewardship for most of NOAA’s environmental and geospatial data, and provides a broad range of user services.

7. Key Definitions Archival Functions The full range of archival information preservation functions defined in the Open Archival Information Systems Reference Model: • Ingest • Archival Storage • Access • Data Management • Preservation Planning • Administration • Data Stewardship

8. Key Definitions Archive System The information technology systems that enable the Archive. NCEI operates the Archive System for NESDIS. The responsibility for planning, developing, operating or maintaining information technology systems that support the Archive may be assigned to other organizations within NESDIS.

9. Recommendations • Ensure the long-term preservation of Environmental Data in accordance with NAO 212-15. • Ensure, at minimum, that Original Data will be archived. • Determine the cost effectiveness of on-demand regeneration of products derived from Environmental Data vs. long-term preservation of the derived products

10. Recommendations • Ensure that required Data Management Plans are written, reviewed, submitted to EDMC, followed throughout the Data Lifecycle, and revised as circumstances warrant. • Ensure that the Archive is involved throughout the concept, design, development, and operations phases of NESDIS Observing Systems and associated Data Management Systems. • Ensure that all data, relevant metadata, and derived products are archived, unless an explicit decision not to archive has been made pursuant to the NOAA Procedure for Scientific Records Appraisal and Archive Approval.

11. Recommendations • Include the review of the Archive System in lifecycle reviews of Data Processing Systems associated with the development of new Observing Systems. • Provide sufficient development, operations, and maintenance resources to support Archive functions and Archive Systems for NESDIS Data Producers and associated Data Management Systems. • Ensure that all its data are publicly discoverable and accessible.

12. Recommendations • Employ standards, conceptual models, methodologies, and terminology that adhere to accepted Federal and international standards for data storage, accessibility, metadata, interoperability, and long-term preservation. • Adopt an acquisitions strategy for NESDIS Observing Systems and associated Data Management Systems that secures all necessary rights and provisions for the data to be archived and used in a free and open manner by NOAA and NOAA external customers. • Any exception to these requirements shall require approval by the NESDIS Assistant Administrator, in consultation with the NOAA Observing Systems Council where required.

13. Next Steps • Define implementation details, e.g., applicability, review and approval process, etc. • Update current Observing System Programs and Associated Data Processing System activities to comply. • Integrate data management planning into NESDIS Systems Engineering Procedural Requirements.

14. Archive must be involved at all stages of observing system development • Archive and stewardship should not be an afterthought. • Early and regular involvement of the Archive saves funds, facilitates future information services, and provides for a better user experience. Preliminary Agreement Submission Agreement Archive Specifications Archive Requirements Preservation and Distribution

15. NEXT STEPS Questions?