NOAA Earth Information Service (NEIS) Overview GSD Technical Seminar February 28 th , 2012

1. NOAA Earth Information Service (NEIS)OverviewGSD Technical SeminarFebruary 28th, 2012 ESRL Global Systems Division

2. NEIS Agenda Introduction Evaluation and Findings Concept Activities and Next Steps Demonstration ESRL Global Systems Division

3. NEIS Background • GSD Director Discretionary Funds (DDF) Proposal • NEIS Requirement • Fast access to any data, any time, right now, on any platform. • NEIS Group Objectives • Build upon NEIS work in the past • Assess GSD capabilities pertinent to the concept of NEIS. • Evaluate existing, developmental technology, and solutions relevant to concept. • Evaluate envisioned needs of primary users. • Provide recommendation of how to best use GSD resources to contribute to a National capability. ESRL Global Systems Division

4. NEIS Team Research Team Jebb Stewart - Lead Chris MacDermaid Woody Roberts Darien Davis Jeff Smith Mike Romberg Paul Schultz Development Team JulienLynge Jeff Smith Chris MacDermaidJebb Stewart Randy Pierce ESRL Global Systems Division

5. NEIS History January 18, 2007, Roberts, et al. A OneNOAA Concept Prototype for Data Viewing,  23rd IIPS. March 10, 2009, Hansen, et al. Earth Information Services, GSD Technical Review on GFE and Other Activities. ESRL Global Systems Division

6. NEIS Evaluation • Interviewed/Discussed concepts with variety of groups across NOAA Line Offices. More than 15 individuals. • Questions Asked: • How are people using your data? • How are you using NOAA data yourself? • What tools/systems/services/standards are you using? • What related recommendations/lessons learned exist? • What obstacles or roadblocks have you encountered with using/disseminating NOAA data? • Listened for items to help facilitate projects or help NOAA data become more interoperable. ESRL Global Systems Division

7. Interviews

8. NEIS System Evaluation xkcd.com • Part of the evaluation looked at existing systems and standards including: • THREDDS LAS ERDDAP • Google Earth NASA World Wind GeoPortal • MERMAid REVERB iRODS • Grads Data Server More…. • Lot of Great Technology, which can be leveraged, no ‘one’ solution. ESRL Global Systems Division

9. NEIS Committee and Group Involvement • GSD is aware of many ongoing activities • We have active membership in a variety of committees and groups including: • OGC Standards Committees • EDMC • DMIT • UAF • NextGen/NNEW program • FIM/WRF Modeling • GIS Committee • GSD Central Facility has vast experience in working with data and standards ESRL Global Systems Division

10. NEIS Findings • Existing metadata systems contain stale or obsolete information and are not useful • Search for common term like wind returns hundreds of results. • Expertise required to understand what metadata you are looking for. • Many “Stovepipes” but no ability to combine/integrate data from different disciplines (Ecological, Meteorological, Oceanographic, Fisheries, etc...) • Data sets are large and getting bigger. Future data sets will be difficult if not impossible to move around network to consumer applications. • Cross data source data processing service does not exist. Some data processing services exist but usually only for data available from that particular service. • Feedback (broken links, data usage, etc…) to data providers and consumers is non-existent. • Many more… ESRL Global Systems Division

11. Data Integration • Physical • Chemical • Biological Carbon Tracker FIM • Biological, Chemical, and Physical data is all interrelated • NEIS allows data integration in 4D time and space, providing a scientific tool for everything NOAA does professionally and the world NOAA affects. ESRL Global Systems Division

12. NOAA Vision - 2020 TerraViz Ability to discover, integrate, understand all NOAA data (past, present, and future) from homepage. ESRL Global Systems Division

13. NOAA Vision - 2020 • Most of NOAA's environmental data will be accessible via web services and will be described by good metadata • A central and intelligent search engine, that has harvested the metadata of NOAA's datasets, will exist to find data • NEIS server side analytics/processing will enable subsetting, aggregating, diff'ing, and other operations to be performed on data • Data will be efficiently streamed to a variety of platforms and clients, including mobile devices • TerraViz will be a general purpose client running on multiple platforms capable of searching for NEIS data and visualizing it in novel ways • Framework is extensible, other groups can add knowledge and services. ESRL Global Systems Division

14. NEIS – Concept Components

15. Looking to the Future • Build new Graphical User Interface system integrating and leveraging new and emerging technologies to meet NEIS goal ‘any data, any location, any platform, now’ • Perform processing within cloud environment and with high speed connectivity to data sources, taking advantage of large processing power within clouds. • Send graphics and server side processed/rendered/streamed data to GUI, improving bandwidth utilization. • Take advantage of fast networking to make remote requests and processing appear like local application. • Similar to how the new Amazon Silk Browser works. ESRL Global Systems Division

16. Looking to the Future – Part 2 • Improve existing Metadata services making information useful. • Improve searching by understanding what data means and providing improved filtering capability similar to how airline or hotel type search engines work. • Incentivize/encourage people to use proper Metadata. • Create a Metadata Dashboard to: • Gather information from users on relevance, ratings, usage patterns, search key words used (ie crowd sourcing). • This information can be used to determine what users are searching for, how they are finding data, what data they are not finding, etc... • Provide feedback on broken links, service uptime • Provide feedback on adherence to standards (Services, Metadata). • Provide feedback to users of similar or related data sets. ESRL Global Systems Division

18. ESRL Global Systems Division

19. ESRL Global Systems Division

20. Impacts • NOAA data ready for action. Services model facilitates agile response to events. Services can be combined or reused quickly • Any data available through NEIS system can be operated on or combined with other data. Integrated standardized formats and access. • New and Existing systems have access to wide variety of NOAA data. Any new data added, easy incorporated with minimal to no changes required. ESRL Global Systems Division

21. NEIS Use Case – Deepwater Horizon There was no way to pull disparate data sets into one display in order to track the impacts of the oil spill Ocean currents Ecosystems Weather Location of booms, ships, and other response equipment Many GIS developer labor hours spent working on converting and formatting data to make it available.

22. Are We Prepared? • What would happen if another hurricane or other disaster occurred? • With a framework such as this: • Improved searching capability would allow those in need to quickly and easily find pertinent data to the situation. • All data available through framework would be readily accessible to be integrated across 4D (time and space). • New data that is properly documented could be added to services and become immediately available to the framework of services. Spin up time for new data reduced dramatically. • Because framework exists as a collection of services, any changes or modification can be completed quickly without taking entire system down. ESRL Global Systems Division

23. Demonstration ESRL Global Systems Division

24. Prototype Demonstration Background • What is TerraViz? • A system developed from the ground up by GSD based on Unity3D. • There are no licensing restrictions in distributing the TerraViz executable. • Unity3D is a well supported commercial game and data visualization engine with 750,000 active users • Unity/C# is designed for 3D development, making it easy to create multiple views, create 3D meshes representing environmental objects, etc. • Develop once, then will run under Windows, Mac, web browsers, iPhones, iPads, Android devices, and even game systems (Wii, Xbox 360, etc.) • We can modify the user interface and add any feature we need. • TerraViz Handles millions of polygons and performs quickly • TerraViz is one possible visualization tool built to develop, test, and showcase NEIS framework. ESRL Global Systems Division

25. Why not Google Earth • Google Earth, works great for certain applications and static products, however: • Handles ~10,000 polygons before slowing to a crawl. • Closed system (Google Earth source code not available) so we can’t add needed features. • Google Earth plug-in works on Windows and Mac 32-bit browsers only (won’t run on iPhones, iPads, or Android devices.) • Developing GE plug-in user interfaces in browser-dependent Javascript is tedious. • Very Limited integrated data discovery or browsing, most uses require a separate application. • Limited animation/data interaction support. • Limited support for non-static data (ieTime Series)