Roadmap for Satellite Data in the Advanced Weather Interactive Processing System (AWIPS) GOES Users’ Conference Madiso

1. Roadmap for Satellite Data in the Advanced Weather Interactive Processing System (AWIPS) GOES Users’ ConferenceMadison, WI Deirdre Jones NOAA’s National Weather Service Office of Science and Technology Systems Engineering Center (SEC) November 5, 2009

2. Outline • Challenges meeting new customer weather needs • Architectural Vision • AWIPS Plan to Achieve Architecture • Next Steps • Summary

3. Customers Need Improved Weather Services • Speed at which decisions are made • Demand for decision support services is increasing • US industry needs the most accurate, accessible, timely and reliable weather data to make critical decisions that impact our national economy • Aviation weather impacts were $41B in 2007 • U.S. modeling and data assimilation lags other nations. We must do better to give the U.S. a competitive advantage in the global economy • Federal deficits and resource constraints • Integrated observations • More efficient R-T-O (projects, modeling) • Every dollar counts!

4. Environmental Data Challenges • Huge data explosion • Rapid data assimilation requirements (e.g., NextGen)—people, models • Demands on data management architecture • Data access on-demand within resource constraints • Integrating all observing data sources to achieve desired effect and outcome

5. Data Volume (non-cumulative view) AWIPS SBN 3-18 Mbps NESDIS Archive Data Rate Projections w/MPAR w/GOES-S w/GOES-R w/NPOESS C2 w/NPOESS C1 w/NPP

6. Importance of Weather Information Database (WIDB) Weather Industry Private Industry Observations Forecasting Private Sector Numerical Prediction Systems Network Enabled Operations Postprocessed Probabilistic Output NWS Forecaster Radars Data Integration WIDB Cube Aircraft Automated Forecast Systems Surface Forecast Integration Soundings Grids Decision Support Systems Custom Graphic Generators Custom Alphanumeric Generators Governmental Decision Making

7. Infrastructure Readiness Approach • View requirements in larger context of all future programs • Develop strategic enterprise infrastructure plan and roadmap for building infrastructure capability over next 10-15 years • Near term tasking to complete analysis of the GOES-R impact on NWS architecture: AWIPS, NCEP and NWS Telecommunications Gateway (NWSTG) (FY08-FY10) • Assess other aspects of the enterprise, as well as other system changes (FY10) • Select architecture for efficient access to new satellite data and products for forecast and warning operations (1QFY11) • Initiate acquisition (FY 11-12)

8. Customers • Experts • Informed • Novice Governance (e.g., SLAs) Data Architecture (e.g., GEO-IDE, IOOS) Service Registry Infrastructure Services Security e.g., Threat Detection, Network Scanning Transporte.g., LAN / MAN / WAN/Internet Commoditye.g., Authentication, Discovery Core e.g., Common, Storage, Compute, Enterprise Mgmt NOAA CIO's Architectural Vision Presentation Tier (i.e., Web Presence) Mobile Devices Portals Collaboration Mission Services Web Service Interfaces- Utilizing Open Standards & Protocols Critical Support (Satellites & Mission Support) Climate Preparation & Response Ecosystem Management & Protection Modeling & Observing Infrastructure Weather & Water Preparation & Response Commerce & Transportation Efficiency, Safety, and Environmental Soundness

9. NOAA Infrastructure Framework • Consider environmental data changes in context of NOAA’s full infrastructure and in concert with each other • Service oriented enterprise focused on data and information • Infrastructure services: core, commodity, transport, and security • Transport migration: consolidate NOAA’s disparate networks into NOAANet architecture • Mission services: monitoring and observing, modeling and forecasting, disseminating, ecosystem management, etc. • Data architecture: data description, context, and sharing • Governance: service level agreements, data management • Dissemination Services • To include emerging technologies and Internet services: portals, mobile devices, and collaboration

10. Notional Architecture

11. AWIPS in the GOES-R EraFitting the Pieces Together • Flexible software infrastructure • Communications bandwidth -- satellite broadcast and terrestrial networks • Data distribution paradigm -- push and pull • SEC leading effort to prepare the enterprise as well as planning AWIPS improvements to support future data • Robust infrastructure with capacity and throughput for larger volume/higher resolution data

12. AWIPS II Software Re-Architecture Approach • Perform “black-box” conversion • Preserve existing functionality, • Preserve look and feel of today’s system • Maintain functionality • Deployed system current with deployed AWIPS capability (i.e., OB9) • Use open source projects - No proprietary code • JAVA and open source projects enable AWIPS II to be platform and OS independent • No plans to move from Linux • AWIPS II enables collaborative development • Operating System and platform independence allows non-Linux based research to be easily integrated into AWIPS II

13. AWIPS II Software Re-Architecture Outcome • Improved software architecture and capabilities to accommodate new data and services • Services oriented architecture • More flexible in production/delivery • Increased access to data for decision making • Improved RTO efficiency through use of AWIPS Development Environment • Enables future capabilities (e.g. flexible data delivery, thin client, collaboration)

14. Client/Presentation Services JMX CAVE External Programs Mission Services Layer Mbean ProductSrv IngestSrv NotifySrv Hydro Models AdapterSrv AutoBldSrv PersistSrv <<Java>> HDF5DataStore <<Java>> DataLayer IndexSrv SubscribeSrv UtilitySrv LAPS Security Services /Demilitarized Zone (DMZ) Enterprise Service Bus - Communication <<abstract>> BaseDao PurgeSrv StagingSrv FORTRAN/C/C++ Command Line Programs Data Access Layer Hibernate HDF5 API Localization Store Platform Layer PostgreSQL HDF5 Data Persistence Store Metadata Index Spatial Index AWIPS SOA Architecture Logical Layered View

15. AWIPS Product Suite On NOAAPort Today* * NOAAPort disseminates over one million products (30+GBytes) every day

16. GOES Products Flowing into AWIPS Product Name WMO Dissem. Format (AWIPS ID) Approx. Approx. Daily Header Path Frequency Volume Imager-based Imagery TIG? ii 1 GINI 30/15/7 min 27200 Mbits TIC? ii Sounder-based Imagery TIG? ii 1 GINI hourly 216 Mbits DPI TIGN ii 1 GINI hourly 280 Mbits Soundings JUTX ii 2 BUFR 30 min 312 Mbits AE SPE Imagery ZETA98 2 GRIB1/Xmrg hourly & event 216-320 Mbits driven Manual SPE Imagery ZEGA98 SPE Text TXUS20 2 Text event driven 0 Mbits SPENES Hi Den Winds (BUFR) J?CX ii 2 BUFR 1-3 hourly 136 Mbits Hi Den Winds (Text) TWNA ii 2 Text 1-3 hourly 80 Mbits TWSA ii (none) ASOS Cloud Products TCUS ii 2 Text hourly 1.6 Mbits SCP?R i Geodetic Subpnt Predictn. TBUS ii 2 Text daily 0 Mbits WBCSAT5/6 Administrative Messages NOUS71 2 Text event driven 0 Mbits NOUS72 ADMNES CONUS RTMA (ECA) LAMA98 2 GRIB2 Hourly 56 Mbits GOES Data in AWIPS Today

17. Future NOAAPort Product Suite* • Environmental Satellite Product Additions • POES AVHRR Imagery (2010) • NPP & NPOESS (From NPP, beginning 2011) • GOES-R Series (2015) • Currently defining process for building, evaluating and implementing new products in the AWIPS II framework • NCEP Model Product Additions • High Resolution Window Model (2010) • GFS-Based DNG (2010) • Bias-corrected SREF (2011) • Numerous additional enhancements to NCEP (ongoing) • Radar (WSR-88D) Product Additions • Dual Polarization (2011) • Higher Pixel Resolution (2013) * Only selected major planned additions are shown; pending bandwidth expansion; dates tentative

18. AWIPS Communications Improvements • New paradigm for data distribution • User readiness working group to work with users/ customers to define product access and delivery reqts. • Migrate AWIPS wide area network (WAN) to NOAAnet • Increased reliability • Increased bandwidth • Enables any-to-any network topology • Increase AWIPS SBN bandwidth to 45 Mbps (full transponder) • 5 times current capacity • Enables flexible delivery over satellite

19. AWIPS Data DeliveryPush/Pull Concept • Robust infrastructure to support “intelligent” access (push/pull) to non-local datasets • Key Benefits • Mitigates impacts on SBN by addressing significant growth in data volumes, e.g., ensembles, GOES-R, NPOESS • Allows users to access just the data they need by space, time, parameter • Enables synergy with NextGen net-enabled data access requirements

20. NWS Infrastructure ArchitectureImplementation Schedule Deployment January 2011 NDE “Distribution Build” Testing, NPP Launch Oct 2015 GOES-R Launch July 2013 GOES-R “Data Operations Tests” April 2013 NPOESS C1 Launch IOC MOC

21. Summary • NOAA is working on the enterprise infrastructure to accept, manipulate, and use data from new capabilities • Ensure all NWS systems on track for user readiness • Complete assessment • Address in design solution • Build toward NOAA target architecture • Efficiencies to reduce cost needed; seek out new technologies for computing and data integration architectures

22. Back-up

23. NEXRAD GOES/POES/NPOESS Warnings NCEPModels Watches Advisories ASOS Forecasts Buoys, River Gauges What is AWIPS? 169 separate AWIPS systems at 137 geographical locations AWIPS Communications AWIPS Workstations and Servers Service provided to 3066 US Counties 24 hrs/day, 365 days/yr. ~900 Workstations (total) ~1200 Servers (total)

24. NOAAPort NOAA’s Satellite Broadcast Network POES GOES AMC-2 Satellite Other Environmental Satellite Products AVHRR HIRS/AMSU DCS GVAR NOAA/NWS NCEP Guidance & Model Products Satellite Product Generation AWIPS NCF TOC NWSTG NOAAPORT Uplink Facility (NY) Satellite imagery NOAA/NESDIS NOAAPORT Users Includes Many Users – Inside and Outside of NOAA WFO/RFC Forecasts, Watches Warnings Hydromet Sensor Products (e.g., radar, ASOS, Profiler, River Gages)

25. NOAAPort Technical Specifications • Product Stream Assembly: AWIPS Network Control Facility (Silver Spring Md) • NOAAPort Satellite: SES Americom (AMC-2), Transponder: 13C • NOAAPort Satellite Location: 101◦ West • Downlink Signal: C BandNetwork Capacity: 10 Mbits/sec (approximate) • Transmission Protocol: DVB/S - Digital Video Broadcast by Satellite • Product Identification: Headings (NOAAPort, WMO/AWIPS & product) • Major Product Formats: GRIB, BUFR, GINI, Radar Level III, Text, Redbook • Compression: zlib for satellite imagery and most radar products; GRIB2 supports JPEG2000 compression • There are four NOAAPort “data streams” or “channels”– each with a unique DVB PID. • GOES/NESDIS • NCEP/NWSTG • NCEP/NWSTG2 • OCONUS Imagery/Model/DCP

26. Other Satellite Data in AWIPS

27. Glossary of Terms AWIPS: Advanced Weather Interactive Processing System CLASS: Comprehensive Large Array-data Stewardship System DMSP: Defense Meteorological Satellites Program GOES: Geostationary Operational Environmental Satellite GOES-R: Geostationary Operational Environmental Satellite R-Series GOES-R: Geostationary Operational Environmental Satellite S-Series IMETS: Incident Meteorologists IOC: Initial Operating Capability MetOp: Meteorological Operational Satellite MPAR: Multi-function Phased Array Radar Mid Operating Capability NDE: NPOESS Data Exploitation NESDIS: National Environmental Satellite, Data, and Information Service NextGen: Next Generation Air Transportation System NEXRAD: Next-Generation Radar NPOESS: National Polar-orbiting Operational Environmental Satellite System NPP: NPOESS Preparatory Project POES: Polar Operational Environmental Satellite R-T-O: Research to Operations SBN: Satellite Broadcast Network