The Proposed Next-Generation NCEP Production Suite

1. The Proposed Next-GenerationNCEP Production Suite NCEP Production Suite Weather, Ocean, Land & Climate Forecast Systems EMC Senior Staff January 2007

2. Overview • Preparing for the future • Production Suite: Conceptual prototype • Benefits • Summary

3. Preparing for the Future (1) • Improved forecast services • Greater focus on high-impact events • Additional environmental information service responsibilities • Provide more information to users and access to more info • Support forecast offices • Efficient Grid Initialization (e.g. SmartInit) • Analysis of Record (and RTMA) • Probabilistic and ensemble methods • Respond to external (NRC) reports • “Completing the Forecast” • “Fair Weather” • Respond to NOAA Science Advisory Board reviews • Ocean modeling (National “backbone”) • Hurricane intensity (ensemble-based system)

4. Preparing for the Future (2) • Observations (number and availability) • Advanced Polar and Geostationary sounders (~100 X greater) • NPOESS (<60 minutes globally) – 2012-2015 (or later) • METOP (1-4) – 2007 • NPP (90-120 minutes globally) – 2009 • GOES-R – 2013 (or later) • Next-generation Doppler radar • Advanced post-processing techniques for multi-model ensemble (e.g. NAEFS project) • Bias correction • 2nd moment correction • CPC “consolidation” to quantify “value-added” • Advanced dissemination strategies • E.g. NOMADS (“Fat server/Thin Client” technology) • Next-Generation Air Traffic-control System (NGATS) • Geographically consistent solutions • Global to terminal scales • At least hourly updating globally

5. Preparing for the Future (3) • Three principals for moving forward • Maturing, ensemble-based, probabilistic systems offer the most potential benefits across wide spectrum of forecast services “Model of the day” is not a scientifically supportable solution for the future • Ensemble composition • Managed component diversity • Components must be institutionally supported (operational or major research institution) • Product delivery • Time is critical (perishable product) • Information availability must be maximized

6. Production Suite: Conceptual PrototypeProducts • Three levels of information • Routinely delivered • Pointwise, single-valued, downscaled Most Likely Forecast from all available guidance on NDGD grid • Description of forecast uncertainty through probability density function (pdf) • “On-demand” (via publicly accessible server) • Individual ensemble member forecasts available • Prototype: NOMADS

7. Production Suite: Conceptual PrototypeApplication Areas • Focus on high impact weather • Hurricane intensity (and track) and coastal impacts • Other “High impact” defined by • Users • Type of event • Goal: “warn on forecast” for highest resolution events (e.g. tornado) • Examples of other new applications • Surface transportation (e.g. “winter weather”) • Environmental monitoring (AQ + Atmos. Constituents) • Ocean (HABs & ecosystems, fog & visibility, coastal inundation, dynamic storm surge) • Hydrology (water quality, drought) • Hourly updating • RTMA  AOR (through Reanalysis/Reforecast) • Regional assimilation • Global assimilation+ + if requirement and available computing and human resources

8. Production Suite: Schematic Overview Analysis Global/Regional Model Domain Model Region 1 Model Region 2 • Concurrent execution of global and regional applications • More efficient execution of rapid updating • In-core updating for analysis increments • Regional (CONUS, Alaska, Hawaii, Caribbean & Puerto Rico) • Global (if requirements and resources) • All ensemble members may exchange information during execution • ESMF-based Common Modeling Infrastructure

9. Application Driver ESMF Superstructure (component definitions, “mpi” communications, etc) Dynamics (1,2) Physics (1,2,3) Analysis -------------- Other Forecast Systems Coupler 1-1 1-2 1-3 2-1 2-2 2-3 Multi-component ensemble + Stochastic forcing ESMF Utilities (clock, error handling, etc) Post processor & Product Generator Verification Resolution change ESMF* Compliant Component System * Earth System Modeling Framework (NCAR/CISL, NASA/GMAO, Navy (NRL), NCEP/EMC) 2, 3 etc: institutionally (non-NCEP) supported

10. NCEP Production Suite Weather, Ocean, Land & Climate Forecast Systems Next Generation Prototype Phase 4 - 2015 Computing factor: 81 Rap Refresh • Concurrent • GFS* • NAM • SREF • Hourly • GDAS • RDAS • Rapid Refresh • Expanded • Hurricane • capability (hires) • Hydro/NIDIS • Reforecast Reforecast SREF WAV HUR NAM CFS MFS Global GENS/NAEFS GFS RTOFS RTOFS AQ Hydro / NIDIS/FF AQ CFS & MFS Regional RDAS Hydro GDAS * Earlier delivery of GFS  concurrent combined products from NAM, GFS, SREF

11. NCEP Production Suite Weather, Ocean, Land & Climate Forecast Systems Next Generation Prototype Final – 2017+ Computing factor: > 240 GLOBAL NGATS SPACE WEATHER Rap Refresh Reforecast GENS/NAEFS HENS ECOSYSTEMS HUR SREF RTOFS CFS MFS Global >100% of 2015 computing NAM RTOFS GFS WAV Regional CFS & MFS AQ Hydro / NIDIS/FF AQ CFS & MFS Hydro RDAS GDAS

12. Summary of Benefits • GDAS and RDAS with advanced assimilation techniques • Use high density time and space observations more completely • Prepare for NPOESS, METOP, next-generation radar obs • Provide potential for expanded Rapid Refresh capability to serve NGATS • Concurrency and unified post-processing provide • Earlier delivery of GFS products • Real time boundary conditions for regional systems • Global and regional ensembles giving complementary uncertainty measures based on different physical mechanisms • Unified “D(prog)/Dt view from all guidance products • Reanalysis/Reforecast capability • Enables maximum forecast skill and independent skill assessment • Provides • Operationally supported probabilistic systems • Updated skill evaluation as forecast systems evolve • Grouping of regional ocean applications allows • Hurricanes to use real-time ocean state • Coastal applications to run concurrently

13. Summary • Phased evolution of the NCEP Production Suite • 2009-2015 • Results in • Improved services for high impact weather • Application of advanced data assimilation techniques for improved model initial conditions • More efficient • Use of computing • Incorporation of new product lines for improved services • Earlier product delivery • More uniform and informative product stream • Advanced ensemble suite including components supported outside NCEP • Improved statistical post-processing • Reforecast and Reanalysis become operationally supported • Consistent with existing ESMF & global data assimilation development and interagency collaborations with • NASA • DOD • NCAR

14. ThanksQuestions?

15. Extras

16. NCEP Production Suite Weather, Ocean, Land & Climate Forecast Systems Current - 2007 Current (2007) GDAS NAM anal GFS anal GFS SREF HUR NAM GENS/NAEFS RDAS AQ RTOFS CFS

17. NCEP Production Suite Weather, Ocean, Land & Climate Forecast Systems Next Generation Prototype Phase 4 - 2015 Rap Refresh Reforecast SREF WAV HUR NAM CFS MFS Global GENS/NAEFS GFS RTOFS RTOFS AQ Hydro / NIDIS/FF AQ CFS & MFS Regional RDAS Hydro GDAS

18. Community-based Development • Strategy and roles: • Focus on single component instead of entire model system • Collaborative, not competitive • NCEP/EMC • Maintains primary components for each part of Production Suite and for each application • Supports ESMF applications in operations • In collaboration with community • Integrates new ESMF-based components into operations • Performs final testing and preparation of upgrades of supported components in operations • Collaborators • Provide • Component upgrades to be tested in operational setting • Institutional support for their contributed components • Diversity and expertise complementary to operations • Work through DTC, JCSDA, CTB, etc.

19. Conceptual Prototype:Numerical Forecast Guidance (1) • Information should be optimally combined from all available sources • Domestic and international models (global, e.g. NAEFS and regional) • Same product format for all time scales (unified post-processing) • Progress in numerical forecast system development should not be constrained by post-processing • Improved products come from development of improved systems • Robust training and outreach program must • Accompany new probabilistic-based system • Support NWS Field Operations, commercial sector and international users • Support advanced dissemination of forecast information on all time and space scales

20. Conceptual Prototype:Forecast System Development Areas • Observations processing • JCSDA • Increased focus on quality control • Contribute to future observing system design • Data assimilation • Coordinated development of advanced techniques • Simplified 4-D Var (NCEP/EMC), annual updates thru 2008 • “Classical” 4-D Var (with NASA/GMAO) • Ensemble Data Assimilation (with ESRL, UMD and others) • Development coordinated with NASA-NOAA-DOD JCSDA • 2008 decision  2010 implementation • Better use of high time and space density, remotely-sensed data • Model accuracy (dynamics and physics) • Hybrid vertical coordinate (sigma-pressure-theta) • Semi-lagrangian, Semi or Fully Implicit • Advanced radiation, shallow convection, deep convection • Stochastic forcing • Land surface tiling • Increased collaboration with community (Test Beds) • Post-processing • Unified system across time and space scales and models • Includes bias correction and downscaling

21. NCEP’s Next Generation Operational Forecast System Commerce Energy Ecosystem Health Forecast Uncertainty Outlook Hydropower Agriculture Boundary Condition Sensitivity Guidance Reservoir control Recreation Threat Assessments Transportation Fire weather Type of Guidance Forecasts Initial Condition Sensitivity Flood mitigation Navigation Watches Warnings & Alert Coordination Protection of Life/Property Lead Time Minutes Hours Days Weeks Months Seasons Years “Forecast Countdown for the Seamless Suite”

22. Concurrent execution of global and regional forecast models (2) Analysis Global/Regional Model Domain Model Region 1 • Real time boundary and initial conditions available hourly • “On-demand” downscaling to local applications • Similar to current hurricane runs but run either • Centrally at NCEP OR • Locally (B.C, I. C. retrieved from on-line data at NCEP) • No boundary or initial conditions older than 1 hour • Flexibility for “over capacity” runs • Using climate fraction must be planned • No impact on remainder of services • Consistent solution from global to local with a single forecast system and ensembles providing estimate of uncertainty Model Region 2 Local Solution

23. ESMF Component Framework

24. NCEP Production Suite Weather, Ocean, Land & Climate Forecast Systems Current - 2007 Data processing Current (2007) GDAS NAM anal GFS anal GFS SREF HUR NAM GENS/NAEFS RDAS AQ RTOFS CFS

25. NCEP Production Suite Weather, Ocean, Land & Climate Forecast Systems Ideal State Rap Refresh DATA ASSIM GA RA GO RO HU Refcst Hydro / NIDIS/FF Med Range Refcst AQ SREF GENS/NAEFS CFS MFS Global RTOFS-ECOS WAV RTOFS HUR AQ Wk2 MFS & CFS GFS Regional NAM Hydro HOURLY

26. NCEP Production Suite Weather, Ocean, Land & Climate Forecast Systems Next Generation Prototype Phase 1 - 2009 Computing factor: 3 Reforecast NAM anal GFS Anal GFS WAV HUR SREF NAM • Added • 1-Hourly RDAS • 3-Hourly GDAS • Reanalysis/ • Reforecast GENS/NAEFS AQ RTOFS CFS 1-hourly RDAS (6) 3-hourly GDAS (2)

27. NCEP Production Suite Weather, Ocean, Land & Climate Forecast Systems Next Generation Prototype Phase 2 - 2011 Computing factor: 9 • Added • Hydro/NIDIS • products • Moved • GFS ½ h earlier • Expanded • Hurricane & wave • products • Incorporated • Multi-domain • rapid updating Reforecast SREF NAM Anal NAM GFS Anal GFS WAV HUR GENS/NAEFS AQ Hydro / NIDIS AQ RTOFS CFS & MFS RDAS GDAS

28. NCEP Production Suite Weather, Ocean, Land & Climate Forecast Systems Next Generation Prototype Phase 3 - 2013 Computing factor: 27 SREF Reforecast GFS WAV • Added • Flash flood • products • Moved • SREF concurrent • to NAM • Expanded • Reforecast • capability GFS Anal HUR NAM NAM Anal GENS/NAEFS AQ Hydro / NIDIS/FF AQ RTOFS CFS & MFS RDAS GDAS

29. EVOLUTION of the Next-Generation NCEP Production Suite (1) * Relative to NCEP’s 2007 computer

30. EVOLUTION of the Next-Generation NCEP Production Suite (2) + If positive upgrade to services * Relative to NCEP’s 2007 computer @ additional 3x computing upgrade in 2009 required

31. Traditional NWP Process

32. Future NWP Process THORPEX PROGRAM

33. Primary & Secondary Components and Models • Primary components • Observations ingest, processing and quality control • Data assimilation • Forecast model • Post-processing • Product delivery • Primary model • Used in data assimilation cycle • Supported by EMC for NCEP’s operations • Risk reduction • Optimum maintenance • Optimum enhancement • Continued exposure to observations • Model improvements impact analysis and forecast • Secondary model • Initialized from analysis • Not cycled • Must add value to operational system • Skill • Diversity • Unique application • Supported institutionally by external (to EMC) organization • E.g. Navy (through NUOPC) • First line of support at EMC (one person) • Applications • Ensemble membership (managed diversity) • Not fully supportable by EMC (e.g. ecosystems)

34. The Environmental Forecast Process Observations Data Assimilation Analysis Model Forecast Numerical Forecast System Post-processed Model Data Forecaster User (public, industry…)

35. EMP Model Strategy & ESMF • Concept of operations • Single system for global and regional models • Performance permitting • Migration to single model or • Multiple dynamics and physics options in single structure • Single verification, observations data base obeying WMO standards • Single analysis code • System perturbations from • Model diversity • Stochastic physics (preferred) • System supports both operational and research components • Dynamics • Physics • Overall positive experience at Met Office • ECMWF maintains single model & data assimilation system for global wx & short-term climate forecasting

36. Forecast System Development Strategy • As far as possible, based on quantitative assessment: • A single physics • Applied across multiple scales • E. g. global weather and climate • Possible extension to mesoscale • Successful for hurricane (GFS physics, NAM microphysics)  GFDL) • Excellent characteristics from all applications tested on other scales and implemented when ready • Single model (and DA) structure makes this feasible • Ensemble application • Requires approximately equal skill and system diversity • Secondary models and diverse components create manageable system if ESMF compatible and institutional support • Disciplined competition fosters the best system

37. Overview of Future Jigsaw • Enabled by ESMF • Hourly global data assimilation analyses • Hourly global rapid refresh • Supports NGATS • Concurrent global and mesoscale and mesoscale ensemble • Reforecast window • Target 2015+

38. Bold=Earlier Red=Later Timing Summary (2015)

39. The ultimate target is a completed NOAA Framework of ESMF Components within which NOAA scientists can work efficiently. One solution is outlined in the next few slides. A Project to Create the NOAA/NCEP Framework of ESMF Components Mark Iredell

40. Proposed NOAA ESMF-based System Components (1) • Change Resolution • Imports ATM state on one grid • Exports ATM state on another grid, possibly changing variables and units • Surface Cycling • Imports ATM boundary • Exports ATM boundary updated • Atmosphere • Imports ATM state • Exports ATM state later in time • Post • Imports ATM state • Exports selected products

41. Proposed NOAA ESMF-based System Components (2) • Dynamics • Imports ATM state • Exports ATM state later in time • Physics • Imports ATM state • Exports ATM state adjusted

42. Proposed NOAA ESMF-based System Components (3) • Vertical Post • Imports ATM state • Exports fields on selected levels but model grid • Product generator • Imports fields on one grid • Exports fields on another grid as requested • Output • Imports fields • Writes GRIB2 files (or BUFR or netCDF)

43. Proposed NOAA ESMF-based System Components (3) • Independent validation for each component that anyone can theoretically run • Examples • Dynamical core • Held-Suarez, etc. • Physics • Single column, etc. • Atmosphere • NWP and climate verifications and diagnostics • GFS with GDAS • Full parallel validation

44. Obs. Processing Variational Analysis Ensemble Members Mesoscale Model Storm Model Ocean Model Ice Model Hydrology Model Chemistry Model Space Model Other ComponentsOther components that could be coupled to the global atmospheric model using ESMF:

45. Model FrameworkTime and People costs • Estimated cost to NOAA for building: • 4 years • 17 man-years • Maintenance cost unknown • Multiple major developers • Possible lack of coordination • Coordination may slow development and system enhancement

46. Transition Steps (Modeling) Identification for Selection 1 Code/Algorithm Assessment and/or Development 2 3 Interface with Operational Codes 4 Level I: Preliminary Testing (Lower Resolution) 5 Level II: Preliminary Testing (DA/Higher Resolution) EMC Pre-Implementation Testing (Packaging/Calibration) 6 NCO Pre-Implementation Testing 7 Implementation/Delivery 8

47. NCEP’s Role in the Model Transition Process EMC and NCO have critical roles in the transition from NOAA R&D to operations Other Agencies & International Effort Service Centers Field Offices EMC NOAA Research NCO EMC Observation System Life cycle Support Service Centers User OPS Test Beds JCSDA CTB DTC JHT Operations R&D Delivery Transition from Research to Operations Launch List – Model Implementation Process Concept of Operations Requirements Criteria Forecast benefits, Efficiency, IT Compatibility, Sustainability

48. Component Requirements • Code standards and documentation • ESMF interfaces fully described • Lightweight so as to not impede research • Some compile-time and run-time flexibility • Somewhat flexible processor and thread layout • Somewhat flexible memory indexing layout • Somewhat flexible import and export fields • Standard metadata • CF convention • GRIB2, etc. • Distributed grid support under ESMF to enable coupling • Standalone validation

49. Given this solution is acceptable, how can we manage to get there? One framework project plan is given in the next few slides.

50. Steps in creating model framework • Planning and coordination • GFS I/O components • GFS ATM gridded component • GFS physics component • GFS dynamics component • UMO dynamics component • GFS post component • Change resolution component • Surface cycling component • Coupler components • Land component • Ocean component • Ice component • Aerosol component • Ionosphere component • Mesoscale components • GSI component • Configuration management