Thomas Karl Director National Oceanic and Atmospheric Administration

1. NOAA Reanalysis User Needs AMS Town Hall Meeting Thomas Karl Director National Oceanic and Atmospheric Administration National Climatic Data Center AMS 88th Annual Meeting Town Hall Meeting January 21,2008

2. Outline • Purpose of Meeting • Overview of NOAA Reanalysis Efforts • Climate Forecast System Reanalysis and Reforecast (CFSRR): • Data Availability Plans • Data Access Optimization • User Input and Open Discussion

3. Tonight’s discussion is a focus on user needs Open Forum and User Input • Has the proposed file organization and groupings reflect current user needs and expectations? • Which data need to be placed on-line given e.g., 150TB of disk? • Are we on the right track? Do we need to re-group data sets to create needed products – such as initialization files, analysis, or by forecast projections? • Can some other mechanism (i.e. LDM) provide some CFSRR output to users in near real-time, shifting the load from the archive? • NCDC and NCEP are committed to provide the highest resolution if possible. Tonight’s discussion is a focus on user needs tempered by available resources.

4. New NOAA Reanalysis Projects 1) Historical SFC Reanalysis (Compo et al.,) • 1850 to present. ~60TB 2) Post WW-II Reanalysis (NCEP) • 1944 to present. ~235TB 3) Climate Forecast System Reanalysis and Reforecast (CFSRR) Project (Saha et al.) • 1978-2008 Reanalysis and Seasonal Reforecast: 915TB (Grib1) • Development of a Reanalysis Clearinghouse • Capability for an “on-going analysis of the climate system”

5. CFSRR Overview (Saha, EMC/NCEP) • Coupled Climate Forecast System (CFS) Reanalysis and Reforecast (CFSRR) (Atm - Ocn - Land – SeaIce) has two parts: • Reanalysis: 31-year (1979-2009) T384L64 (~32 km and 64 hybrid layers for the atmosphere, 0.50 and 40 levels for the ocean, 4 soil levels) • Reforecast: 28-year (1982-2009) T126L64 (~32 km and 64 hybrid layers for the atmosphere, 0.50 and 40 levels for the ocean, 4 soil levels) • 6 hourly Reforecast for 1 year • NCDC and NCEP are seeking community input for archive and access priorities to this massive dataset • Approximately 915 Terabytes in GRIB1. Note: this presentation assumes conversion is needed to Grib1 as many user applications cannot yet deal with GRIB2 directly (including NCDC).

6. Proposed CFS Reanalysis Data • CFS High Resolution Initial Conditions: 72 TB • CFS Low Resolution Initial Conditions: 35 TB • Full Data Ingest : 46TB (satellite radiances, RAOB, in-situ, etc.) • Pgbh Pressure Grib Files 0.5 x 0.5 Hourly: 91TB • Flxf T382 Gaussian Hourly: 25TB • Ocnh 0.5 x 0.5 Hourly: 18TB • Diabf 1.0 x 1.0 Hourly: 27TB • Ipvh 0.5 x 0.5 Hourly: 17TB • Monthly: 1TB TOTAL: 332 TB

7. Proposed CFS Reanalysis Hi-Resolution File Level Organization • Siganl : 3-D Hybrid Analysis • Sfcanl : Surface Analysis • Ocnanl : 3D Ocean Analysis • Pgbh : 3-D Pressure level data • Flxf : SFC fluxes, radiative fluxes, precip. • Ocnh : 3-D Ocean data • Ipvh : 3-D Isentropic level data • Diabf : Diabatic Heating, Moistening rates, etc. • Egyh : Energetics, u’ v’, TKE, etc. T382L64 + Ocean (1/4 x ½) 6 hourly = 72TB T126L64 + Ocean (1/2 x 1) 6 hourly = 35 TB 0.5 x 0.5 degree Hourly= 91TB T382 Gaussian Hrly= 25TB 0.5 x 0.5 degree Hourly= 18TB 0.5 x 0.5 degree Hourly: 17TB 1.0 x 1.0 6-Hourly: 27TB 0.5 x 0.5 Monthly: 1 TB

8. CFS 28-Year Reforecast Data1982-2009 • 6-hourly Pressure Grib: 412TB • 6-hourly Ocean : 102TB • 6-hourly Ipv: 78TB • Monthly and Time-series: 3TB 1.0 x 1.0 degree for first 6 months of forecast 2.5 x 2.5 for next 6 months of forecast TOTAL: 595 TB

9. National Academies Recommendation • National Research Council, Board of Atmospheric Sciences and Climate (BASC): “Completing the Forecast: Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts” Recommendation 3.4: The NOAA National Operational Model Archive and Distribution System (NOMADS) should be maintained and extended to include (a) long-term archives of global and regional ensemble forecasting systems and their native resolution, and (b) re-forecast datasets to facilitate post-processing”

10. Open Discussion Proposed Data and System Access Priorities

11. CFSRR System Access Prioritization 1 • Basic ftp and http scripting services to the highest resolution data though the NCDC tape archive services. • NOMADS on-line sub-setting services for most requested data (tonight’s discussion). • Advanced GIS and other Web Based Portal Services as resources and time permit • Issue: Data access may be throttled based on number of concurrent users, I/O restraints, and communications bandwidth at NCDC.

12. Reanalysis Data Access Prioritization 1 • Highest Resolution CFSRR data provided • Low Res users can subset or use Post WW-II • Access assisted by user feedback as to file level organization (this meeting!) • Historical “20th Century” SFC Reanalysis • 60TB (Grib and HDF) • FY10 Post WW-II U/A Reanalysis • Low resolution 6-hourly (235TB Grib-1) • When funded

13. FEEDBACK and QUESTIONS OPEN DISCUSSION Suranjana.Saha@noaa.gov CFSRR PI Glenn.Rutledge@noaa.gov nomads.ncdc.noaa.gov David.Easterling@noaa.gov

14. Backup Slides

15. Overview • To overcome a deficiency in model data access, some of the Nations top scientists are actively engaged in a grass-roots framework to share data and research findings over the Internet. • NCDC, NCEP and GFDL initiated the NOAA Operational Model Archive and Distribution System. • NOMADS is a distributed data services pilot for format independent access to climate and weather models and data.

16. The NOAA Operational Model Archive and Distribution System NOMADS Goals • Establish a unified climate and weather model archive providing format independent access to retrospective models; • promote model evaluation and community feedback; • foster research within the geo-science communities (ocean, weather, and climate) to study multiple earth systems using collections of distributed data; • develop institutional partnerships and access via distributed open standard technologies.

17. Multiple paths to format independent data access The NOMADS System Design

18. NCDC HDSS Access System (HAS = NCDC Archive) Other incoming data Data request from access service—eg, NOMADS Data delivery to access service—eg, NEXRAD NOMADS Model data NEXRAD Radar Major Access Services CLASS Satellite CDO, GIS Services In-situ NOAA Customers A very simplified diagram of data flow from NCDC‘s HDSS tape robotics system & incoming sources