Evaluation of Convective Storm Initiation Nowcasts and Forecaster Involvement 60 min Nowcasts Cold Front Regime 60 min Verification Rita Roberts, Amanda Anderson, Barbara Brown National Center for Atmospheric Research WMO WWRP Workshop on Use of NWP for Nowcasting Boulder, Colorado 26 October 2011
Challenges in Nowcasting Convective Initiation • Difficult to accurately predict the timing and location of convective storm initiation. • Numerical models are not able to routinely and accurately provide site-specific nowcasts of convective storms on the short 1-2 hr time frame. • In the interim, forecasters and heuristic systems can provide the rapidly updated, high-resolution, spatio-temporal nowcast products needed by aviation and other end-users.
Forecaster-Over-the-Loop AutoNowcaster DemonstrationNWS and NCAR NWS AWIPS Display 1 hr nowcasts 6 min updates 2 km grid 14 radars Satellite Soundings Surface stations NWP RUC output VDRAS fields Forecaster input Dallas/Ft. Worth WFO Thunderstorm Initiation Nowcast Field See Nelson et al paper, this conference
FOTL AutoNowCaster (ANC) system was demonstrated at the National Weather Service Dallas/Ft. Worth WFO from 2005-2010. • Improve the consistency, reliability, and accuracy of one hour convective storm nowcasts for inclusion in automated aviation weather digital products (NextGen 4-D data cube). • Assess NWS forecaster involvement in providing value-added enhancements to gridded, automated ANC products for NextGen. Goals:
The Federal Aviation Administration NextGen 4-D Weather Cube: Provides a common weather picture for the National Air Space by 2025 Observations Forecasting Numerical Modeling Systems Satellites Network Enabled Operations Statistical Forecasting Systems NWS Forecaster Radars Data Integration 4D Wx Cube Aircraft 4D Wx SAS Automated Forecast Systems Surface Forecast Integration Soundings Grids Decision Support Systems Custom Graphic Generators Custom Alphanumeric Generators Integration into User Decisions
What is needed to meet the NextGen requirements - and be successful?? • High resolution observations Boundaries, winds, stability, gravity waves, capping inversions, elevated dry layers, moisture variability and gradients • NWP - high resolution data assimilation for boundary layer winds and precipitation forecasts - frequent update cycle - at least 1 hr analysis and forecast fields - ability to represent multiple scale interactions - accurate output fields (CAPE, CIN, RH, vertical shear, convergence, vorticity, temperature gradients) • Forecaster - convergence boundary locations - climatology data base - conceptual models and documented weather regimes - QC of final products • Basic understanding (most important) Of elevated convection, secondary convection, outflow characteristics
Operational Role of Forecaster with the ANC system • Entering convergence boundaries periodically during shift • Nudging storm initiation nowcast fields based on forecaster understanding of NWP performance, the observations and their experience. • Specification and selection of weather regime
Storm initiation likelihood fields for different synoptic regimes Dry Line Regime Air Mass Regime • 7 different regimes (fuzzy-logic rule sets ) available • cold front, warm front, dryline, air mass, MCS, no storms, and mixed (default) regimes • Developed in collaboration with NWS forecasters • User Selectable
RUC NWP Model Forecasts “Too Hot” or “Too Cold” Forecaster has sliding scale toolbar to adjust (nudge) thunderstorm initiation likelihood values over the whole domain 0 + -
Challenges in Evaluating the Performance of Convection Initiation Nowcasts • Difficult to validate forecasts of these “rarer” convective initiation events. • Evaluation is hampered by the impacts of large-scale storm characteristics as well as small-scale errors in timing and location. • Difficult to statistically highlight the impact of human forecaster on an automated nowcast system.
Methodology for Evaluation • Focus the evaluation on small regions rather than the broader ANC domain. • Stratifying cases into similar forcing categories (i.e., weather regimes). • Examination of time-series performance for specific cases. • Use of contingency table performance diagrams to simultaneously investigate changes in multiple forecast performance attributes.
FWD CWA and ANC Domain Multiple Radar Reflectivity Mosaic Forecaster-entered boundary 4 Year Period Evaluated (2006-2009) 44 days analyzed in great detail ANC Convective Storm Initiation 60 min Nowcast Field ANC Final 60 min Nowcast Field: Initiation, Growth & Decay
Statistics Over Whole ANC Domain Convective storm initiation period
Calculating Statistics Over Sub-Domains Evaluation of nowcasts on the scale of the phenomena being nowcast.
Forecaster Interaction with ANC System 19:21 UTC 19:21 UTC 4 July 2006 Cold Front Regime 20:21 UTC 21:21 UTC Black contours: Location of > 35 dBZ storms at validation time
Time-Series Plots – 4 July 2006 for selected sub-domain boxes
Summary Histograms of Change in Verification Scores With and Without Human Input for all sub-domain boxes 4 July 2006
Performance Diagrams* 4 July 2006 *After Paul Roebber, Clive Wilson Forecaster-entered convergence boundaries have largest impact on convection initiation nowcasts. Examining only those domains that contain boundaries allows us to analyze change in nowcast accuracy due to forecaster involvement.
24 April 2007 12 May 2007 1 August2007 8 October 2007
8 October 2007 Mixed Regime
30 July 2008 MCS Regime 25 March 2009 Warm Front Regime 10 June 2009 Cold Front Regime
Summary • PODs of new storms increased by 20-60% with little or no change observed in false alarm ratios. • Bias values increased from 0.8-1.0 to 1.0-3.0 with human involvement. • Range of CSI values increased from 0.15-0.25 (ANC) to 0.2-0.4 (FOTL-ANC). • Values of CSI as large as 0.6 were observed. • A true improvement of the nowcast performance was achieved through forecaster involvement. • Forecasters played an important role in improving the quality of automated, high-resolution, gridded, convective initiation nowcasts for end-users.