1 / 41

Advanced Study Program Research Review

Advanced Study Program Research Review. Tom Hopson April 6, 2007. Overview:. I. Operational Bangladesh Flood Forecasting 1. Background of project 2. Precipitation inputs: ECMWF ensemble forecasts and satellite rainfall corrections 3. Seasonal Forecasts 4. Brahmaputra Pilot programs

skule
Download Presentation

Advanced Study Program Research Review

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Advanced Study Program Research Review Tom Hopson April 6, 2007

  2. Overview: I. Operational Bangladesh Flood Forecasting 1. Background of project 2. Precipitation inputs: ECMWF ensemble forecasts and satellite rainfall corrections 3. Seasonal Forecasts 4. Brahmaputra Pilot programs II. Verifying the Relationship between Ensemble Forecast Spread and Skill

  3. Operational Flood Forecasting for Bangladesh: Tom Hopson Peter Webster GT A. R. Subbiah and R. Selvaraju, ADPC Climate Forecast Applications for Bangladesh (CFAB): NCAR/USAID-OFDA/GT/ADPC/ECMWF Bangladesh Stakeholders: Bangladesh Meteorological Department, Flood Forecasting and Warning Center, Bangladesh Water Development Board, Department of Agriculture Extension, Disaster Management Bureau, Institute of Water Modeling, Center for Environmental and Geographic Information Services, CARE-Bangladesh

  4. Bangladesh background • About 1/3 of land area floods the monsoon rainy season • Size: slightly smaller than Iowa • Border countries: Burma (193 km), India (4,053 km) • Population: 140 million • 36% of population below poverty line • Within the top 5 of: poorest and most densely populated in the world Sample of Flood History: 1988: 3/4 of country inundated, 1300 people killed, 30 million homeless, $1 billion in property loss 1998: 60% of country inundated for 3 months, 1000 killed, 40 million homeless 2004: flooding in Brahmaputra basin killed 500 people, displaced 30 million for 3 weeks, 40% of capitol city Dhaka (10 million people) under water

  5. (World Food Program)

  6. The Climate Forecast Applications Project CFAB • Bangladesh at confluence of Brahmaputra and Ganges Rivers • Limited warning of upstream river discharges • CFAB’s GOAL: Provide operational upper catchment flood-stage discharge and precipitation forecasts at differing time-scales • => Utilize good quality daily border discharge measurements

  7. Three-Tier Overlapping Forecast SystemDeveloped for Bangladesh SEASONAL OUTLOOK: “Broad brush” probabilistic forecast of rainfall and river discharge. Updated each month. Produced out to 6 months based on ECMWF’s seasonal 40 member ensemble forecasts. Currently most useful skill out 3 months. Information for strategic planning for agriculture and allied sectors and also for disaster preparedness. 20-25 DAY FORECAST: Forecast of average 5-day rainfall and river discharge 3-4 weeks in advance. Updated every 5 days. Strategic and tactical decisions in the agricultural, water resources and disaster management sectors, particularly for the management of floods and drought. 1-10 DAY FORECAST: Forecast of rainfall and precipitation in probabilistic form updated every day. Based on ECMWF’s 51 member ensemble weather forecasts. Skillful out 7-10 days. Provide probability of flood level exceedance at the entry point of the Ganges & Brahmaputra. Useful for emergency planning, and selective planting or harvesting to reduce potential crop losses at the beginning or end of the cropping cycle.

  8. Daily Operational Flood Forecasting Sequence

  9. ECMWF Ensemble Precipitation Forecast Adjustments -- mapping forecasts from “model-” to “observational-”space Brahmaputra Catchment-avg Forecasts • Hydrology model initial conditions driven by near-real-time GPCP / CMORPH / Raingage precipitation • Ideally, observations would be statistically “just another ensemble member” • Approach: calculate historical NWP-climatology PDF and observation-climatology PDF for each grid using a “kernel” method • For each forecast ensemble, determine its quantile in model-space and extract equivalent quantile in observation-space

  10. Quantile to Quantile Mapping Model Climatology “Observed” Climatology Pmax Pmax Precipitation Pfcst Padj 25th 50th 75th 100th 25th 50th 75th 100th Quantile Quantile

  11. ECMWF Ensemble Precipitation Forecast Adjustments -- mapping forecasts from “model-” to “observational-”space Brahmaputra Adjusted Forecasts • Benefits: --Gridded “realistic” forecast values --spatial- and temporal covariances preserved • Drawbacks: --limited sample set for model-space PDF (2 yrs) --rank histograms show “under-variance” Mean-Square-Error of the Ensemble-Mean shows skill out to 7-8 days

  12. Quantile Regression approach:maintaining skill no worse than “persistence” for non-Gaussian PDF’s (ECMWF Brahmaputra catchment Precipitation) 1 day 4 day • “Multi-model” statistical approach applied to RAL’s ATEC mesoscale ensemble forecasts (Josh Hacker) 7 day 10 day

  13. Precipitation Estimates Rain gauge estimates: NOAA CPC and WMO GTS 0.5 X 0.5 spatial resolution; 24h temporal resolution approximately 100 gauges reporting over combined catchment 24hr reporting delay Satellite-derived estimates: Global Precipitation Climatology Project (GPCP) 0.25X0.25 spatial resolution; 3hr temporal resolution 6hr reporting delay geostationary infrared “cold cloud top” estimates calibrated from SSM/I and TMI microwave instruments 3) Satellite-derived estimates: NOAA CPC “CMORPH” 0.25X0.25 spatial resolution; 3hr temporal resolution 18hr reporting delay precipitation rain rates derived from microwave instruments (SSM/I, TMI, AMSU-B), but “cloud tracking” done using infrared satellites => New Project (Dave Gochis, Gyuwon Lee): optimally blend products together along with uncertainty estimates Incorporate under a now-casting framework

  14. 2004 Discharge Forecast Results Brahmaputra Discharge Ensembles Confidence Intervals 2 day Critical Q black dash Observed Q black dot Ensemble Members in color 50% 95% 7 day 8 day 7 day 8 day 3 day 4 day 3 day 4 day 5 day 5 day 9 day 10 day 9 day 10 day

  15. 2004 Danger Level Probabilities Brahmaputra 7-10 day Forecasts Ganges 7-10 day Forecasts

  16. 2006 Ensemble Forecasts Brahmaputra 7-10 day Forecasts Ganges 7-10 day Forecasts

  17. 2006 Danger Level Probabilities Brahmaputra 7-10 day Forecasts Ganges 7-10 day Forecasts

  18. Forecasts Improvements • Quantile regression approach to improve hydrologic multi-model and final error correction algorithm • Automated “seamless” daily to seasonal discharge forecasts merging ECMWF weather and seasonal forecasts, updated daily

  19. Five Pilot Sites chosen in 2006 consultation workshops based on biophysical, social criteria: Rajpur Union -- 16 sq km -- 16,000 pop. Uria Union -- 23 sq km -- 14,000 pop. Kaijuri Union -- 45 sq km -- 53,000 pop. Gazirtek Union -- 32 sq km -- 23,000 pop. Bhekra Union -- 11 sq km -- 9,000 pop. (annual income: 30,000 Tk; US$400)

  20. Livelihoods What can be done with useful forecasts?

  21. Conclusions • 2003: Daily operational probabilistic discharge forecasts “experimentally” disseminated • 2004: -- Multi-model approach operational -- Forecasts fully-automated -- CFAB became an institutionalized entity of the Bangladesh federal government • 2006: -- USAID-OFDA/CARE 4-year funding commitment -- Forecasts incorporated into Bangladesh flood warning program • 2007: 5 pilot studies implemented for 1-10day forecasts along the Brahmaputra

  22. Overview: I. Bangladesh Flood Forecasting Project 1. Background of project 2. Precipitation inputs: ECMWF ensemble forecasts and satellite rainfall corrections 3. Seasonal Forecasts 4. Brahmaputra Pilot programs II. Verifying the Relationship between Ensemble Forecast Spread and Skill

  23. Motivation for generating ensemble forecasts: • Greater accuracy of ensemble mean forecast (half the error variance of single forecast) • Likelihood of extremes • Non-Gaussian forecast PDF’s • Ensemble spread as a representation of forecast uncertainty

  24. Probability “skill” or “error” “dispersion” or “spread” Rainfall [mm/day] Ensemble “Spread” or “Dispersion”Forecast “Skill” or “Error”

  25. ECMWF Brahmaputra catchment Precipitation Forecasts vs TRMM/CMORPH/CDC-GTS Rain gauge Estimates 1 day 4 day Points: -- ensemble dispersion increases with forecast lead-time -- dispersion variability within each lead-time -- Provide information about forecast certainty? How to Verify? -- rank histogram? No. (Hamill, 2001) -- ensemble spread- forecast error correlation? 7 day 10 day

  26. Overview -- Useful Ways to Measure Ensemble Forecast System’s Spread-Skill Relationship: • Spread-Skill Correlation misleading (Houtekamer, 1993; Whitaker and Loughe, 1998) • Propose 3 alternative scores 1) “normalized” spread-skill correlation 2) “binned” spread-skill correlation 3) “binned” rank histogram • Considerations: -- sufficient variance of the forecast spread? (outperforms ensemble mean forecast dressed with error climatology?) -- outperform heteroscedastic error model? -- account for observation uncertainty and under-sampling

  27. Naturally Paired Spread-skill measures: • Set I (L1 measures): • Error measures: • absolute error of the ensemble mean forecast • absolute error of a single ensemble member • Spread measures: • ensemble standard deviation • mean absolute difference of the ensembles about the ensemble mean • Set II (squared moments; L2 measures): • Error measures: • square error of the ensemble mean forecast • square error of a single ensemble member • Spread measures: • ensemble variance

  28. Spread-Skill Correlation … 4 day ECMWF r = 0.33 “Perfect” r = 0.68 ECMWF r = 0.41 “Perfect” r = 0.56 1 day • ECMWF spread-skill (black) correlation << 1 • Even “perfect model” (blue) correlation << 1 and varies with forecast lead-time 7 day 10 day ECMWF r = 0.39 “Perfect” r = 0.53 ECMWF r = 0.36 “Perfect” r = 0.49

  29. Limits on the spread-skill Correlation for a “Perfect” Model Governing ratio, g: (s = ensemble spread: variance, standard deviation, etc.) Limits: Set I Set II What’s the Point? -- correlation depends on how spread-skill defined -- depends on stability properties of the system being modeled -- even in “perfect” conditions, correlation much less than 1.0

  30. One option … Assign dispersion bins, then: 2) Average the error values in each bin, then correlate 3) Calculate individual rank histograms for each bin, convert to a scalar measure

  31. Option 2: “binned” Spread-skill Correlation 1 day 4 day • “perfect model” (blue) approaches perfect correlation • “no-skill” model (red) has expected under-dispersive “U-shape” • ECMWF forecasts (black) generally under-dispersive, improving with lead-time • Heteroscedastic model (green) slightly better(worse) than ECMWF forecasts for short(long) lead-times 7 day 10 day

  32. Option 2: PDF’s of “binned” spread-skill correlations -- accounting for sampling and verification uncertainty 4 day 1 day • “perfect model” (blue) PDF peaked near 1.0 for all lead-times • “no-skill” model (red) PDF has broad range of values • ECMWF forecast PDF (black) overlaps both “perfect” and “no-skill” PDF’s • Heteroscedastic model (green) slightly better(worse) than ECMWF forecasts for short(long) lead-times 7 day 10 day

  33. Conclusions • Spread-skill correlation can be misleading measure of utility of ensemble dispersion • Dependent on “stability” properties of environmental system • 3 alternatives: 1) “normalized” (skill-score) spread-skill correlation 2) “binned” spread-skill correlation 3) “binned” rank histogram • ratio of moments of “spread” distribution also indicates utility -- if ratio --> 1.0, fixed “climatological” error distribution may provide a far cheaper estimate of forecast error • Truer test of utility of forecast dispersion is a comparison with a heteroscedastic error model => a statistical error model may be superior (and cheaper) • Important to account for observation and sampling uncertainties when doing a verification

  34. Thank You!

More Related