1 / 25

Most popular “traditional statistics”: ETS, Bias Problem: what does the ETS tell us ?

Bias Adjusted Precipitation Scores Fedor Mesinger NOAA/Environmental Modeling Center and Earth System Science Interdisciplinary Center (ESSIC), Univ. Maryland, College Park, MD VX-Intercompare Meeting Boulder, 20 February 2007. Most popular “traditional statistics”: ETS, Bias

ornice
Download Presentation

Most popular “traditional statistics”: ETS, Bias Problem: what does the ETS tell us ?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Bias Adjusted Precipitation ScoresFedor MesingerNOAA/Environmental Modeling CenterandEarth System Science Interdisciplinary Center(ESSIC), Univ. Maryland, College Park, MDVX-Intercompare MeetingBoulder, 20 February 2007

  2. Most popular “traditional statistics”: ETS, Bias Problem: what does the ETS tell us ?

  3. “The higher the value, the better the model skill is for the particular threshold” (a recent MWR paper)

  4. Example: Three models, ETS, Bias, 12 months, “Western Nest” Is thegreenmodel loosing to red because of a bias penalty?

  5. What can one do ?

  6. J12.6 17th Prob. Stat. Atmos. Sci.; 20th WAF/16th NWP (Seattle AMS, Jan. ‘04) BIAS NORMALIZED PRECIPITATION SCORES Fedor Mesinger1 and Keith Brill2 1NCEP/EMC and UCAR, Camp Springs, MD 2NCEP/HPC, Camp Springs, MD

  7. Two methods of the adjustment for bias(“Normalized” not the best idea) • dHdF method: Assume incremental • change in hits per incremental change in • bias is proportional to the “unhit” area, O-H • Objective: obtain ETS adjusted to unit bias, • to show the model’s accuracy in placing precipitation • (The idea of the adjustment to unit bias to arrive at placement accuracy: • Shuman 1980, NOAA/NWS Office Note) 2. Odds Ratio method: different objective

  8. Forecast, Hits, and Observed (F, H, O) area, or number of model grid boxes:

  9. dHdF method, assumption: can be solved; a function H(F) obtained that satisfies the three requirements:

  10. Number of hits H -> 0 for F -> 0; • The function H(F) satisfies the known value of H for the model’s F, the pair denoted by Fb, Hb, and, • H(F) -> O as F increases

  11. Bias adjusted eq. threats West Eta GFS NMM

  12. A downside: if Hb is close to Fb, or to O, it can happen that dH/dF > 1 for F -> 0 Physically unrealistic ! Reasonableness requirement:

  13. “dHdM” method: Assume as F is increased by dF, ratio of the infinitesimal increase in H, dH, and that in false alarms dM=dF-dH, is proportional to the yet unhit area:

  14. One obtains ( Lambertw, or ProductLog in Mathematica, is the inverse function of )

  15. H(F) now satisfies the additional requirement: dH/dF never > 1

  16. dHdF method H=O H=F H(F) Fb,Hb

  17. dHdM method H=O H=F H(F) Fb,Hb

  18. Results for the two “focus cases”, dHdM method (Acknowledgements: John Halley Gotway, data;Dušan Jović, code and plots)

  19. 5/13 Case dHdM wrf2caps wrf4ncar wrf4ncep

  20. 6/01 Case dHdM wrf2caps wrf4ncar wrf4ncep

  21. Impact, in relative terms, for the two cases is small, because the biases of the three models are so similar !

  22. One more case, for good measure:

  23. 5/25 Case dHdM wrf2caps wrf4ncar wrf4ncep

  24. Comment: Scores would have generally been higher had the verification been done on grid squares greater than ~4 km This would have amounted to a poor-person’s version of “fuzzy” methods !

More Related