slide1 l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
The Impact of Probabilistic Information on Deterministic & Threshold Forecasts Susan Joslyn, Earl Hunt & Karla PowerPoint Presentation
Download Presentation
The Impact of Probabilistic Information on Deterministic & Threshold Forecasts Susan Joslyn, Earl Hunt & Karla

Loading in 2 Seconds...

play fullscreen
1 / 1

The Impact of Probabilistic Information on Deterministic & Threshold Forecasts Susan Joslyn, Earl Hunt & Karla - PowerPoint PPT Presentation


  • 136 Views
  • Uploaded on

The Impact of Probabilistic Information on Deterministic & Threshold Forecasts Susan Joslyn, Earl Hunt & Karla Schweitzer University of Washington. EXPERIMENT

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

The Impact of Probabilistic Information on Deterministic & Threshold Forecasts Susan Joslyn, Earl Hunt & Karla


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

The Impact of Probabilistic Information on Deterministic

& Threshold Forecasts

Susan Joslyn, Earl Hunt & Karla Schweitzer

University of Washington

    • EXPERIMENT
  • OUR QUESTION: Can weather forecasters incorporate uncertainty information expressed in probabilities into a non-probabilistic forecasting decision in a way that improves the forecast? Without bias?
  • Background
  • How do people understand and use probabilistic information?
    • People, in general, do not treat probability linearly
    • (Gonzalez & Wu, 1999)
    • Even experts have trouble incorporating prior probabilities
    • (Eddy 1982 )
    • However weather forecasters are good at estimating probabilities of e.g precipitation
    • (Baars & Mass, 2004)
  • Probabilistic information is increasingly important product of ensemble forecasts.
  • • Yet few operational weather forecasters make use of available probability products
  • (Joslyn, Jones & Tewson, 2005)

PROCEDURE

10 Participants: Atmospheric Science Students

Task: Forecast wind speed and direction for a 48 hour period. .Decide whether to issue high wind advisory (20 + knots). Forecasts were made for 4 days for four different locations on each day.

  • Information Provided
  • Historical data for all products used in previous cognitive task analysis (Joslyn et al, submitted)
      • Radar Imagery
      • Satellite Imagery
      • TAFs and current METARs
      • Model output (AVN, MM5 & NGM)
  • Design: Within subjects
    • • Each subject made 2 forecasts with the probability product,
    • 2 forecasts without probability product
    • • All subjects saw weather data from all of the same dates
    • • Probability product rotated through the dates, making sure that no subject saw the same date twice
  • Manipulation
  • On 1/2 of the forecasts, participants had the
  • MM5 ACME Ensemble Probability of winds
  • greater than 20 knots
  • Instructions:
    • Free choice of products
    • Except: forecasts with the probability product
    • Required to read & record the range provided by the probability product

RESULTS

Wind speed forecast: Deterministic forecast

R2 Predicting observed from forecasted wind speeds

  • We conducted regression analyses predicting observed wind speeds from forecasted wind speeds. See R2 for each condition in the table to the right.
  • For 3 dates people did better with the probability product
  • Note especially 3/21 which was the most difficult and for which the probability product made a significant difference.

Posting wind advisories: Thresholdforecast

  • d with and without the probability product were similar
  • However, WITH the probability product:
    • the criterion was higher (less willing to post advisory)
    • Fewer advisories overall.
  • Same forecasters/same weather
  • Difference must be due to probability product

CONCLUSIONS

Probability product had a slight impact on the deterministic wind speed forecast.

Wind Advisory Forecasts:

  • Forecasters, in general, had a liberal bias in the low to mid probability ranges: Biased to post wind advisories
    • Makes sense in this task: being cautious, keeping people off the water if there is any chance of danger
    • However, boaters may come to disregard the advisory if it often proves to be a false alarm (Roulston & Smith, 2004)
  • Probability product attenuated this tendency without causing them to lose sensitivity--they weren’t more likely to miss high wind situations
  • Forecasters, in general, had a conservative bias in the highest probability ranges
    • They did not post advisory as often as they should have
  • Probability product attenuated this tendency
    • more advisories in high range with probability product

References

Gonzalez, R., & Wu, G. (1999). On the form of the probability weighting function. Cognitive Psychology, 38, 129-166.

Eddy, D.M. (1982). Probabilistic reasoning in clinical medicine: Problems and opportunities. In D. Kahneman, P. Slovic, &

A. Tversky (Eds), Judgment under uncertainty: Heuristics and biases pp. 249-267)

Baars, J. A. & Mass, C. F. (2004) Performance of National Weather Service Forecasts Compared to Operational, Consensus

and Weighted Model Output Statistics.

Joslyn, S. Jones, D.W. & Tewson, P.(2005) Designing Tools for Uncertainty Estimation in Naval Weather Forecasting. 7th

International Conference on Naturalistic Decision Making

Roulston, M.S. & Smith, L. A. (2004) The boy who cried wolf revisited: The impact of false alarm intolerance on cost-loss

scenarios. Weather & Forecasting (19) 391-397.

Probability product

  • Good in very low probability situation
  • Overforecast in the midranges
  • In general, improved subjects performance

Subjects tended to

  • Over forecast in low-mid probability situations
  • Underforecast in high probability ranges
  • Attenuated with probability product

This research was supported by the DOD Multidisciplinary University Research Initiative (MURI) program administered by the Office of Naval Research under Grant N00014-01-10745