evaluation of selected winter 04 05 performance results
Download
Skip this Video
Download Presentation
Evaluation of Selected Winter ’04/’05 Performance Results

Loading in 2 Seconds...

play fullscreen
1 / 38

Evaluation of Selected Winter ’04/’05 Performance Results - PowerPoint PPT Presentation


  • 378 Views
  • Uploaded on

Evaluation of Selected Winter ’04/’05 Performance Results. Seth Linden and Jamie Wolff NCAR/RAL. Weather Forecast Verification. Consensus (RWFS) forecast is compared to individual model components Air-temperature, dewpoint, wind-speed and cloud-cover forecasts

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Evaluation of Selected Winter ’04/’05 Performance Results' - tad-butler


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
evaluation of selected winter 04 05 performance results

Evaluation of Selected Winter ’04/’05 Performance Results

Seth Linden and Jamie Wolff

NCAR/RAL

weather forecast verification
Weather Forecast Verification
  • Consensus (RWFS) forecast is compared to individual model components
  • Air-temperature, dewpoint, wind-speed and cloud-cover forecasts
    • 18 UTC runs for the entire season (1 November 2004 to 15 April 2005)
  • Error (RMSE) calculated for:
    • Colorado Plains: 176 sites
    • Mountains: 119 sites

Blizzard of March 2003

slide3

Air temperature RMSE

Colorado Mountains

RWFS

Colorado Plains

slide4

Colorado Plains

Colorado Mountains

Forward Error Correction

Due to 3-hour MOS data

Dewpoint

RMSE

slide5

Colorado Plains

Colorado Mountains

Wind Speed

RMSE

slide6

Colorado Mountains

Colorado Plains

Cloud Cover

RMSE

slide7

Summary/Recommendations

  • The ensemble approach utilized by the RWFS does improve the predictions on average for all verifiable parameters
  • No single model performs better for all parameters
  • A blend of weather models will provide better results
forecast model weights used by the rwfs
Forecast Model Weights Used by the RWFS
  • System automatically weights forecasts based on skill
  • Distribution of weight values per lead time for air-temperature, dewpoint, and wind-speed
    • 18 UTC run on 3 May 2005
  • Weights looked at for two sites:
    • Denver International Airport
    • I-70 at Genesse

Which models have the most skill?

slide9

Denver Int. Airport

Air Temperature

Model Weights

ETA

I-70 at Genesee

MOS

GFS

RUC

MOS

slide10

Denver Int. Airport

Dewpoint

Model Weights

I-70 at Genesee

slide11

Denver Int. Airport

Wind Speed

Model Weights

I-70 at Genesee

MM5

WRF

insolation weights

Clear Conditions

Insolation Weights
  • For MDSS static weights were applied:
  • - 50/50 split between MM5 and WRF for the
  • 0-23 hour forecast
    • - All Eta for the 24-48 hour forecast
  • No one model consistently outperforms the others
  • MM5 and WRF forecast hourly instantaneous values,
  • ETA forecasts 3-hour instantaneous values and
  • GFS forecasts 3-hour averages
qpf weights
QPF Weights
  • Due to a lack of quality precipitation observations static weights were applied
  • Weights fixed based on expert opinion
  • MM5 and WRF were given 80% of the total weight
slide14

Summary/Recommendations

  • Weight distribution reflects that the corrected (dynamic MOS) NWS models (ETA, GFS, and RUC) had the most overall skill
  • WRF and MM5 were given the highest static weights for Insolation and QPF
  • No one model dominates for all parameters
  • The limitation of the NWS models is their 3-hr temporal resolution
road temp observation variance
Road Temp Observation Variance
  • Tr variance across E-470 corridor
    • Shading by permanent structures or passing clouds
    • Make/model/installation/age of temperature sensors
e 470 road bridge sites
E-470 Road/Bridge Sites

Platte Valley

(road and bridge)

Colorado Blvd

6th Ave Pkwy

Smokey Hill Rd

(road and bridge)

Plaza A

slide17

SCT

BKN

OVC

27 Nov 2004

28 Nov 2004

LOCAL TIME (19 = noon, 07 = midnight)

slide18

OVC

BKN

SCT

CLR

29 Nov 2004

30 Nov 2004

LOCAL TIME (19 = noon, 07 = midnight)

summary recommendations
Summary/Recommendations
  • Large variations in observed road and bridge temperatures
    • Over relatively small area (10s of miles)
  • Makes prediction and verification of pavement temperatures very challenging
    • Difficult to establish ground truth
road bridge forecast verification
Road/Bridge Forecast Verification
  • Road and bridge temperature forecasts
    • Using recommended treatments from MDSS
  • Error (MAE) and bias calculated for:
    • For each lead time (0-48hrs)

18 UTC runs

    • E-470: 6 roads/2 bridge

(1 Nov 2004 – 15 Apr 2005)

    • Mountains: 5 roads

(1 Feb 2004 – 15 Apr 2005)

East bound lane of I-70

at the summit of Vail Pass

slide21

Peak insolation

Morning hours

Consistent low bias

Perfect forecast

E-470 road sites

Lead Time (0 = 18 UTC = noon, 18 = 12 UTC = 6am)

slide22

evening

Shadowing?

morning

E-470 bridge sites

Lead Time (0 = 18 UTC ~ noon, 18 = 12 UTC ~ 6am)

slide23

evening

morning

CDOT mountain road sites

Lead Time (0 = 18 UTC = noon, 18 = 12 UTC = 6am)

summary recommendations1
Summary/Recommendations
  • Larger Tr differences during times of high solar insolation likely due to several factors:
    • Errors in measuring pavement skin temp
    • Mountain shading during low sun angle
    • Limitations in insolation prediction in models
    • Limitations in pavement heat balance model
      • Simplified assumptions about pavement characteristics
  • Tb analysis compromised by:
    • Sensors shadowed by bridge rail
    • Bias results suggest tuning may be beneficial
  • Overall Issue:
    • Actual/Recommended treatments not the same
case study analysis
Case Study Analysis
  • 183 day demonstration
    • 16 winter weather days
      • 10 light snow
      • 5 moderate snow
      • 1 heavy snow
november 27 29 2004
November 27-29, 2004
  • First significant snow storm of the season
    • 5-8” in the Denver area
  • Large variations in parameter predictions
    • Forecast vs. observations
      • Denver International Airport
      • Ta, Td, Wspd, Cloud Cover and Precipitation
  • 12 UTC 28th examined
    • Captured the start time of event
slide27

8C/14F diff

2C/4F diff

Air Temperature

Snow

28 Nov 2005

LOCAL TIME (19 = noon, 06 = midnight)

slide28

6C/11F diff

Dewpoint Temperature

Snow

28 Nov 2005

LOCAL TIME (19 = noon, 06 = midnight)

slide29

Wind Speed

Snow

28 Nov 2005

LOCAL TIME (19 = noon, 06 = midnight)

slide30

FEC

Cloud Cover

Snow

28 Nov 2005

LOCAL TIME (19 = noon, 06 = midnight)

slide31

Quantitative Precipitation Forecast

Snow

28 Nov 2005

LOCAL TIME (19 = noon, 06 = midnight)

march 13 2005
March 13, 2005
  • Moderate Snow Event
    • 4-6” along the E-470 corridor
  • Warm air temps before start of snow
    • Dropped from 11C (52F) to -2C (29F) in 5 hours
  • Large variations in parameter predictions
    • Forecast vs. observations
      • Denver International Airport
      • Ta, Wspd, Cloud Cover and Precipitation
  • 00 UTC 13 March 2005 run examined
    • Captured both start and end times
slide33

Air Temperature

Snow

13 March 2005

LOCAL TIME (18 = noon, 07 = midnight)

slide34

Wind Speed

Snow

13 March 2005

LOCAL TIME (18 = noon, 07 = midnight)

slide35

SCT - OVC

Cloud Cover

Snow

13 March 2005

LOCAL TIME (18 = noon, 07 = midnight)

slide36

actual

actual

forecast

forecast

Start time

End time

Quantitative Precipitation Forecast

13 March 2005

LOCAL TIME (18 = noon, 07 = midnight)

summary recommendations2
Summary/Recommendations
  • Large discrepancies between weather models in predicting state weather parameters
    • All too dry for Td and cloud cover
    • Low wind speed bias during windy conditions
    • Overall, no ONE model outperforms => Ensemble approach key
  • Supports probabilistic forecast presentation
    • Atmosphere is unpredictable
    • Best approach to present uncertainty to end users?
ad