Evaluation of selected winter 04 05 performance results
This presentation is the property of its rightful owner.
Sponsored Links
1 / 38

Evaluation of Selected Winter ’04/’05 Performance Results PowerPoint PPT Presentation


  • 91 Views
  • Uploaded on
  • Presentation posted in: General

Evaluation of Selected Winter ’04/’05 Performance Results. Seth Linden and Jamie Wolff NCAR/RAL. Weather Forecast Verification. Consensus (RWFS) forecast is compared to individual model components Air-temperature, dewpoint, wind-speed and cloud-cover forecasts

Download Presentation

Evaluation of Selected Winter ’04/’05 Performance Results

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Evaluation of Selected Winter ’04/’05 Performance Results

Seth Linden and Jamie Wolff

NCAR/RAL


Weather Forecast Verification

  • Consensus (RWFS) forecast is compared to individual model components

  • Air-temperature, dewpoint, wind-speed and cloud-cover forecasts

    • 18 UTC runs for the entire season (1 November 2004 to 15 April 2005)

  • Error (RMSE) calculated for:

    • Colorado Plains: 176 sites

    • Mountains: 119 sites

Blizzard of March 2003


Air temperature RMSE

Colorado Mountains

RWFS

Colorado Plains


Colorado Plains

Colorado Mountains

Forward Error Correction

Due to 3-hour MOS data

Dewpoint

RMSE


Colorado Plains

Colorado Mountains

Wind Speed

RMSE


Colorado Mountains

Colorado Plains

Cloud Cover

RMSE


Summary/Recommendations

  • The ensemble approach utilized by the RWFS does improve the predictions on average for all verifiable parameters

  • No single model performs better for all parameters

  • A blend of weather models will provide better results


Forecast Model Weights Used by the RWFS

  • System automatically weights forecasts based on skill

  • Distribution of weight values per lead time for air-temperature, dewpoint, and wind-speed

    • 18 UTC run on 3 May 2005

  • Weights looked at for two sites:

    • Denver International Airport

    • I-70 at Genesse

Which models have the most skill?


Denver Int. Airport

Air Temperature

Model Weights

ETA

I-70 at Genesee

MOS

GFS

RUC

MOS


Denver Int. Airport

Dewpoint

Model Weights

I-70 at Genesee


Denver Int. Airport

Wind Speed

Model Weights

I-70 at Genesee

MM5

WRF


Clear Conditions

Insolation Weights

  • For MDSS static weights were applied:

  • - 50/50 split between MM5 and WRF for the

  • 0-23 hour forecast

    • - All Eta for the 24-48 hour forecast

  • No one model consistently outperforms the others

  • MM5 and WRF forecast hourly instantaneous values,

  • ETA forecasts 3-hour instantaneous values and

  • GFS forecasts 3-hour averages


QPF Weights

  • Due to a lack of quality precipitation observations static weights were applied

  • Weights fixed based on expert opinion

  • MM5 and WRF were given 80% of the total weight


Summary/Recommendations

  • Weight distribution reflects that the corrected (dynamic MOS) NWS models (ETA, GFS, and RUC) had the most overall skill

  • WRF and MM5 were given the highest static weights for Insolation and QPF

  • No one model dominates for all parameters

  • The limitation of the NWS models is their 3-hr temporal resolution


Road Temp Observation Variance

  • Tr variance across E-470 corridor

    • Shading by permanent structures or passing clouds

    • Make/model/installation/age of temperature sensors


E-470 Road/Bridge Sites

Platte Valley

(road and bridge)

Colorado Blvd

6th Ave Pkwy

Smokey Hill Rd

(road and bridge)

Plaza A


SCT

BKN

OVC

27 Nov 2004

28 Nov 2004

LOCAL TIME (19 = noon, 07 = midnight)


OVC

BKN

SCT

CLR

29 Nov 2004

30 Nov 2004

LOCAL TIME (19 = noon, 07 = midnight)


Summary/Recommendations

  • Large variations in observed road and bridge temperatures

    • Over relatively small area (10s of miles)

  • Makes prediction and verification of pavement temperatures very challenging

    • Difficult to establish ground truth


Road/Bridge Forecast Verification

  • Road and bridge temperature forecasts

    • Using recommended treatments from MDSS

  • Error (MAE) and bias calculated for:

    • For each lead time (0-48hrs)

      18 UTC runs

    • E-470: 6 roads/2 bridge

      (1 Nov 2004 – 15 Apr 2005)

    • Mountains: 5 roads

      (1 Feb 2004 – 15 Apr 2005)

East bound lane of I-70

at the summit of Vail Pass


Peak insolation

Morning hours

Consistent low bias

Perfect forecast

E-470 road sites

Lead Time (0 = 18 UTC = noon, 18 = 12 UTC = 6am)


evening

Shadowing?

morning

E-470 bridge sites

Lead Time (0 = 18 UTC ~ noon, 18 = 12 UTC ~ 6am)


evening

morning

CDOT mountain road sites

Lead Time (0 = 18 UTC = noon, 18 = 12 UTC = 6am)


Summary/Recommendations

  • Larger Tr differences during times of high solar insolation likely due to several factors:

    • Errors in measuring pavement skin temp

    • Mountain shading during low sun angle

    • Limitations in insolation prediction in models

    • Limitations in pavement heat balance model

      • Simplified assumptions about pavement characteristics

  • Tb analysis compromised by:

    • Sensors shadowed by bridge rail

    • Bias results suggest tuning may be beneficial

  • Overall Issue:

    • Actual/Recommended treatments not the same


Case Study Analysis

  • 183 day demonstration

    • 16 winter weather days

      • 10 light snow

      • 5 moderate snow

      • 1 heavy snow


November 27-29, 2004

  • First significant snow storm of the season

    • 5-8” in the Denver area

  • Large variations in parameter predictions

    • Forecast vs. observations

      • Denver International Airport

      • Ta, Td, Wspd, Cloud Cover and Precipitation

  • 12 UTC 28th examined

    • Captured the start time of event


8C/14F diff

2C/4F diff

Air Temperature

Snow

28 Nov 2005

LOCAL TIME (19 = noon, 06 = midnight)


6C/11F diff

Dewpoint Temperature

Snow

28 Nov 2005

LOCAL TIME (19 = noon, 06 = midnight)


Wind Speed

Snow

28 Nov 2005

LOCAL TIME (19 = noon, 06 = midnight)


FEC

Cloud Cover

Snow

28 Nov 2005

LOCAL TIME (19 = noon, 06 = midnight)


Quantitative Precipitation Forecast

Snow

28 Nov 2005

LOCAL TIME (19 = noon, 06 = midnight)


March 13, 2005

  • Moderate Snow Event

    • 4-6” along the E-470 corridor

  • Warm air temps before start of snow

    • Dropped from 11C (52F) to -2C (29F) in 5 hours

  • Large variations in parameter predictions

    • Forecast vs. observations

      • Denver International Airport

      • Ta, Wspd, Cloud Cover and Precipitation

  • 00 UTC 13 March 2005 run examined

    • Captured both start and end times


Air Temperature

Snow

13 March 2005

LOCAL TIME (18 = noon, 07 = midnight)


Wind Speed

Snow

13 March 2005

LOCAL TIME (18 = noon, 07 = midnight)


SCT - OVC

Cloud Cover

Snow

13 March 2005

LOCAL TIME (18 = noon, 07 = midnight)


actual

actual

forecast

forecast

Start time

End time

Quantitative Precipitation Forecast

13 March 2005

LOCAL TIME (18 = noon, 07 = midnight)


Summary/Recommendations

  • Large discrepancies between weather models in predicting state weather parameters

    • All too dry for Td and cloud cover

    • Low wind speed bias during windy conditions

    • Overall, no ONE model outperforms => Ensemble approach key

  • Supports probabilistic forecast presentation

    • Atmosphere is unpredictable

    • Best approach to present uncertainty to end users?


Thank You!Questions?


  • Login