Evaluation of selected winter 04 05 performance results
This presentation is the property of its rightful owner.
Sponsored Links
1 / 38

Evaluation of Selected Winter ’04/’05 Performance Results PowerPoint PPT Presentation


  • 85 Views
  • Uploaded on
  • Presentation posted in: General

Evaluation of Selected Winter ’04/’05 Performance Results. Seth Linden and Jamie Wolff NCAR/RAL. Weather Forecast Verification. Consensus (RWFS) forecast is compared to individual model components Air-temperature, dewpoint, wind-speed and cloud-cover forecasts

Download Presentation

Evaluation of Selected Winter ’04/’05 Performance Results

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Evaluation of selected winter 04 05 performance results

Evaluation of Selected Winter ’04/’05 Performance Results

Seth Linden and Jamie Wolff

NCAR/RAL


Weather forecast verification

Weather Forecast Verification

  • Consensus (RWFS) forecast is compared to individual model components

  • Air-temperature, dewpoint, wind-speed and cloud-cover forecasts

    • 18 UTC runs for the entire season (1 November 2004 to 15 April 2005)

  • Error (RMSE) calculated for:

    • Colorado Plains: 176 sites

    • Mountains: 119 sites

Blizzard of March 2003


Evaluation of selected winter 04 05 performance results

Air temperature RMSE

Colorado Mountains

RWFS

Colorado Plains


Evaluation of selected winter 04 05 performance results

Colorado Plains

Colorado Mountains

Forward Error Correction

Due to 3-hour MOS data

Dewpoint

RMSE


Evaluation of selected winter 04 05 performance results

Colorado Plains

Colorado Mountains

Wind Speed

RMSE


Evaluation of selected winter 04 05 performance results

Colorado Mountains

Colorado Plains

Cloud Cover

RMSE


Evaluation of selected winter 04 05 performance results

Summary/Recommendations

  • The ensemble approach utilized by the RWFS does improve the predictions on average for all verifiable parameters

  • No single model performs better for all parameters

  • A blend of weather models will provide better results


Forecast model weights used by the rwfs

Forecast Model Weights Used by the RWFS

  • System automatically weights forecasts based on skill

  • Distribution of weight values per lead time for air-temperature, dewpoint, and wind-speed

    • 18 UTC run on 3 May 2005

  • Weights looked at for two sites:

    • Denver International Airport

    • I-70 at Genesse

Which models have the most skill?


Evaluation of selected winter 04 05 performance results

Denver Int. Airport

Air Temperature

Model Weights

ETA

I-70 at Genesee

MOS

GFS

RUC

MOS


Evaluation of selected winter 04 05 performance results

Denver Int. Airport

Dewpoint

Model Weights

I-70 at Genesee


Evaluation of selected winter 04 05 performance results

Denver Int. Airport

Wind Speed

Model Weights

I-70 at Genesee

MM5

WRF


Insolation weights

Clear Conditions

Insolation Weights

  • For MDSS static weights were applied:

  • - 50/50 split between MM5 and WRF for the

  • 0-23 hour forecast

    • - All Eta for the 24-48 hour forecast

  • No one model consistently outperforms the others

  • MM5 and WRF forecast hourly instantaneous values,

  • ETA forecasts 3-hour instantaneous values and

  • GFS forecasts 3-hour averages


Qpf weights

QPF Weights

  • Due to a lack of quality precipitation observations static weights were applied

  • Weights fixed based on expert opinion

  • MM5 and WRF were given 80% of the total weight


Evaluation of selected winter 04 05 performance results

Summary/Recommendations

  • Weight distribution reflects that the corrected (dynamic MOS) NWS models (ETA, GFS, and RUC) had the most overall skill

  • WRF and MM5 were given the highest static weights for Insolation and QPF

  • No one model dominates for all parameters

  • The limitation of the NWS models is their 3-hr temporal resolution


Road temp observation variance

Road Temp Observation Variance

  • Tr variance across E-470 corridor

    • Shading by permanent structures or passing clouds

    • Make/model/installation/age of temperature sensors


E 470 road bridge sites

E-470 Road/Bridge Sites

Platte Valley

(road and bridge)

Colorado Blvd

6th Ave Pkwy

Smokey Hill Rd

(road and bridge)

Plaza A


Evaluation of selected winter 04 05 performance results

SCT

BKN

OVC

27 Nov 2004

28 Nov 2004

LOCAL TIME (19 = noon, 07 = midnight)


Evaluation of selected winter 04 05 performance results

OVC

BKN

SCT

CLR

29 Nov 2004

30 Nov 2004

LOCAL TIME (19 = noon, 07 = midnight)


Summary recommendations

Summary/Recommendations

  • Large variations in observed road and bridge temperatures

    • Over relatively small area (10s of miles)

  • Makes prediction and verification of pavement temperatures very challenging

    • Difficult to establish ground truth


Road bridge forecast verification

Road/Bridge Forecast Verification

  • Road and bridge temperature forecasts

    • Using recommended treatments from MDSS

  • Error (MAE) and bias calculated for:

    • For each lead time (0-48hrs)

      18 UTC runs

    • E-470: 6 roads/2 bridge

      (1 Nov 2004 – 15 Apr 2005)

    • Mountains: 5 roads

      (1 Feb 2004 – 15 Apr 2005)

East bound lane of I-70

at the summit of Vail Pass


Evaluation of selected winter 04 05 performance results

Peak insolation

Morning hours

Consistent low bias

Perfect forecast

E-470 road sites

Lead Time (0 = 18 UTC = noon, 18 = 12 UTC = 6am)


Evaluation of selected winter 04 05 performance results

evening

Shadowing?

morning

E-470 bridge sites

Lead Time (0 = 18 UTC ~ noon, 18 = 12 UTC ~ 6am)


Evaluation of selected winter 04 05 performance results

evening

morning

CDOT mountain road sites

Lead Time (0 = 18 UTC = noon, 18 = 12 UTC = 6am)


Summary recommendations1

Summary/Recommendations

  • Larger Tr differences during times of high solar insolation likely due to several factors:

    • Errors in measuring pavement skin temp

    • Mountain shading during low sun angle

    • Limitations in insolation prediction in models

    • Limitations in pavement heat balance model

      • Simplified assumptions about pavement characteristics

  • Tb analysis compromised by:

    • Sensors shadowed by bridge rail

    • Bias results suggest tuning may be beneficial

  • Overall Issue:

    • Actual/Recommended treatments not the same


Case study analysis

Case Study Analysis

  • 183 day demonstration

    • 16 winter weather days

      • 10 light snow

      • 5 moderate snow

      • 1 heavy snow


November 27 29 2004

November 27-29, 2004

  • First significant snow storm of the season

    • 5-8” in the Denver area

  • Large variations in parameter predictions

    • Forecast vs. observations

      • Denver International Airport

      • Ta, Td, Wspd, Cloud Cover and Precipitation

  • 12 UTC 28th examined

    • Captured the start time of event


Evaluation of selected winter 04 05 performance results

8C/14F diff

2C/4F diff

Air Temperature

Snow

28 Nov 2005

LOCAL TIME (19 = noon, 06 = midnight)


Evaluation of selected winter 04 05 performance results

6C/11F diff

Dewpoint Temperature

Snow

28 Nov 2005

LOCAL TIME (19 = noon, 06 = midnight)


Evaluation of selected winter 04 05 performance results

Wind Speed

Snow

28 Nov 2005

LOCAL TIME (19 = noon, 06 = midnight)


Evaluation of selected winter 04 05 performance results

FEC

Cloud Cover

Snow

28 Nov 2005

LOCAL TIME (19 = noon, 06 = midnight)


Evaluation of selected winter 04 05 performance results

Quantitative Precipitation Forecast

Snow

28 Nov 2005

LOCAL TIME (19 = noon, 06 = midnight)


March 13 2005

March 13, 2005

  • Moderate Snow Event

    • 4-6” along the E-470 corridor

  • Warm air temps before start of snow

    • Dropped from 11C (52F) to -2C (29F) in 5 hours

  • Large variations in parameter predictions

    • Forecast vs. observations

      • Denver International Airport

      • Ta, Wspd, Cloud Cover and Precipitation

  • 00 UTC 13 March 2005 run examined

    • Captured both start and end times


Evaluation of selected winter 04 05 performance results

Air Temperature

Snow

13 March 2005

LOCAL TIME (18 = noon, 07 = midnight)


Evaluation of selected winter 04 05 performance results

Wind Speed

Snow

13 March 2005

LOCAL TIME (18 = noon, 07 = midnight)


Evaluation of selected winter 04 05 performance results

SCT - OVC

Cloud Cover

Snow

13 March 2005

LOCAL TIME (18 = noon, 07 = midnight)


Evaluation of selected winter 04 05 performance results

actual

actual

forecast

forecast

Start time

End time

Quantitative Precipitation Forecast

13 March 2005

LOCAL TIME (18 = noon, 07 = midnight)


Summary recommendations2

Summary/Recommendations

  • Large discrepancies between weather models in predicting state weather parameters

    • All too dry for Td and cloud cover

    • Low wind speed bias during windy conditions

    • Overall, no ONE model outperforms => Ensemble approach key

  • Supports probabilistic forecast presentation

    • Atmosphere is unpredictable

    • Best approach to present uncertainty to end users?


Thank you questions

Thank You!Questions?


  • Login