Evaluation of selected winter 04 05 performance results
Download
1 / 38

Evaluation of Selected Winter ’04/’05 Performance Results - PowerPoint PPT Presentation


  • 378 Views
  • Uploaded on

Evaluation of Selected Winter ’04/’05 Performance Results. Seth Linden and Jamie Wolff NCAR/RAL. Weather Forecast Verification. Consensus (RWFS) forecast is compared to individual model components Air-temperature, dewpoint, wind-speed and cloud-cover forecasts

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Evaluation of Selected Winter ’04/’05 Performance Results' - tad-butler


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Evaluation of selected winter 04 05 performance results

Evaluation of Selected Winter ’04/’05 Performance Results

Seth Linden and Jamie Wolff

NCAR/RAL


Weather forecast verification
Weather Forecast Verification Results

  • Consensus (RWFS) forecast is compared to individual model components

  • Air-temperature, dewpoint, wind-speed and cloud-cover forecasts

    • 18 UTC runs for the entire season (1 November 2004 to 15 April 2005)

  • Error (RMSE) calculated for:

    • Colorado Plains: 176 sites

    • Mountains: 119 sites

Blizzard of March 2003


Air temperature RMSE Results

Colorado Mountains

RWFS

Colorado Plains


Colorado Plains Results

Colorado Mountains

Forward Error Correction

Due to 3-hour MOS data

Dewpoint

RMSE


Colorado Plains Results

Colorado Mountains

Wind Speed

RMSE


Colorado Mountains Results

Colorado Plains

Cloud Cover

RMSE


Summary/Recommendations Results

  • The ensemble approach utilized by the RWFS does improve the predictions on average for all verifiable parameters

  • No single model performs better for all parameters

  • A blend of weather models will provide better results


Forecast model weights used by the rwfs
Forecast Model Weights Used by the RWFS Results

  • System automatically weights forecasts based on skill

  • Distribution of weight values per lead time for air-temperature, dewpoint, and wind-speed

    • 18 UTC run on 3 May 2005

  • Weights looked at for two sites:

    • Denver International Airport

    • I-70 at Genesse

Which models have the most skill?


Denver Int. Airport Results

Air Temperature

Model Weights

ETA

I-70 at Genesee

MOS

GFS

RUC

MOS


Denver Int. Airport Results

Dewpoint

Model Weights

I-70 at Genesee


Denver Int. Airport Results

Wind Speed

Model Weights

I-70 at Genesee

MM5

WRF


Insolation weights

Clear Conditions Results

Insolation Weights

  • For MDSS static weights were applied:

  • - 50/50 split between MM5 and WRF for the

  • 0-23 hour forecast

    • - All Eta for the 24-48 hour forecast

  • No one model consistently outperforms the others

  • MM5 and WRF forecast hourly instantaneous values,

  • ETA forecasts 3-hour instantaneous values and

  • GFS forecasts 3-hour averages


Qpf weights
QPF Weights Results

  • Due to a lack of quality precipitation observations static weights were applied

  • Weights fixed based on expert opinion

  • MM5 and WRF were given 80% of the total weight


Summary/Recommendations Results

  • Weight distribution reflects that the corrected (dynamic MOS) NWS models (ETA, GFS, and RUC) had the most overall skill

  • WRF and MM5 were given the highest static weights for Insolation and QPF

  • No one model dominates for all parameters

  • The limitation of the NWS models is their 3-hr temporal resolution


Road temp observation variance
Road Temp Observation Variance Results

  • Tr variance across E-470 corridor

    • Shading by permanent structures or passing clouds

    • Make/model/installation/age of temperature sensors


E 470 road bridge sites
E-470 Road/Bridge Sites Results

Platte Valley

(road and bridge)

Colorado Blvd

6th Ave Pkwy

Smokey Hill Rd

(road and bridge)

Plaza A


SCT Results

BKN

OVC

27 Nov 2004

28 Nov 2004

LOCAL TIME (19 = noon, 07 = midnight)


OVC Results

BKN

SCT

CLR

29 Nov 2004

30 Nov 2004

LOCAL TIME (19 = noon, 07 = midnight)


Summary recommendations
Summary/Recommendations Results

  • Large variations in observed road and bridge temperatures

    • Over relatively small area (10s of miles)

  • Makes prediction and verification of pavement temperatures very challenging

    • Difficult to establish ground truth


Road bridge forecast verification
Road/Bridge Forecast Verification Results

  • Road and bridge temperature forecasts

    • Using recommended treatments from MDSS

  • Error (MAE) and bias calculated for:

    • For each lead time (0-48hrs)

      18 UTC runs

    • E-470: 6 roads/2 bridge

      (1 Nov 2004 – 15 Apr 2005)

    • Mountains: 5 roads

      (1 Feb 2004 – 15 Apr 2005)

East bound lane of I-70

at the summit of Vail Pass


Peak insolation Results

Morning hours

Consistent low bias

Perfect forecast

E-470 road sites

Lead Time (0 = 18 UTC = noon, 18 = 12 UTC = 6am)


evening Results

Shadowing?

morning

E-470 bridge sites

Lead Time (0 = 18 UTC ~ noon, 18 = 12 UTC ~ 6am)


evening Results

morning

CDOT mountain road sites

Lead Time (0 = 18 UTC = noon, 18 = 12 UTC = 6am)


Summary recommendations1
Summary/Recommendations Results

  • Larger Tr differences during times of high solar insolation likely due to several factors:

    • Errors in measuring pavement skin temp

    • Mountain shading during low sun angle

    • Limitations in insolation prediction in models

    • Limitations in pavement heat balance model

      • Simplified assumptions about pavement characteristics

  • Tb analysis compromised by:

    • Sensors shadowed by bridge rail

    • Bias results suggest tuning may be beneficial

  • Overall Issue:

    • Actual/Recommended treatments not the same


Case study analysis
Case Study Analysis Results

  • 183 day demonstration

    • 16 winter weather days

      • 10 light snow

      • 5 moderate snow

      • 1 heavy snow


November 27 29 2004
November 27-29, 2004 Results

  • First significant snow storm of the season

    • 5-8” in the Denver area

  • Large variations in parameter predictions

    • Forecast vs. observations

      • Denver International Airport

      • Ta, Td, Wspd, Cloud Cover and Precipitation

  • 12 UTC 28th examined

    • Captured the start time of event


8C/14F diff Results

2C/4F diff

Air Temperature

Snow

28 Nov 2005

LOCAL TIME (19 = noon, 06 = midnight)


6C/11F diff Results

Dewpoint Temperature

Snow

28 Nov 2005

LOCAL TIME (19 = noon, 06 = midnight)


Wind Speed Results

Snow

28 Nov 2005

LOCAL TIME (19 = noon, 06 = midnight)


FEC Results

Cloud Cover

Snow

28 Nov 2005

LOCAL TIME (19 = noon, 06 = midnight)


Quantitative Precipitation Forecast Results

Snow

28 Nov 2005

LOCAL TIME (19 = noon, 06 = midnight)


March 13 2005
March 13, 2005 Results

  • Moderate Snow Event

    • 4-6” along the E-470 corridor

  • Warm air temps before start of snow

    • Dropped from 11C (52F) to -2C (29F) in 5 hours

  • Large variations in parameter predictions

    • Forecast vs. observations

      • Denver International Airport

      • Ta, Wspd, Cloud Cover and Precipitation

  • 00 UTC 13 March 2005 run examined

    • Captured both start and end times


Air Temperature Results

Snow

13 March 2005

LOCAL TIME (18 = noon, 07 = midnight)


Wind Speed Results

Snow

13 March 2005

LOCAL TIME (18 = noon, 07 = midnight)


SCT - OVC Results

Cloud Cover

Snow

13 March 2005

LOCAL TIME (18 = noon, 07 = midnight)


actual Results

actual

forecast

forecast

Start time

End time

Quantitative Precipitation Forecast

13 March 2005

LOCAL TIME (18 = noon, 07 = midnight)


Summary recommendations2
Summary/Recommendations Results

  • Large discrepancies between weather models in predicting state weather parameters

    • All too dry for Td and cloud cover

    • Low wind speed bias during windy conditions

    • Overall, no ONE model outperforms => Ensemble approach key

  • Supports probabilistic forecast presentation

    • Atmosphere is unpredictable

    • Best approach to present uncertainty to end users?


Thank you questions
Thank You! ResultsQuestions?


ad