slide1
Download
Skip this Video
Download Presentation
Marine Verification at WFO Key West Where to go and how to get there… Andy Devanas, SOO and

Loading in 2 Seconds...

play fullscreen
1 / 24

Marine Verification at WFO Key West Where to go and how to get there… Andy Devanas, SOO and - PowerPoint PPT Presentation


  • 124 Views
  • Uploaded on

Marine Verification at WFO Key West Where to go and how to get there… Andy Devanas, SOO and Matt Parke, Verification Focal Point. There are three kinds of lies: lies, damned lies, and statistics. Benjamin Disraeli (1804 - 1881). Figure 1. USF Buoy Not Operational. X. X. X. X. X.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Marine Verification at WFO Key West Where to go and how to get there… Andy Devanas, SOO and' - bryar-orr


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Marine Verification at WFO Key West

  • Where to go and how to get there…
  • Andy Devanas, SOO and
  • Matt Parke, Verification Focal Point
slide2

There are three kinds of lies:

  • lies, damned lies, and statistics.
    • Benjamin Disraeli (1804 - 1881)
slide3

Figure 1

USF Buoy

Not Operational

X

X

X

X

X

X

Marine Area approx.

22000 square miles

Massachusetts –

10555 square miles

X– No Surface Data Available

NO WAVE DATA AT ALL

slide6

Key West Verification System Needs…

Need to integrate verification into the forecast process

Need to provide meaningful, frequent and timely verification data to forecasters ( e.g. Percent Correct vs. ESS)

Need to identify forecast and guidance biases

Enhanced discrimination in critical categories

slide7

How To Get There…

  • *Use MVerify for rapid feedback to forecasters
  • -Local AWIPS application
  • -Provides most flexibility in stratification of desired parameters
  • Minimum forecast period adjustable
  • Can segregate by forecaster
  • *Investigate various statistical measures/distributions which best track forecast performance
  • *Set reasonable goals of forecast performance.
  • *Monitor forecast performance.
    • -Accountability
    • *Use verification statistics in post-mortem evaluations as feedback to forecasters.
slide8

APPLICATION OF THE NATIONAL MARINE VERIFICATION PROGRAM (NMVP) TO LOCAL MARINE VERIFICATION

  • DRAWBACKS
  • *NMVP is not timely. Must wait until after end of month to access statistics.
  • *NVMP does not allow defining of periods.
    • Minimum period is one month.
    • *Can’t define inter-month periods.
  • *NMVP does not provide access to individual forecast-observation pairs.
    • -Cannot do post-mortem analysis.
    • -Do not know which matched pairs are producing statistics.
    • -No Quality control of forecasts and observations.
  • *Some measures (ESS) are heavily weighted towards rare climatological events.
    • -ESS is volatile.
      • -Critical missing matched pairs can have large impact on ESS.
  • BENEFITS
  • *Provides relatively easy and quick statistics spanning > 3 year dataset
  • *Can stratify data with regards to forecast period, buoy (cman), and season
slide9

Equitable Skill Score reward/penalty Characteristics (from NMVP)-

  • A relatively small reward for correctly forecasting common events
  • A large reward for correctly forecasting rare events
  • Less penalty is assigned to an incorrect forecast of a rare event than a similar size error of a common event. "Near hits" of rare event receive a modest reward.
  • Uses a graduated reward/penalty system, whereby a large forecast error is penalized more than a small forecast error for a given category of events.

What’s wrong with this picture??

slide10

ESS Does Not Account For Value of Forecast to Our Customers…

MOST CRAFT

REMAIN

IN PORT

S

C

E

C

S

C

A

slide12

EYW Forecasts: January 2004

This Forecast Was Never Made

Where Did This Number Come From?

This Will Severely Skew ESS Score. As Well As Other Statistical Measures.

slide14

Percent Correct Definition From

Marine Verification Program Glossary

Percent Correct

Definition: Percentage of the correct forecasts (hits) from the total number of events. Formula:

In Other Words: A/(A+B+C+D)

slide16

Some EYW verification results:

  • The majority, almost 89%, of our observations (marginal distribution p(x)) were ≤ 17 kts.
  • Our forecast system had a hard time discriminating between light winds (< 8kts) and winds of 8 – 12 kts.
  • Our forecast system is slightly better discriminating in the 8 – 12 kt range, but still tends to overforecast more often than underforecast.
  • At 13 – 17 kts, our system is the least biased.
  • We underforecast (bias > 1) moderate to high wind events.
  • Forecasters tend to “game” light wind forecasts which do not have much value to customers.
slide17

Gridded Verification at EYW….

Otherwise known as…

What are you Kidding me??

slide18

From Murphy 1993…

“Since Forecasters’ judgments necessarily contain an element of uncertainty, their forecasts must reflect this uncertainty accurately in order to satisfy the basic maxim of forecasting. In general, then, forecastsmust be expressed in probabilistic terms.”

slide19

Melbourne’s ADAS

Closer to grid verification

*Supported by relatively dense surface data network over land

*Buoy in Marine Zones

slide20

LAPS analysis highly influenced by single data point.

Winds in this region most likely NE 10-15 knots.

slide22

We do not have ANY guidance to forecast wave height inside the reef at this resolution and detail, and little skill – therefore we do not attempt to forecast this region in our grids and zero out this region – but do include in our CWF.

Wavewatch III is poor at best outside of the reef – and does not account for gulf stream interactions.

AND NOT ONE WAVE HEIGHT DATA POINT!!

So How Do You Verify Wave Height??

Answer: You don’t.

slide23

Marine Forecast

730 PM EST SUN MAR 14 2004

SMALL CRAFT ADVISORY

TONIGHT: EAST WINDS AROUND 20 KNOTS. SEAS OUTSIDE THE REEF 5 TO 7 FEET...EXCEPT HIGHER IN THE GULF STREAM. SEAS INSIDE THE REEF 2 TO 3 FEET. NEARSHORE WATERS CHOPPY TO ROUGH. SCATTERED SHOWERS.

MONDAY: EAST WINDS 15 T0 20 KNOTS...DECREASING TO 10 TO 15 KNOTS BY NOON. SEAS OUTSIDE THE REEF 4 TO 7 FEET...EXCEPT HIGHER IN THE GULF STREAM. SEAS INSIDE THE REEF 2 TO 3 FEET. NEARSHORE WATERS CHOPPY. SCATTERED SHOWERS.

MONDAY NIGHT: SOUTHEAST WINDS NEAR 15 KNOTS. SEAS OUTSIDE THE REEF 3 TO 5 FEET. SEAS INSIDE THE REEF 2 TO 3 FEET. NEARSHORE WATERS A MODERATE CHOP. ISOLATED SHOWERS.

slide24

GRID VERIFICATION “THOUGHTS”…

  • You must have relatively dense surface data network to support gridded verification field.
  • Perception that numerical models are the better performer are incorrect and not supported.
  • Need high resolution shallow water wave guidance.

Given the current state of model guidance and surface data available for the EYW marine area –

- It is entirely possible that our forecasters will have to DEGRADE our forecasts to match verification fields…

ad