1 / 30

AUTOMATED VALIDATION FOR SUMMARY OF THE DAY TEMPERATURE DATA

AUTOMATED VALIDATION FOR SUMMARY OF THE DAY TEMPERATURE DATA. Authors: William E. Angel, Michael L. Urzen, Stephen A. Del Greco, and Matthew W. Bodosky Special Thanks to Arthur G. DeCotiis. COMMON ACROYMNS. ASOS – Automated Surface Observing System AWOS – Automated Weather Observing System

zandra
Download Presentation

AUTOMATED VALIDATION FOR SUMMARY OF THE DAY TEMPERATURE DATA

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. AUTOMATED VALIDATION FOR SUMMARY OF THE DAY TEMPERATURE DATA Authors: William E. Angel, Michael L. Urzen, Stephen A. Del Greco, and Matthew W. Bodosky Special Thanks to Arthur G. DeCotiis

  2. COMMON ACROYMNS • ASOS – Automated Surface Observing System • AWOS – Automated Weather Observing System • COOP – Cooperative Observers Network • GEA – Graphical Edit Analysis • GMT – Generic Mapping Tools • NEWAE – NEW Areal Edit • TEMPVAL – Automated Temperature Validation System

  3. COOPERATIVE DATA – WHAT IS IT? COOP OBSERVER AM, PM, MID READERS

  4. ~8500 Stations

  5. SIMPLIFIED COOPERATIVE DATA FLOW Archive NWS Publication Data Entry Monthly QC Internet COOP Observer NCDC COOP Forms

  6. COOPERATIVE DATA PROCESSING (TEMPVAL Plug In) Auto Aerial Edit (NEWAE) Interactive Validation (GEA) Records Transcription Last Look TEMPVAL

  7. PRE-TEMPVAL OPERATIONAL OVERVIEW • Validation Based on Cooperative Neighbors • Approximately 70% Automated, 30% Manual • Rules Based – Automated • Unresolved Data – Manual Review by Validator • Compared Daily Departures From Station Monthly Mean Temperature • 3-Sigma Mean Value

  8. PRE-TEMPVAL OPERATIONAL OVERVIEW • Looked For Shifter Signature • Observation Day Versus Calendar Day • Common With AM Readers • Detected Flatliners • Repeating Same Temperature 3 Or More Days In A Row

  9. WHY TEMPVAL? • Improve Validation Accuracy • Change in Methodology • Use of Grid Fields Instead of COOP Neighbors • High Quality of “Ground Truth” Data • Eliminate Validator Subjectivity • Make Edits Uniform and Consistent • Reduce Manual Effort And Production Time • Reason For Automation • Previous System In Use Since Early 1990’s • Time For An Update

  10. “GROUND TRUTH” DATA • Use Of Approximately 900 ASOS Sites • Use Of Approximately 450 AWOS Sites • Use Of Approximately 300 Canadian ASOS Sites • Data Is Gridded On A ½ Degree Resolution Using GMT Software • Inverse Distance Squared Interpolation • Data Undergoes A “Bull’s Eye” Grid Test • Slope Test Comparing A Grid Point Intersection Value With Surrounding Grid Point Values

  11. HOW DOES TEMPVAL WORK? • Computes Daily Max/Min Departure From Monthly Mean Max/Min (same as previous system) • Builds 1/2 Degree Grid Fields By Reader Time From Quality Controlled ASOS/AWOS Hourly And Summary Of Day Data • Compares Cooperative Daily Departure With ASOS/AWOS Grid Departures • Generates Edits By Adding Grid Departure To The Cooperative Station Monthly Mean Max/Min

  12. TEMPVAL FLOWCHART Build ASOS Daily Max/Min Departure Grid Analyses (AM,PM,Mid) Detect and Remove Bull’s Eyes Test Cooperative Max/Min Departure Against Grid Value No Validates/Not Missing? |OV - grid| < 7 + grad Apply Grid Departure To Station Mean Max/Min Yes Continue Edit OV Note: OV represents the coop max/min departure value. Note: stations on “Exception List” bypass this process End

  13. PARALLEL TESTING RESULTS • Tested for |COOP - GRID| >= 7.0 + Gradient • TEMPVAL Generated 12,904 (8.63%) Edits • TEMPVAL Generated 6060 (4.05%) Temperatures For Missing Observed Temperatures • Manually Validated 100 Outlier Cases (Operational Aerial Edit versus TEMPVAL) • TEMPVAL Results Are More Realistic Than Operational Results In 90% Of Cases

  14. ASOS Values GEA Values

  15. ASOS Values GEA Values

  16. Skewness = 1.279 Difference of the TempVal Value and the Observed Value

  17. SHORTFALLS OF TEMPVAL • Limited In Capability To Effectively Interpret Particular Types Of Weather, Such As • Warm Air Advection Episodes For High Elevation Stations • Inversions • Strong Frontal Passages • “Backdoor Cold Fronts” • Lack Of Sufficient ASOS/AWOS Data Does Not Allow TempVal To Be Performed In Puerto Rico, Virgin Islands, Or The Remaining Territories

  18. BENEFITS OF TEMPVAL • Fully Automated “Plug-In” Module • Edits Are Uniform and Systematic • Generate Daily Replacement Temperature Values • Re-Analysis Is Possible • Additional Algorithms Easily Added • Currently Testing A “Time Shifter” Algorithm • Retain Original Data • Database Contains Original And Edited Values • Same As Previous System

  19. CONCLUSIONS • ASOS/AWOS Station Density And Data Quality Are Sufficient To Produce Departure Grid Fields • Parallel Testing Shows That TEMPVAL Outperforms The Previous System • Eliminates Manual Processing And Removes Associated Validator Subjectivity • TEMPVAL Been Operational Since The January 2002 Data Month • Currently Processing November 2002 Data

  20. QUESTIONS - COMMENTS THANKS!

More Related