SHEF* Primer *Standard Hydrometeorological Exchange Format - PowerPoint PPT Presentation

shef primer standard hydrometeorological exchange format n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
SHEF* Primer *Standard Hydrometeorological Exchange Format PowerPoint Presentation
Download Presentation
SHEF* Primer *Standard Hydrometeorological Exchange Format

play fullscreen
1 / 72
SHEF* Primer *Standard Hydrometeorological Exchange Format
109 Views
Download Presentation
duer
Download Presentation

SHEF* Primer *Standard Hydrometeorological Exchange Format

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. SHEF* Primer*Standard Hydrometeorological Exchange Format Mike Callahan WFO Louisville September 29, 2009

  2. Objectives • Know the three different types of SHEF reports and when they are used • Understand the various elements in a SHEF report • Understand the different descriptors in a SHEF data element • Know how to encode/decode SHEF reports

  3. The SHEF Manual • The SHEF Manual Version 2.2 (NWSM 10-944) is the ultimate source of all information about SHEF • Every office should have a copy of this manual • It is available online in PDF format at www.nws.noaa.gov/directives • This primer refers to it constantly, get it and be prepared to use it

  4. The Three Type of Reports • The .E format is used to encode an EVENLY spaced sequence of data from a SINGLE location • The .B format is used to encode a collective of SIMILAR data from a GROUP of locations • The .A format is used for everything else

  5. A Simple ExampleWe want to encode this data: • The river gage reading at Deputy IN was 13.45 at 7:00 AM EST on 4/1/2009 • Since we have a single piece of data for one location we will use the .A format. • We need: • The location ID: DEPI3 • The date and time of the data: 20090401 E DH0700 • The type and value of the data: HG 13.45

  6. .A Reports • .A reports are the simplest SHEF report • These reports can encode any type of data but may not be the most efficient choice • Use for a set of non-repeating or random sequence of data for a single location • Split into two sections, positional fields and a data string • Using the data from the previous slide, lets generate our first SHEF report: .A DEPI3 20090401 E DH07/HG 13.45 data string positional fields

  7. .A Positional Fields .A DEPI3 20090401 E Time Zone Date Location ID Format Specifier

  8. Format Specifier • Specifies the report type • Only choices are .A, .B, or .E • Optionally followed by an R for a revision or a number for a continuation • Data stored in the AWIPS database as missing will be overwritten regardless of the revision code • The revision code must be present to overwrite data non-missing already in the AWIPS database • When revising, make sure the date and time of the new data exactly matches the old data

  9. Location ID • An approved ID listed in the NWSLI database • It is possible to encode data from non-ID locations using stranger reports • Stranger reports use the latitude and longitude as an ID • Xlatlong: North latitude, West longitude • Ex. X4511001 = 45.1 N latitude, 100.1 W longitude • Ex. X3090988 = 30.9 N latitude, 98.8 W longitude • Be sure to include the leading zero for longitudes less than 100 degrees • Ylatlong: North latitude, East longitude • Ex. Y5030202 = 50.3 N latitude, 20.2 E longitude

  10. Date • See SHEF Manual, Section 4.1.4 • mmdd: Month and Day • If year is missing, the decoder will look 6 months in the future and 6 months in the past to assign the year nearest to the current date • Ex: 0401, 1225 • yymmdd: Year, month, and day • If a two digit year is used, the decoder will look 10 years in the future and 90 years in the past to assign the century • Ex: 090915, 990210 • yyyymmdd: 4 digit year, month, and day • Ex: 20091010, 19950415

  11. Time Zone • See SHEF manual, Table 8 • Common codes are: • Z: UTC (GMT) default • E: Eastern Local Time (follows shift between standard and daylight saving time) • C: Central Local Time • M: Mountain Local Time • P: Pacific Local Time • L: Alaskan Local Time • H: Hawaiian Local Time • Followed by a D to force daylight time or S to force standard time

  12. Quick Quiz: .A Positional FieldsGrab a pen and write down your answers • 1. List the elements of the positional fields • 2. What format specifier is used to correct a previous .A report • 3. Encode the date for July 4, 2009 • 4. Encode Mountain Daylight Time

  13. Answers to .A Positional Fields • 1. Format specifier, Location ID, Date, Time Zone • 2. .AR • 3. 20090704 or 090704 • 4. MD

  14. Data String DH07/HG 13.45 • All elements in the data string must be separated by slashes (/) expect for the parameter code and data pair Value Parameter Code Time

  15. Time • See SHEF Manual, Table 9 • Default is 12:00:00 for Z time zone, 00:00:00 for all others, missing time defaults are rarely used and could lead to some confusion • DHhhmmss = Data hour, minutes, seconds • mm and ssdefault to 00 • DDddhhmmss = Data day, hours, minutes, seconds • Use this if the day is not the same as the day in the positional code • DMmmddhhmmss = Data month, day, hours, minutes, second • DYyyyymmddhhmmss = Data year, month, day, hours, minutes, second

  16. Time Relative • DRt±xx • Adds or subtracts xx units of time t to/from the last explicit time • t values • S: seconds, N: minutes, H: hours, D: days, M: months, Y: years, E: end of month • Ex: • DRN+60 or DRH+1 = add an hour • DRD-1 = subtract one day

  17. Date Relative End of Month • DRE±x • Allows you to encode end of the month values without knowing how many days are in the month • x: the number of months • DRE+1 = the last day of the next month • DRE+2 = the last day of the next month • Ex: encode spring (Mar., Apr., May) monthly precipitation for ALTP1 • .A ALTP1 0331 E DH07/PPM 4.46/DRE+1 .A1 3.56/DRE+2/6.80

  18. Parameter Codes PEDTSEP • All codes except PE have a default Probability Extremum Source Type Duration Physical Element

  19. Physical Element • See SHEF Manual, Table 1 & Appendix G • First Letter Category, Second Letter Type, no default • Categories: • A-C: agricultural, E: evaporation, F: fish, • G: ground, H: height, I: ice, L: lake, • M: moisture & fire/fuel, N: dam & gate, • P: pressure & precipitation, Q: discharge, • R: radiation, S: snow, T: temperature, • U: wind, V: electrical, W: water quality, • X: weather, Y: other

  20. Physical Element (con’t) • Some common codes • HG: river stage height • HP: pool elevation • HT: tailwater elevation • PC: accumulated precipitation • PP: incremental precipitation • SD: snow depth • SF: snowfall • SW: snow water equivalent • QT: total discharge • TA: air temperature

  21. Special Physical Elements • Some physical elements are complex and have their own tables in the SHEF Manual • IR: ice report, Table 14 • SR: snow report, Table 15 • GR: frost report, Table 16 • PT: precipitation type, Table 17 • GS: ground state, Table 18 • HI: river trend, Table 19 • AF: surface frost intensity, Table 20 • AM: surface dew intensity, Table 21 • PM: precipitation probability, Table 22 • XW: present weather, Appendix C • XP: past weather, Appendix D

  22. Duration • See SHEF Manual, Table 3 • Some common codes: • I: instantaneous, default for most PE codes • D: 1 day, default for PP, SF, & others, see Table 7 • E: 5 minutes • G: 10 minutes • C: 15 minutes • H: 1 hour • T: 3 hours • Q: 6 hours • K: 12 hours • P: total since the previous 7AM • V: other, duration defined using DV code

  23. Special Durations • If the duration you want is not listed in Table 3, use the DVtxx code: • t values • S: seconds, N: minutes, H: hours, D: days, • M: months, Y: years • Z: reset duration back to default • Ex: • DVH2/SFV = 2 hour snowfall • DVN45/PPV = 45 minute precipitation

  24. Be Careful with P Durations • C DH07/PPP 0.35 • 24 hours of precipitation, 7:00AM yesterday to 7:00AM today • C DH06/PPP 0.50 • 23 hours of precipitation, 7:00AM yesterday to 6:00AM today • C DH0730/PPP 1.20 • 30 minutes of precipitation, 7:00AM today to 7:30AM today! • C DH0730/PPD 1.20 • 24 hours of precipitation, 7:30AM yesterday to 7:30AM today. • PPP is can be used for non-7AM observer reports if the observer dumps the gage at 7AM • Use PPD for morning reports

  25. Type • See SHEF Manual, Table 4 • R: Reading (observed), default • C: Contingency • F: Forecast • H: Historical • M: Model • P: Non-forecast Processed

  26. Source • Meaning changes with Type code, see SHEF Manual, Table 4 • Some common Type & Source code pairs • RZ: non-specified (observer) reading, default • RG: DCP reading • RP: phone reading (LARC, CR10, etc.) • RR: radio reading (ALERT, IFLOWS, etc.) • FZ: non-specified forecast, default for F type • FF: forecast including QPF

  27. Extremum • See SHEF Manual, Table 5 • Some common codes • Z: no extremum, default • N: minimum of day • H: minimum of 6 hours • P: minimum of 12 hours • X: maximum of day • R: maximum of 6 hours • Y: maximum of 12 hours

  28. Probability • See SHEF Manual, Table 6 • Some common codes • Z: no probability, default • A: .2% • 1: 10% • G: 25% • 3: 30% • 5: 50% • H: 75% • 8: 80% • Y: 99.8% • M: mean expected value

  29. Send Codes • Many parameter code combinations are so common that a new PE has been created • See SHEF Manual, Table 2 • Some common codes • TNTAIRZNZ = minimum air temperature of the day • TXTAIRZXZ = maximum air temperature of the day • HNHGIRZNZ = minimum gage height of the day • HXHGIRZXZ = maximum gage height of the day • PFPPTCFZZ = 3 hour flash flood guidance • SFSFDRZZZ = 24 hour snowfall

  30. Values • The data value • Trace amounts are encoded as T or t, stored as .001 • Missing data are encoded as M, m, MM, mm, +, -, or -9999 • For precipitation data: values missing a decimal will be divided by 100, thus PP 1 is stored as 0.01! This feature is obscure and may lead to confusion. • Data can be followed by a qualifier • See SHEF Manual, Table 10 • Some common codes • E:estimated, Q: questionable, F: flagged • R: rejected, M: manual edit, B: bad • Ex: PP 5.07Q

  31. Retained Comments • It is possible to attach comments to data values that will be stored • Follow the data value with the comment enclosed in single or double quotes • The comment should not exceed 80 character in length • Ex: we want to note that the temperature was from a backup thermometer • TA 88 "backup thermometer"

  32. Quick Quiz: .A Data Strings • 5. List the elements in data strings • 6. Encode 12:00 UTC • 7. Encode 12 hour new snowfall • 8. Encode an estimated river stage at 18:15 UTC of 3 feet

  33. Answers: .A Data Strings • 5. Time, parameter code, value • 6. Z DH1200 or Z DH12 or DH1200 or DH12 or nothing • 7. SFK • 8. DH1815/HG 3.0E

  34. Vector Physical Elements • Some Physical Elements have paired values, see SHEF Manual section 7.4.6 • Lets look at TV: temperature in vegetated soil at depth • The paired values are the depth and the temperature • The two values are encoded like this: dd.tttt, where dd is the depth in inches and tttt is the temperature in degrees and tenths in °F • Ex: • 12.0621 = 62.1 °F at 12 inches depth • 5.450 = 450.0 °F at 5 inches depth (meant 5.045) • -1.0022 = -2.2 °F at 1 inch depth

  35. Back to the Simple Example • The river gage reading at Deputy IN was 13.45 at 7:00 AM EST on 4/1/2005 • Data time: 20090401 E DH0700 • We could leave off the century or year in the date • We will use DH since the data day is the same as the positional field day • We could leave off the minutes • River gage reading is HGIRZZZ 13.45 • We could leave off IRZZZ since they are the default

  36. Putting the Report Together .A DEPI3 20090401 E DH07/HG 13.45 • The data elements are separated by a / • To continue the report on the next line use .An, you don’t need slashes at the end or beginning of a line • .A DEPI3 0401 E DH07/HG 13.45 .A1 PPD 0.15/TA 45/TX 60/TN 30 • To correct a bad report use .AR • .AR DEPI3 0401 E DH07/HG 12.45 • Each station is on a different line • .A ABEK2 0401 E DH07/PPD 0.25 • .A BNNK2 0401 E DH07/PPD 0.34

  37. Other Data String Elements • DCccyymmddhhnn:Creation Date • Defaults to same as observation time • Required for river forecasts • Similar to Date codes • DUa:Units Code • DUE = English units, default • DUS = Standard units • DQa:Qualifier Code • Same as data qualifier but applies qualifier to ALL data

  38. Internal Comments • Allowed anywhere in report • Colon (:) turns comments on • End of line or another colon turns comments off • SHEF decoder ignores them and they are not stored • Ex: • .A ABYV1 1123 DH11/PPD 0.23 : Albany VT • .A BELV1 : Belle VT : 1123 DH11/PPD 0.12

  39. Quick Quiz: .A Reports • 9. Encode data AAWC1 for 5/20 at 08:30 PDT: 24 hour pcpn 1.23, 24 hour max temp 70 °F, 24 hour min temp 50 °F • 10. At BLWW1, a DCP, the river crested on 7/4 at 34.5 feet at 1623Z, correcting an earlier incorrect crest of 34.2 feet • 11. On 6/1 at 1500 EDT, you get a questionable report of 5 inches of rain in the last 12 hours at latitude 40.3° and longitude 105.0°

  40. Quick Quiz: .A Reports (con’t) • 12. BNNM8, an ALERT gage, collects 3.45 inches over the last 5 hours ending 8/9 at 13Z. • 13. LARC river readings at 6Z on 10/12: BSNK2 5.67, CYCK2 12.34, FFTK2 10.78 • 14. Hourly river readings for PTTP1 on 8/9 starting at 18Z: 9.45, 9.47, 9.50, 9.53

  41. Answers: .A Reports • 9..A AAWC1 0520 PD DH0830/PP 1.23/TX 70 .A1 TN 50 • PP defaults to PPDRZZZ, TX defaults to TAIRZXZ, TN defaults to TAIRZNZ • 10. .AR BLWW1 0704 DH1623/HGIRGX 34.5 • Must use .AR to correct a bad report • Don’t need Z time zone since this is the default • Cannot use HX since the send code translates to HGIRZXZ • 11. .A X4031050 0601 ED DH15/PPK 5.0Q • Be sure to include the Q qualifier

  42. Answers: .A Reports (con’t) • 12. .A BNNM8 0809 DH13/DVH5/PPVRR 3.45 • Must use DVH05/PPVRR since there is no duration code for 5 hours • 13. .A BSNK2 1012 DH06/HGIRP 5.67 .A CYCK2 1012 DH06/HGIRP 12.34 .A FFTK2 1012 DH06/HGIRP 10.78 • There is a better way to encode this data • 14. .A PPTP1 0809 DH18/HG 9.45/DRH+1/HG 9.47 .A1 DRH+2/HG 9.50/DRH+3/HG 9.53 • Good use of the time relative code • There is also a better way to encode this data

  43. The .B Format • Problem 13 shows that the .A format can get redundant for collectives with the same data and time • This is why the .B format exists • The repeated information is gathered into a header and is separated from the data

  44. .B Reports .B LMK 1012 DH06/HGIRP BSNK2 5.67 CYCK2 12.34 FFTK2 10.78 .END Header • .B reports consist of three parts • Header, which describes the data • Body, which contains the data • Terminator, .END Body Terminator

  45. .B Header .B LMK 1012 DH06/HGIRP • Similar to the .A format • Positional fields are the same except that the location ID is replaced with the message source ID • The data string is replaced with the parameter control string positional fields parameter control string

  46. .B Parameter Control String • Similar to the .A data string without the data • Possible elements are the same as .A data string • DH: data time, DC: creation time • DU: units, DQ: qualifier, DV: duration • Parameter codes are listed together separated by slashes

  47. .B Body BSNK2 5.67 CYCK2 12.34 FFTK2 10.78 • Each line in the body: • Location ID • One or more spaces • The data for that location separated by slashes

  48. .B Terminator .END • Required after the body • Other SHEF reports may follow

  49. Another .B Example • Encode the following collection: Location Ztime Pcpn Max Temp Min Temp ATDK1 1200 1.23 78 45 BELK1 1200 M 67 43 CBDK1 1200 0.56 DSOK1 1100 1.56 75 40 ECLK1 1200 0.89 78 M • The data is similar but differs: • BELK1 pcpn report is missing • CBDK1 does not have a thermometer • DSOK1 time was at 11Z not 12Z • ECLK1 minimum temp is missing

  50. First: Create the Header .B TOP 0614 DH12/PPD/TX/TN Parameter Codes Time Date Message Source Format Specifier