1 / 16

Decoder Discussion Wednesday, November 18

NextGen Network Enabled Weather Product Formats Workshop, Boulder CO Decoder Discussion Brian Gockel NOAA/NWS Office of Science and Technology November 18, 2009. Decoder Discussion Wednesday, November 18. Scope of Decoder Discussion What Decoder-Related Problems Should Be Anticipated?

kamil
Download Presentation

Decoder Discussion Wednesday, November 18

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. NextGen Network Enabled WeatherProduct Formats Workshop, Boulder CODecoder DiscussionBrian GockelNOAA/NWS Office of Science and TechnologyNovember 18, 2009

  2. Decoder Discussion Wednesday, November 18 • Scope of Decoder Discussion • What Decoder-Related Problems Should Be Anticipated? • Based on experience • Based on foresight [More of the same?] • What Standards Apply (OFCM, WMO, …)? • Differing standards versions • Dealing with overlapping standards • Legacy Systems Transition • Decoder Metadata & Need for Consistency • Example: Site Identification Inconsistency • Rules (what is allowed, not allowed?) • Example: METAR remarks field • Example: Concatenated messages Slide 2

  3. Decoder Discussion - continued Wednesday, November 18 • Local Tables • Other Decoder Challenges (Input from Group) • Governance • Definition and establishment of governance • Leveraging existing governance bodies • Authoritative (oversight) and technical (execution) layers needed • Need for ongoing interagency collaboration– a long-term activity • Next Steps • Very near term (e.g., FY10) and beyond FY10 Slide 3

  4. Scope of Decoder Discussion Software for encoding and decoding environmental products, such as netCDF, GRIB, Radar, BUFR, METAR, etc. This includes the executable software as well as the configuration files (i.e., “decoder metadata”*) upon which the encoders and decoders rely. Motivation for Discussion: maximize compatibility between data-provider encoders and data-consumer decoders. Incompati-bilities or inconsistencies prevent end users from fully decoding and/or using the desired information (even in cases where product delivery works). * Note that the term “decoder metadata,” as used here, is different from reg-rep metadata. Slide 4

  5. From Mike Asmussen (Skjei) slide package, NOAA/NextGen System of Systems Workshop. Annotations in green & red. Grid Decode Grid Encode Grid Decode Grid Decode Grid Decode Grid Decode Slide 5

  6. Decoder Discussion Establishing compatibility is a near-term (developmental) and a long-term (O&M) challenge. . . 1. New environmental models are developed; legacy models are eventually phased out. 2. With time, new parameters are added (e.g., model-simulated radar reflectivity at certain heights or elevations).* 3. Stations renamed, added, subtracted & moved (e.g., radar relocation or new airport).* * Examples in backup slides. Slide 6

  7. Decoder Issues – METAR Example • METAR – The domestic METAR code format is described in the Federal Meteorological Handbook (FMH) No. 1 "Surface Observations and Reports", (OFCM). The METAR format contains a REMARKS field… can be of 2 types: • Automated/Manual/Plain Language • Additive and Maintenance Data • Example: • METAR KOKC 082252Z 14016KT 6SM HZ SCT035 OVC240 26/23 A2957 RMK AO2 PK WND 29031/2201 TSE37RAE25GSE2158GRB2158E10 SLP999 HAILSTONES>3/4TSE MOV NE P0075 T02610233 • From FMH #1: “Hailstone Size [coded as] (GR_[size]) [Plain Language]. At designated stations, the hailstone size shall be coded in the format, GR_[size]… Where plain language is called for, authorized contractions, abbreviations, and symbols should be used to conserve time and space. However, in no case should an essential remark, of which the observer is aware, be omitted for the lack of readily available contractions. In such cases, the only requirement is that the remark be clear. For a detailed list of authorized contractions, see FAA Order 7340 Series, Contractions.”TSEshould probably be “TS E” (thunderstorm East). “TSE” most often means “thunderstorm ended.”From:http://www.srh.noaa.gov/oun/storms/20030508/metarobs.php (May 8, 2003 case) Slide 7

  8. Decoder Issues More METAR Examples Denver: KDEN 150053Z 07013KT 4SM -SN BR SCT018 OVC036 01/M01 A2988 RMK AO2 SLP123 P0000 T00061011 Amsterdam: EHAM 150125Z 24012KT 9999 FEW028 11/07 Q1006 NOSIG Kiev: UKKK 150100Z 04002MPS 9999 SCT010 OVC023 04/01 Q1022 NOSIG MPS = Meters per sec (versus KT=knots); NOSIG=No significant changes in weather since last report. Slide 8

  9. BUFR Table B – Multiple Versions dx3-napo> ls -lt | grep -i bufr | grep -i tableb-rwxrwxr-x 1 fxa fxalpha 43062 Nov  3  2008 bufrTableB-rwxrwxr-x 1 fxa fxalpha 52780 Mar 14  2006 LAMPBufrTableB-rwxrwxr-x 1 fxa fxalpha 14029 Jun 29  2005 GFSBufrTableB-rwxrwxr-x 1 fxa fxalpha 14029 Jun 29  2005 ETABufrTableB-rwxrwxr-x 1 fxa fxalpha 12028 Apr 29  2004 NGMBufrTableB-rwxrwxr-x 1 fxa fxalpha 13413 Apr 24  2003 AVNBufrTableB-rwxrwxr-x 1 fxa fxalpha 13337 Apr 14  2003 MRFBufrTableB-rwxrwxr-x 1 fxa fxalpha 12028 Mar  2  2001 HPCBufrTableB-rwxrwxr-x 1 fxa fxalpha 12028 Oct 11  2000 MosBufrTableB dx3-napo> wc -l HPCBufrTableB MosBufrTableB  157 HPCBufrTableB  157 MosBufrTableB  314 totaldx3-napo> diff HPCBufrTableB MosBufrTableB 144a145,146> 060071|PROB. OF PCPN AMT GE .01 INCH PAST 6HRS |%   |  0|    0|  7> 060072|PROB. OF PCPN AMT GE .01 INCH PAST 12HRS|%   |  0|    0|  7156,157d157< 060071|UNKNOW                        |NUMERIC       |  0|    0|  1< 060072|UNKNOW                        |NUMERIC       |  0|    0|  1 Slide 9

  10. Decoder & Decoder Metadata Governance • Definition and establishment of governance – oversight body that establishes and maintains consistency in decoders and metadata; includes NextGen participating-system membership, is interagency, and exploits existing oversight/CM bodies. • Existing governance bodies:* a. NOAA Data Management Committee/DMIT (NOAA oversight) • b. CCBs (system-level) • Need for NextGen-related interagency collaboration (in the development and maintenance phases). How? • Is there a need for a NextGen developmental phase decoder team to work demonstration and IOC decoder issues? • Other questions: who should participate? how to collaborate? * adequate??? Slide 10

  11. Decoder & Decoder Metadata Proposal for a Long-Term Governance Structure NOAA DMIT FAAoversight organization DOD (and other) oversight organizations(?) ?? NextGen Decoder Work Group ?? Individual-System CCBs NOAANet CCB FAA CCBs NCEP CCB JMBL CCBs SPSRB (satellite) CCB/DRG AWIPS CCB MDL CCB NWSTG CCB/DRG Radar/ROC CCB Slide 11

  12. Questions & Possible Next Steps Decoder & Decoder Metadata • Role (charge or scope) of any new decoder working group may need more clear definition and NOAA/FAA buy in. • Identify participants. • Inform system owners. • Other next steps??? Slide 12

  13. Decoder Discussion - Backup Slide Addition of New Parameter* WHY CAN'T I FIND THE [Model simulated] RADAR PRODUCTS….. Assuming you are using the nam.tHHz.awip12FF.tm00 files at ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/nam/prod/nam.YYYYMMDD/ where HH is the cycle time = 00, 06, 12 or 18, FF is the forecast hour = 00, 01, 02 ... 24 and YYYYMMDD is year, month & day; you are probably not seeing them because the new radar reflectivity fields are defined in NCEP GRIB parameter table version #129 (http://www.nco.ncep.noaa.gov/pmb/docs/on388/table2.html#TABLE129), and you are expecting them to be defined in the normal/default Table #2 (http://www.nco.ncep.noaa.gov/pmb/docs/on388/table2.html). We had to put them in this alternative table because Table 2 had no room for new parameters to be added. According to the GRIB documentation (see http://www.nco.ncep.noaa.gov/pmb/docs/on388/section1.html) you can find which Parameter Table is being used in octet 4 of the GRIB PDS (Product Definition Section). Note the kpds5 entried above and that in Table 129, derived model reflectivity is variable #211 and composite reflectivity is variable #212. In Table 2 these would have been upward short wave and long wave flux, and this is probably what your existing processing assumed they were. * Excerpt from EMC model information page. Slide 13

  14. Decoder Discussion - Backup Slide Station Change NWS Region email to NWS/OS&T: “…in March 2008 we had asked at the request of WFO BTV to change the Plattsburgh, NY sounding point to account for the ASOS (and TAF) relocation from KPLB to KPBG. I'm not sure if NCEP has made this change yet.” New Location CD STATION ICAO IATA LAT LONG ELEV(m) M N V U A C NY PLATTSBURGH INTL KPBG PBG 44 39N 073 28W 71 Z T 8 US Old Location NY PLATTSBURGH KPLB PLB 44 41N 073 32W 106 X V A 6 US ASOS METAR TAF NAM Sounding Slide 14

  15. Decoder Discussion - Backup Slide Station Lists: NWS Location Identifier Database https://ops13jweb.nws.noaa.gov/nwsli/liu/ApprovedStations.jsp (Requires UID & password) Slide 15

  16. Decoder Metadata Example – Backup Slide Existing Data Format Standard (html format) http://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc.shtml Slide 16

More Related