100 likes | 191 Views
This document outlines the necessary format and consistency checks, validation of parameters, date verification, and quality flag examination to ensure data accuracy. It also highlights the importance of profile metadata and calibration information checks for reliable data. Stay updated on the status with the awaited deployment of new server hardware. Aim to enhance data quality standards by following the outlined procedures diligently.
E N D
Format and Consistency Checks DMQC-3 Mark Ignaszewski FNMOC
General Checks “Highly desirable parameters” Dates: Check all dates for validity and consistency QC flags: Check that all QC flags have valid values <PARAM> variables: Check <PARAM> variables for consistency D-files: Perform specific checks identified by the DMQC workshops DMQC-3 September 2008
Highly-desirable Parameters Check for valid values for: • DATA_CENTER • INST_REFERENCE • WMO_INST_TYPE • POSITIONING_SYSTEM • Will only generate warnings DMQC-3 September 2008
Date checks Perform validity and consistency checks on: • REFERNCE_DATE_TIME • JULD • JULD_LOCATION • DATE_CREATION • CALIBRATION_DATE • HISTORY_DATE • DATE_UPDATE Valid date; reasonable; in this sequence (roughly) DMQC-3 September 2008
QC Flag Checks • All QC flags checked for valid values • Includes: JULD_QC, <PARAM>_QC, etc • <PARAM>_QC and <PARAM>_ADJUSTED_QC • Consistent with the respective <PARAM> variable • PROFILE_<PARAM>_QC • Consistent with <PARAM>_QC values ADMT-8 14-16 November 2007
Profile Meta-data Checks • DIRECTION: valid value • DATA_STATE_INDICATOR: valid value • STATION_PARAMETERS • Valid values • Consistent with “<PARAM>” variables DMQC-3 September 2008
Profile Meta-data Checks (cont) • DATA_MODE • R-file: ‘R’ or ‘A’ • D-file: ‘D’ • Check consistency of <PARAM>_ADJUSTED variables • If ‘R’: Set to fill values • If ‘A’ or ‘D’: Set for every parameter DMQC-3 September 2008
D-file Checks • DATA_MODE = ‘D’ • DATA_STATE_INDICATOR = ‘2C’ • Filled for every parameter: • <PARAM>_ADJUSTED • <PARAM>_ADJUSTED_QC • <PARAM>_ADJUSTED_ERROR • Check “sequences” of D-file cycles • Warn if there are R-files cycles embedded within D-file sequences. DMQC-3 September 2008
D-file ChecksCalibration Information For every PARAMETER, verify • SCIENTIFIC_CALIB_COMMENT • CALIBRATION_DATE • Consistent with other dates • Mandatory but cannot validated • SCIENTIFIC_CALIB_EQUATION • SCIENTIFIC_CALIB_COEFFICIENT DMQC-3 September 2008
Status Currently only format is being checked The enhanced format checking is overdue and is awaiting the deployment of new US GODAE server hardware. The target date is now sometime in October 2008. DMQC-3 September 2008