1 / 8

What is new at GDACs ?

What is new at GDACs ?. codac@ifremer.fr mark.ignaszewski@navy.mil. What is new at GDACs. Action 1 : Argo profile detailed index Calculate time delay for getting R-files and D-Files onto the GDAC. Investigate files slowly arriving.

rae-sears
Download Presentation

What is new at GDACs ?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. What is new at GDACs ? codac@ifremer.fr mark.ignaszewski@navy.mil

  2. What is new at GDACs • Action 1 : Argo profile detailed indexCalculate time delay for getting R-files and D-Files onto the GDAC.Investigate files slowly arriving. • ftp://ftp.ifremer.fr/ifremer/argo/etc/argo_profile_detailled_index.txt.gz • ftp://ftp.ifremer.fr/ifremer/argo/etc/argo_profile_detailled_index.txt.gz.md5

  3. Argo detailed index • file, date, latitude, longitude, ocean • profiler_type, institution • date_update: date of last update of the file • profile_temp_qc, profile_psal_qc, profile_doxy_qc • ad_psal_adjustment_mean : for delayed mode or adjusted modeMean of psal_adjusted – psal on the deepest 500 meters with good psal_adjusted_qc (equal to 1) • ad_psal_adjustment_deviation : for delayed mode or adjusted modeStandard deviation of psal_adjusted – psal on the deepest 500 meters with good psal_adjusted_qc (equal to 1) • gdac_date_creation : creation date of the file on GDAC • gdac_date_update : update date of the file on GDAC • n_levels :maximum number of pressure levels contained in a profile

  4. File removal : under validation • Action 10 : Automate file removal according to the agreed procedure • A DAC submit a "removal file" to GDAC • The "removal file" contains one line per file to remove."Removal file" collection from DAC to GDAC : • Query xxx_removal.txt file in each DAC submit directory;xxx must be identical to the DAC (eg : aoml, coriolis); otherwise the file is rejected. • Check the format of xxx_removal.txt . The whole file is rejected is the format check fails. • File name : valid Argo file name; the corresponding meta-data file must exist for this DAC • Move all the named files from GDAC into a etc/removed directory • The removed files are kept for 3 months in the etc/removed directory and erased after that delay.

  5. Action 12 : Implement an MD5 signature to secure file transfer and document it • MD5 signature • For each update of a GDAC file, an MD5 signature is produced. The MD5 signature file allows user to check that the file he collected through FTP is identical to the original file. • The MD5 signatures are stored in etc/md5 directory • Still under test, should be available in October.

  6. Action 16 : Document Grey list submission • Each DAC maintains a greylist that is submitted to the GDAC for updates. The DACs greylist are collected by the GDAC and merged into a global Argo greylist. • Greylist file collection from DAC to GDAC : • Query xxx_greylist.csv file in each DAC submit directory;xxx must be identical to the DAC (eg : aoml, coriolis); otherwise the file is rejected. • Check the format of xxx_greylist.csv . The whole file is rejected is the format check fails. • Floatid : valid Argo float id; the corresponding meta-data file must exist • Parameter : PSAL, TEMP, PRES or DOXY • Start date : YYYYMMDD valid, mandatory • End date : YYYYMMDD valid, fill value : ',,' • Flag : valid argo flag • Comment : free • DAC : valid DAC, mandatory • Remove all the floats of the DAC from the GDAC grey list and add the content of the submitted xxx_greylist.csv file • Note : after each submission, a copy of the Argo greylist is stored in etc/greylist/ar_greylist.txt_YYYYMMDD • The global Argo greylist is sorted by DAC, PLATFORM_CODE and START_DATE in alphabetical order.

  7. Action 19 : Coriolis to provide feedback on anomalies detected by statistical analysis in text files • Every day, an objective analysis is performed on the 30 latest days of observations. The objective analysis detects anomalies that are visually quality controlled by an operator. • Each flag correction performed by the operator is recorded. • The flag correction is not reported to Argo NetCDF files. The list of corrections is created once a day for each DAC : ftp://ftp.ifremer.fr/ifremer/argo/etc/ObjectiveAnalysisWarning ar_scoop2_BO_20090923013646.csv

  8. Action pending • Add the parameter list to detailled index file • Action 11 : Modify the “latest data” directory to handle a sliding of 3 months and separate R and D data. • Synchronize metadata files

More Related