1 / 29

Improved and automated QC process Reinhard Hanuschik and the DPQC group

Improved and automated QC process Reinhard Hanuschik and the DPQC group. fast data transfer: July 2008 full data stream VLT and VLTI (ex burst-mode data) > 50% within 10 minutes new ways in data processing @ DPQC impact on shared QC 2 main goals: calibration completeness

rane
Download Presentation

Improved and automated QC process Reinhard Hanuschik and the DPQC group

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Improved and automated QC process Reinhard Hanuschik and the DPQC group • fast data transfer: July 2008 • full data stream VLT and VLTI (ex burst-mode data) • > 50% within 10 minutes • new ways in data processing @ DPQC • impact on shared QC • 2 main goals: • calibration completeness • calibration quality

  2. Mission #1: calibration completeness (“save last night”) 1. Setting the stage • calChecker • calibrations incomplete: science OBs to be repeated • focus on calibration plan: • known, predictable, configured • can be checked automatically

  3. Mission #2: calibration product quality (“save next night”) combining Health Check monitor and scores: quick-look HC monitor

  4. any problem not discovered on site: up to 10 days affected strong need for 2 QC processes:daytime astronomer Paranal (preliminary),QC scientist Garching (final) basic real-time QC with opslog data until 2008: data acq (Paranal) data in Garching (after ~ 10 days) opslog

  5. now (2009): fast data delivery • fast QC feedback • improved QC feedback • measure quality (QC1 parameters) • compare quality (trending) • rate quality (scoring) new! data acq (Paranal) data in Garching

  6. 2. calChecker • www.eso.org/qc/ALL/daily_cal.html • evaluation for 7 last days (columns) and all science setups (rows) • driven by science data • checks: • all science setups for a given instrument • checks calibration plan • knows about validity and required number of files  life examples

  7. evaluates each box in colors: • green (OK):all calibrations complete and within validity(e.g.: 5 BIAS & 3 FLATs & 1 ARC & 1 STD within validity) • yellow (NOK):calibrations complete but some outdated(e.g.: as above but 3 FLATs are 2 days old) • red (MISS): calibrations incomplete, at least one missing(e.g.: 5 BIAS & 3 FLATs ok, 1 ARC & 1 STD missing) • missing or outdated calibrations are indicated

  8. links to details • for each box: complete list of science files • prog_id, obs_id etc. for easy reference • OB grade and comment: useful for analysis

  9. BOCAL, EOCAL • signal about daytime calib queue • calChecker displays “pending”, “ongoing”, “finished” for current date • ANALYSIS • QC scientist to provide analysis/comments, but this requires interaction ( not always available!) • may override tool NOK/MISS results

  10. called: once per hour by cronjob • has update information, knows about its age don’t hesitate to contact QC!

  11. documentation • association rules • contact • planned: • include ticket mechanism (PSO scheme) • memory function for long-term calibrations (once a month etc.) and HC calibrations

  12. general rule: calChecker follows calOBbuilder • not planned: • short-term night calibrations (like standard stars within 2 hrs) • complex dependencies (multiple standard stars a night; alternative calibrations; calibrations for calibrations) • to be done on Paranal: • optimize calOBbuilder queue (input provided by QC)

  13. 3. Links to product quality calChecker HC monitor products/AB monitor • goes beyond completeness, includes quality • links to the QC processing page for that date (AB product monitor)

  14. AB product monitor

  15. Scores • automatic evaluation scheme: • 2 thresholds defined • QC1 parameter evaluated • total AB score: 0 / 1 / more than 1 • counts outliers

  16. links to other reports • data report • nightlog (password protected, sciops)

  17. 4. Quick-look HC monitor calChecker HC monitor products/AB monitor • www.eso.org/HC • final QC Garching data now display as quick as/quicker than opslog!(forget 10 day gap…)

  18. full version score version (Quick-look)

  19. Information on demand:click on “more” if you want graphs, on “less” if you want scores only

  20. AB product monitor: all scores of a date • HC monitor: selected scores • selection: “relevant for instrument health”

  21. score propagation: • data scores for last week only • plot: last data scores summed upplot score green if last data score is green (previous red scores ignored) • report score: plot scores are aggregated in report score • report scores in group scores, and finally in instrument score • all higher-level scores turn red if at least one lower-level score is red

  22. big advantage of score hierarchies: • instrument score green  you know within a second: all is OK! • red: the score overview can tell you more • scheme intended to guide you to issues

  23. Health Check monitor • update pattern: refresh most important ones upon new opslog data • less important ones at least once per 24 hour • process health • monitored: opslog transfer, parser (1x per hour) • coming soon: data transfer, ngas access

  24. planned: • defocus the OPSLOG data (to become a fallback solution except for VIRCAM/OCAM) • improve update pattern for HC reports (immediate after new data) • product quality links (same as for calChecker) • agree on mail alert mechanism

  25. calChecker HC monitor products/AB monitor

  26. the end …

  27. Other QC services within this context Agora: FAQ page for PSO-QC related issues refs: reference frames web page QC: tutorial pages, documentation

  28. avoid being under-sensitive: overlook that raw file is saturated or empty miss important properties = QC1 parameters avoid being too sensitive: have too narrow thresholds have unstable thresholds (3SIG) use not fully understood parameters select data sets (setups) rarely acquired ( trending impossible or uncertain) use same QC1 parameter in various HC plots ( multiple alarms) rule of thumb: every red score should have an explanation likely trigger an action [Excursion: The art of scoring … ] [E. Munch, Cry]

  29. 5. Operational impact check Health Check monitor • check scores • pick red ones • analyze problem • trigger actions • daytime astronomer: check completeness • take action for ylw/red boxes: e.g. trigger calibrations check product quality (AB monitor)

More Related