1 / 17

Scoring: A Novel Approach Towards Automated Certification of Pipeline Products

Explore a groundbreaking solution for scoring and certifying pipeline products in massive data production at the European Southern Observatory. Learn how automatic procedures help measure, compare, and assess quality, flag outliers, and ensure instrument health monitoring.

freddiej
Download Presentation

Scoring: A Novel Approach Towards Automated Certification of Pipeline Products

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Scoring: a novel approach towards automated certification of pipeline products Reinhard Hanuschik, Wolfgang Hummel, Mark Neeser, Burkhard WolffData Processing and Quality Control Group European Southern Observatory

  2. Complexity and mass production • VLT: 11 instruments • VLTI: 2 instruments • 2009/10: 2 survey telescopes • roughly 50 modes • about 150 data types • most of them pipeline-supported

  3. Complexity and mass production • each data type or instrument component controlled by QC1 parameters • metrics for instrument performance extracted by data pipelines • stored in database • data frequency varies on timescales between a day and a semester

  4. Complexity and mass production two challenges … • … the complexity challenge!

  5. avg: 30 GB/night OMEGACAM: VIRCAM: 400 GB/night Complexity and mass production • … the volume challenge! current VLT instruments:

  6. The solution: The solution: scoring QC process: • measure quality • compare quality • assess quality: score and certify scoring: well-established outside astronomy …

  7. Scoring: the solution • measure quality: done by automatic procedures • pipelines • other QC procedures • implemented for CALIB data • progress on SCIENCE data

  8. Scoring: the solution • compare quality: trending • first step towards assessment: • same/different behaviour as others?

  9. Scoring: the solution • assess quality: scoring • compare new result to trending • pick out outliers and flag them

  10. Scoring: the solution saturation noise • required: thresholds • statistical thresholds: e.g. +/- 3 sigmas+ independent from specs- sensitivity variablecontrol charts • specified thresholds: e.g. lower/upper level+ stable thresholds- requires careful configuration and thinking

  11. Scoring of pipeline products • each new pipeline product is scored • most relevant QC1 parameters:compare to trending, score as • OK: 0 • NOK: 1 (outlier) • count scores, assign total score, e.g.

  12. Scoring of pipeline products score report:

  13. Scoring of pipeline products score reports for multi-detector instruments (CRIRES: N=4) … detailed information on demand!

  14. cal file1 scores cal file2 scores FLAT instrument scores cal LAMPs ARC Scoring of instrument health • all scores go into database • re-arrange them by instrument properties • detector health • calibration lamp performance • system efficiency score-based instrument QC

  15. Scoring of instrument health • example: detector properties for GIRAFFE instead of …

  16. Scoring of instrument health • and finally: www.eso.org/HEALTH

  17. Conclusions • scoring: automatic flagging of outliers • useful to auto-certify pipeline products • quick-look monitoring of instrument health • powerful concept to handle large and complex data sets • hierarchical approach: from global overview (instrument score) to QC1 parameter per product and detector (6 levels)

More Related