1 / 25

WEB DATA QUALITY ANALYSIS

This presentation outlines the methods of data quality checking utilized at CERN, specifically through the frameworks of SICBCHK and WDQA. It discusses the production and analysis of test data through Monte Carlo (MC) and Data Stream (DST) processes, detailing how subdetector groups contribute detailed checking routines. The results of this analysis, including current data trends and the need for larger sample sizes, will be highlighted. Additionally, tools and scripts developed to streamline the data quality checking process will be introduced.

malia
Download Presentation

WEB DATA QUALITY ANALYSIS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. WEB DATA QUALITY ANALYSIS Jonathan Levell Jonathan.Levell@cern.ch (Summer Student) • Working with: • Eric van Herwijnen (EvH) • Agnieszka Jacholkowska (AJ)

  2. TALK OUTLINE • Introduction • How is the Data Quality Checking done? • What is SICBCHK? • What is WDQA? • Results • Typical Data • CERN/MAP • Summary

  3. AIMS • Check Data Quality (DQ): • During MC and DST production of test data • After each detector modification How • Produce a standalone DQ program which calls • checking routines supplied by subdetector groups • Provide WDQA package to display and compare • sets of histograms for each subdetector

  4. SICBMC RAWH SICBDST DST1/DST2 SICBCHK NTUPLE WDQA GIFS PostScript HTML

  5. SICBCHK • Fortran Program • Separate from SICBMC/SICBDST • (package called: SICBCHK) • Split into branches for each subdetector • Branches maintained individually • Latest version installed in September

  6. SICBCHK * = JL leaves on 15th September

  7. SICBCHK SICBCHK.F UGINIT.F CHKRUN.F UGLAST.F Initialisation Tidy Up For each event CHKTRIG.F SUANAL.F Subdetector Routines SUCHECK.F Miscellaneous Checks

  8. W.D.Q.A. • Package controlled by .lmac scripts • similar to PAW’s .kumac scripts • Small Interpreter written in C++ • Based on ROOT • Converts ntuple files to ROOT files • LMAC scripts can call ROOT macros • Easy to use • Manual currently being written • Eventually details in Computing meeting

  9. An Example .LMAC script // Conversion to ROOT files. hbookin ~/work/data/<<eventtype>>.hbook file 1 hbookin ~/work/data2/<<eventtype>>.hbook opt nowait outputdir velo ps velo.ps outputdir velo/gifs settitle Event Type: <<eventtype>> (Vertex) zone 1 2 set style default plot 1001 set ymax 250 set fillcolor 5 file 1 overlay 1001 // Call standard ROOT macro rootpad fit2gauss.c gifs page1 wait ps close close file 1 close

  10. Are the differences significant?

  11. Conclusions • Bigger sample required before real conclusions • can be drawn. • Differences currently at 3s level • Larger sample (500 events) will be requested, • if possible with same random seeds. More Generally... SICBCHK/WDQA still require more work but a new version will be released in mid-September

More Related