Online and Data Quality Monitoring
This presentation is the property of its rightful owner.
Sponsored Links
1 / 26

Online and Data Quality Monitoring PowerPoint PPT Presentation


  • 65 Views
  • Uploaded on
  • Presentation posted in: General

Online and Data Quality Monitoring. Eduardo Rodrigues University of Glasgow LHCb VELO Software meeting, CERN, 6 April 2009. Online coordination : Kurt Rinnert Data Quality coordination : Eduardo Rodrigues

Download Presentation

Online and Data Quality Monitoring

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Online and data quality monitoring

Online and Data Quality Monitoring

Eduardo Rodrigues

University of Glasgow

LHCb VELO Software meeting, CERN, 6 April 2009

Online coordination: Kurt Rinnert

Data Quality coordination: Eduardo Rodrigues

Contributors: Eduardo (cluster monitoring), Sadia (track monitoring), Marco & Silvia (vertex monitoring), Marco (alignment monitoring), Malcolm (monitoring occupancy in HLT), Sara (Noisy & Dead strips), Abdi (noise vs voltage), James & Abdi (Landau fitting), Grant (Gain monitoring), Tomasz (TELL1 algorithm monitoring), Paula.


Online and data quality monitoring

Monitoring: outline

  • A lot of recent work and progress – thanks everyone !

  • Monitoring algorithms:

    - Most algorithms fairly stable

    - A couple of new kinds on the block: e.g. gain monit., “expert” noise monit.

  • Monitoring analysis scripts:

    - area with main effort  a lot of new tools available

  • Wiki pages with documentation and HowTo’s are also being updated

Eduardo Rodrigues


Online and data quality monitoring

But we’re not quite at this stage …

Eduardo Rodrigues


Online and data quality monitoring

Monitoring: algorithms packages

“VELO moni group”

VeloRecMonitors

  • Package for “high-level” (= ZS) data

  • 6 monitoring algorithms in CVS- 3 related to cluster monitoring: “standard”, “expert”, for HLT

    - 1 for time-alignment studies (sampling)

    - 1 for track-based monitoring

    - 1 relevant to monitoring of closing

  • Alignment monitoring part of track-based monit. algorithm

  • Vertex-based monitoring taken from Tr/TrackMonitors

    VeloDataMonitor

  • Package for NZS data

  • Several algorithms for error banks / noise / pedestals / TELL1s / … monitoring

  • New algorithm for gain monitoring

Eduardo Rodrigues


Online and data quality monitoring

Monitoring: scripts package VetraScripts

“VELO moni group”

  • Centrally collect scripts, macros, Python modules for monitoring and data quality analyses

  • Area on critical path a couple of months ago

  • Glad to see so much effort went into this recently: - 37 macros/scripts in CVS!(we may even need some clean-up due to large overlaps in code)

  • If you still have handy private scripts then let us know … ;-)

  • And please commit your code, even if not in its “final” version(remember it’s a CVS repository)

  • http://isscvs.cern.ch/cgi-bin/cvsweb.cgi/Velo/VetraScripts/?cvsroot=lhcb

Eduardo Rodrigues


Online and data quality monitoring

Online presenter (1/2)

Kurt

  • Monitors ZS as well as NZS data

  • 2 monitoring tasks exist: - VeloDAQMon: clusters, noise, TAE, errors - VeloBeamMon: beam position for closing - track-based & alignment monitoring will either be added to VeloDAQMon or will get its own task

  • VELO online monitoring depends on Vetra project - has to follow faster release cycle of Online project by patching the current installation :S

Eduardo Rodrigues


Online and data quality monitoring

Online presenter (2/2)

  • Now much easier to efficiently declare several pages (using new handy tools from Nicola for manipulating HistDB)

  • Available pages at present: default cluster monit., TAE monit. and per TELL1 noise pages

  • Implementation of pages is not yet complete. But it’s much easier compared to the past

  • Page layouts and histograms display options will evolve once we have user feedback

Eduardo Rodrigues


Online and data quality monitoring

Online presenter: snapshots (1/2)

Clustermonitoring

Eduardo Rodrigues


Online and data quality monitoring

Online presenter: snapshots (2/2)

Noisemonitoring

Eduardo Rodrigues


Online and data quality monitoring

Online presenter: cluster-based monitoring

1 page inpresenter

OfflinePresenter

Yes

Yes

Yes

Yes

Yes

No

Yes

Yes

Yes

  • # of clusters per event

  • # of strips per cluster

  • Cluster ADC value (all sensors)

  • Cluster ADC value (R sensors)

  • Cluster ADC value (f sensors)

  • # of clusters for sensor “i”(1 plot produced per sensor if nClus>0)

  • # of strips per cluster versus sensor

  • Cluster ADC value versus sensor

  • Active chip links versus sensor

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Eduardo Rodrigues


Online and data quality monitoring

Online presenter: track-based monitoring

Eduardo Rodrigues


Online and data quality monitoring

Online presenter: track-based monitoring

Module mismatch

Biased residuals vs sensor

Pseudo-efficiency

Local f coordinate

Presenter layout

 pop-up pages when clicking on each histo:

Original histo + “Used sensors” histo

The 8 histos related with residuals mean and s for R and f and A- and C-side

(clicking disabled)

Original histo + eta & f histos(global coords.)

Eduardo Rodrigues


Online and data quality monitoring

Online presenter: vertex monitoring

Presenter layout

PV left-right delta x

PV position x

PV position y

PV position z

  • TrackVertexMonitor in Tr/TrackMonitors

  • (Need to include this algo. In the Vetra monitoring sequence)

Eduardo Rodrigues


Online and data quality monitoring

New VeloExpertClusterMonitor algorithm

Abdi, James

  • New algo in Velo/VeloRecMonitors for “expert” cluster monitoring

  • Creates histograms of the ADCs for each sensor

  • Makes plots for raw and on-track clusters

  • Splits sensors into different regions of r, q and f

  • Histograms are viewed by new dispCluster.py script (in Velo/VetraScripts)

Eduardo Rodrigues


Online and data quality monitoring

New dispCluster.py script (1/2)

Abdi, James

  • Displays plots of the ADCs of clusters and the number of strips in a cluster

  • ADCs are fitted with a Landau-Gaussian convolution to find MPV and FWHM

Eduardo Rodrigues


Online and data quality monitoring

New dispCluster.py script (2/2)

  • Also plots MPV vs sensor number

  • ADC values are angle corrected in the case of on-track clusters

Eduardo Rodrigues


Online and data quality monitoring

Gain calibration (1/3)

Grant, Kazu

Can the “full header swing” (FHS) be used for this purpose? As long as low/high header distributions are distinct (like plot below) this is a clearly-defined and stable quantity.

Monitor the gain of links:

Initially through by looking at link headers

Next will look at test-pulse data

Goal is to calibrate links:

Can we somehow normalize link gains to some standard value?

Would be useful for eventual dE/dx measurements

Full Header Swing (FHS)

“Header-Low” distribution

Eduardo Rodrigues

“Header-High” distribution


Online and data quality monitoring

Gain calibration (2/3)

Code to make these plots works with Vetra v7r0, and is in CVS of Velo/VeloDataMonitor, with ROOT macros in Velo/VetraScripts.

Can be used to quickly spot bad links

ROOT code in Velo/VetraScripts is able to plot:

FHS, FHS scaled by link noise

Average header value, average header value scaled by noise

RMS of HH, HL distributions

Link headers with unusually high (>600) and low (<400) ADC counts

Eduardo Rodrigues


Online and data quality monitoring

Gain calibration (3/3)

Unusually high/low ADC counts:

Avg headers:

Link header distributions:

RMS of HH/HL distributions:

Eduardo Rodrigues


Online and data quality monitoring

VELO monitoring & analysis GUI

Eduardo, Karol, Kurt, Paula

  • Consensus on how to run monitoring/analysis jobs: 1) run a PVSS recipe 2) “on button” run of a corresponding Vetra task 3) GUI to ease the display/analysis of histograms produced

  • Status of matters 1 month ago:- PVSS recipes exist since long (renamed recently)

  • In the meantime:- PVSS button to run Vetra task defined by Karol

    - New VELO monitoring GUI- has received many contributions ever since

Eduardo Rodrigues


Online and data quality monitoring

VeloMoniGUI: snapshots (1/2)

Eduardo Rodrigues


Online and data quality monitoring

VeloMoniGUI: snapshots (2/2)

Eduardo Rodrigues


Online and data quality monitoring

Next on the TODO list (1/2)

  • Update on Vetra/Brunel monitoring sequences: - Not all algorithms set to run in our monitoring sequences - On-/off-line specific options to be checked(keep in mind that one needs .opts files online!)

  • Installation of Vetra and monitoring packages for shifters:- Regularly updated - But needs better handling: e.g., never change anything locally that is not in CVS or immediately committed to CVS ;-) - Some issues for the online version?

  • Discussion of shifters tasks: - we need to start thinking about our monitoring procedures at the pit - important for VELO shifters … and you’ve just learned this morning that we will have shifts very soon to start the training

Eduardo Rodrigues


Online and data quality monitoring

Next on the TODO list (2/2)

  • We need to collect reference plots!- If you haven’t done so yet, please get in touch with me to get this sorted by the end of this week

  • Tutorials:- We recently had (on the 13th March) a first Vetra tutorial - More tutorials are foreseen, with special emphasis on monitoring

  • Documentation – my usual blabla: - Update twiki, remove old & obsolete pages, converge on overlapping pages, etc. - There’s been some activity. To be continued …

Eduardo Rodrigues


Online and data quality monitoring

Outlook (apologies if I’m not aware of very latest updates)

Eduardo Rodrigues


Online and data quality monitoring

Any comments?

Eduardo Rodrigues


  • Login