DQM Services: Status and Plans
1 / 19

DQM Services: Status and Plans - PowerPoint PPT Presentation

  • Uploaded on

DQM Services: Status and Plans. Christos Leonidopoulos CERN − PH for the EvF/DQM group. DQM Workshop (CMS week) – 28 February 2007. “What is DQM?”. DQM is: A monitoring project originally created for the Event Filter Farm

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' DQM Services: Status and Plans' - preston-nunez

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Dqm services status and plans

DQM Services: Status and Plans

  • Christos Leonidopoulos

  • CERN − PH

    • for the

    • EvF/DQM group

DQM Workshop (CMS week) – 28 February 2007

What is dqm
“What is DQM?”

  • DQM is:

    • A monitoring project originally created for the Event Filter Farm

    • A set of monitoring tools that can be used either on- or off-line:

      • Tree-like structures w/ histograms, profiles, scalars, strings

      • Quality tests that produce warnings, errors, alarms

      • Visualization tools

      • Transfer of monitoring information to remote nodes

    • A wrapper around ROOT objects that offers the above functionality

  • DQM is not:

    • A validation suite (just like ROOT is not a physics analysis)

    • A set of “out-of-the-box” templates that are ready to use

What can dqm do for me
“What can DQM do for me?”

  • Main reasons for using DQM:

    • Transfer of monitoring information over network

      • Creation/updates of histograms/profiles/etc transparent to user

      • Suitable for distributed systems

    • Library with quality tests, alarm generation/storage/distribution

      • Often updated w/ “algorithm donations” from users

    • Bonus: Extended usability by CMS helps w/ bug discovering/fixing

      (EvF, OnSel, all subdetector groups, beam-tests, MTCC, …)

  • Ongoing development over last 2 years:

  • Emilio Meschi (original design, run-control, mentoring)

  • CL (core functionality, rules & alarms library, tech support)

  • Dimitrios Tsirigkas (Web interface), Giulio Eulisse, Andrea Carboni (Qt-GUI)

  • Ilaria Segoni (specialized clients, coordination with detector groups)

  • Kurt Biery (DQM data & Storage Manager)

Dqm on line mode
DQM: On-line mode

Monitoring producers

See Emilio’s slides

Monitoring consumers

  • On-line operating mode:

  • Monitoring producers → server → monitoring consumers

  • Useful when:

  • Need to access info from multiple monitoring producers

  • Need clear separation of production (sources) and processing (client)

  • of monitoring information (e.g. Filter Farm)

Dqm on line mode 2
DQM: On-line mode #2

CERN Network

“Super Clients”


Non-CERN Network

See Emilio’s slides

What if you want something simpler?

Dqm off line standalone mode
DQM: Off-line (standalone) mode

Single program/node

Monitoring producer

Monitoring consumer

  • Merge source & client; eliminate collector

  • Useful when one wants to:

  • First access/produce monitoring information

  • Then process it

  • In the same program!

  • Equivalent to “off-line” or “validation” mode

  • Equivalent to a “standalone” ROOT macro

Dqm from a user s perspective
DQM from a user’s perspective


Possible today, but not

managed by DQM

“System ok”


  • Create monitoring information

  • Process: Visualize, analyze,

  • determine quality, store

  • (à la ROOT or kumac macro)






  • “Comparison-to-reference”

  • Collation of similar objects

  • Configuration

  • Reference objects

  • Historic plots

  • Etc…

Core features 1 the basics
Core Features #1: The Basics




  • Support for all the “usual stuff”: static and dynamic sets of objects

  • 1,2,3-D histograms, 1,2-D profiles, integers, floats, strings

  • (ROOT objects behind the scenes)

  • Support for tree-like directory structures

Core features 2 quality tests
Core Features #2: Quality Tests

  • Library with “rules” for assigning “quality” value to tests

    • Comparison to reference (χ2, Kolmogorov tests)

    • Contents within range ([xmin, xmax], ([ymin, ymax])

    • Exact match

    • Mean of (e.g. gaussian) distribution “near” expected value

    • Check for dead channels

    • Check for noisy channels

    • Mean & RMS of bins within allowed range

    • Quality tests run automatically by DQM

    • Results “attached” to Monitor Elements

    • Can be saved in ROOT file for “off-line” use

  • “Alarm” library

  • Warnings & error messages propagating to all clients downstream

Core features 3 output input
Core Features #3: Output/Input

  • Support for ROOT-files

  • Create and save ROOT-files w/ (any subset of ) monitoring structure

  • on the fly (including results of quality tests)

Can also read back file (“playback”),

create MonitorElements, “attach”

quality test results (1_2_0 feature)

Dqm iguana qt gui 2
DQM Iguana Qt-GUI #2

Cosmic test data for

calorimeter detector

(reading from file)

Recent highlights
Recent highlights

  • Added the ability to group MEs from different folders/nodes: “Tags”

    • Similar to google mail’s “labels”

  • Improved interface for accessing MEs

  • Can open ROOT files in “playback” mode

  • Fit functions/results can be transferred to remote nodes

  • Improvements in code stability

Historic plots dqm
“Historic” Plots & DQM


Open items to do lists savannah
Open items & to-do lists (savannah)

DQM support requests:

DQM bugs:

DQM tasks:

Dqm documentation
DQM documentation

  • Release notes for latest developments

  • http://cmsevf.web.cern.ch/cmsevf/DQM_doc/Release_Notes_v033.txt

  • Instructions for getting started

  • http://cmsevf.web.cern.ch/cmsevf/DQM_doc/DQM_instructions.html

  • Archive with presentations from DQM group:

  • http://cmsevf.web.cern.ch/cmsevf/DQMMeetings.html

  • DQM status for subdetector groups

  • https://twiki.cern.ch/twiki/bin/view/CMS/DQMSubDetectors

  • Twiki page with discussions, notes, manuals, links

  • https://twiki.cern.ch/twiki/bin/view/CMS/DQMInfrastructure