Cwg9 and event display
This presentation is the property of its rightful owner.
Sponsored Links
1 / 22

CWG9 and Event Display PowerPoint PPT Presentation


  • 85 Views
  • Uploaded on
  • Presentation posted in: General

CWG9 and Event Display. B. von Haller. 17.07.2014. CERN. CWG 9 in a nutshell. Started in May 2013 along with O 2 Group working on the Data Quality Monitoring the Quality Assurance the Visualization For Run 2 and Run 3 . CWG9 Members. Renu Bala Francesca Bellini Mihaela Gheata

Download Presentation

CWG9 and Event Display

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Cwg9 and event display

CWG9 and Event Display

B. von Haller

17.07.2014

CERN


Cwg 9 in a nutshell

CWG 9 in a nutshell

  • Started in May 2013 along with O2

  • Group working on

    • the Data Quality Monitoring

    • the Quality Assurance

    • the Visualization

  • For Run 2 and Run 3

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014


Cwg9 members

CWG9 Members

  • RenuBala

  • Francesca Bellini

  • MihaelaGheata

  • Lukasz Graczykowski

  • MalgorzataJanik

  • Andreas Morsch

  • MihaiNiculescu

  • JeremiNiedziela

  • Ankita Sharma

  • Maciej Szymanski

  • Barthélémy Von Haller

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014


Definitions of dqm and qa

Definitions of DQM and QA

Data Quality Monitoring & Quality Assurance

  • Feedback on the quality of data

  • Online (DQM)

    • Make sure to recordhigh quality data

    • Identify and solve problem(s) early

  • Offline (QA)

    • Make sure to analyze high quality data

    • Identify high quality runs

  • Involves

    • [Online gathering of data]

    • Analysis by user-defined algorithm

      • Production of monitoring objects such as histograms

      • Assessment of the quality of the data based on the objects

    • Storage of monitoring data

    • Visualization (+ human assessment of quality)

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014


Cwg9 tasks

CWG9 Tasks

  • Group focusing on the data Quality Assurance, online and offline, and the visualization of its results and the data itself

  • Run 3

    • Study how to monitor data efficiently and in plenty without interfering with the data taking

    • Discuss QA output and results, incremental QA and procedures to formalize if the results are acceptable or not

    • Determine the needs, and design the software, to access, visualize and interpret the results

    • Define and develop the software to visualize data, raw and reconstructed

    • Participate to the writing of the Technical Design Report and to the possible prototyping

  • Run 2

    • Production data taking period -> coordinate maintenance and improvements of software

    • Opportunity to test concepts and software for Run 3

Storage

Dataflow

Monitoring Object Generation

Automatic

QualityAssessment

Visualization

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014


Event display

Event Display

People currentlyworkingdirectly on ED

Maintenance and developmenttaken over by Warsaw group

  • JeremiNiedziela – PhDstudentat CERN

  • MaciejSzymański – Service Task on ED

  • Jakub Sala, Jakub Abelski, Adam Felis – Summerstudents

  • Warsaw group @ WUT for support

  • Mihai – Offline, former ED developer

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014


Event display1

Event Display

Offline recoEvent Display

Off.Reco

Off.

Reco

ED

ED

Run 2 tasks

  • Run 2 requirements

    • Split reconstruction and online ED

    • Allow multiple sources and unifyexistingEDs

    • Bookmarks

  • Under development by J. Niedziela and M. Szymanski

  • Workwill continue over the summer

  • Readyfor the commissioning and cosmics

HLT

HLT

HLT

HLTED

HLTED

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014


Future work

Future work

  • Event Display

    • Alternativeplatforms (mostly web and/or mobile)

      • SummerstudentsfromFaculty of Mathematics and Computer Science startedworking on it

      • Longer term collaboration on thistopicis possible in order to implement a full fledgeevent display

    • Possible use of earlyalfa components

      • Prototypingneededhere

  • DQM/QA

    • Prototype of merging QA objectsusingmap-reducewithZeroMQ

    • Web based ROOT objects browser

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014


Future work cwg10

Future work – CWG10

Control & Configuration

  • Control and Configuration of hierarchical distributed systems

    • Development of a zeroMQ-based prototype for configuration distribution and process control

    • Evaluation of performance based on number of nodes, hierarchical levels, processes and configuration size

  • Process Management

    • Evaluation of technologies for process execution in sandbox/container (Docker, etc.)

    • Evaluation of performance based on number of nodes, hierarchical levels, processes

  • Evaluation of ZooKeeper for possible usage in O2

    • Configuration management

    • Synchronization

    • Process control

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014


Conclusion

Conclusion

  • CWG9 is an important and dynamic working group in the O2 project

  • WUT is a key player in CWG9

    • In terms of people and responsibility

    • Crucial for the future of Visualization in ALICE

    • The ED is already profiting greatly from its involvement

  • There are many more tasks and opportunities for collaboration in CWGs

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014


Cwg9 and event display

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014


Cwg9 and event display

Backups

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014


Tdr and feasibility tests

TDR and feasibility tests

  • April 2014: First draft

  • August 2014: Submission

  • CWG9 participates to

    • Chapter 4 System architecture : Quality control and assessment

      • Table 1: detectors needs

      • Explain DQM/QA architecture and the choices made

      • Figure 1: DQM/QA architecture

      • Explain Event Display architecture and the choices made

      • Figure 2: Event Display architecture

    • Chapter 5 Technology survey, evaluations and prototypes

      • Mergers architecture and feasibility tests with 0MQ

      • Results of storage tests (e.g. DB technologies)

      • [Web gui architecture (ROOT JS lib + DABC)]

      • Event display design as tested (cf Run 2)

    • Chapter 6 System Design

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014


Past and current tasks

Past and Current tasks

  • Bring everyone aboard [done]

    • Make people aware of others’ work in the field

    • Give a picture of the current situation to everyone

  • Run 2

    • Event Display review and meetings, Warsaw involvement

      • Core refactoring

      • New features

      • Knowledge transfer

         Gain stability and support for Run 2

    • DQM/QA review and preparation

    • Proposal for the online reconstruction and calibration

  • Run 3

    • System requirements and system functionalities document [done]

    • Detectors needs survey

    • Definition of the future architecture and design

    • Prototypes and feasibility tests

    • Technical Design Report redaction

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014


O 2 technical design report

O2 Technical Design Report

Schedule

  • October ‘13:

    • Define table of content

    • Establish editorial board

  • December ‘13:

    • System Requirement Document

    • High-level dataflow model

    • Computing platforms benchmarks

    • Networking benchmark

  • June ‘14

    • Software framework architecture

  • Sep ‘14

    • TDR

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014


Cwg9 tdr timeline

CWG9 TDR Timeline

Proposal

  • January 14

    • Define list of tables and figures

    • Draft of the architecture of the system

    • Launch subsystems exhaustive survey

       Submit list of tables and figures to TDR EC

  • February 14

    • Draft tables and figures

    • Skeleton of 4.2.4 and 5.6

       Submit skeleton to TDR EC

  • March 14

    • Finalize tables and figures, including subsystems input

    • Iterate on text using input of TDR EC

       Submit text and final tables and figures to TDR EC

  • April 14

    • Finalize text

    • Submit final text to TDR EC

  • May 14

    • Iterate over our sections using CWGs input

    • Review work of other CWGs (especially what concerns us!)

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014


Tdr skeleton tables figures

TDR – skeleton, tables & figures

Chapter 4 System architecture

  • 4.2 Data processing and computing model

    • 4.2.4 DQM and QA

      • « Quality control and assessment »

      • Table 1: detectors needs

      • Explain architecture

      • Figure 1: architecture

      • Explain the choices

      • Figure 2: Event display arch.

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014


Tdr skeleton tables figures1

TDR – skeleton, tables & figures

Chapter 5 Technology survey, evaluations and prototypes

  • 5.6 DQM and QA

    • « Quality control and assessment »

    • Technologies and design choices available concerning key points of our system

      • Storage

      • Access to results worldwide

      • Event display

    • Feasability tests & prototypes

      • Table 1: results of storage tests (e.g. DB technologies)

      • Figure 1: Web gui architecture (ROOT JS lib + DABC)

      • Figure 2: Event display design as tested (cf Run 2)

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014


Subsystems survey

Subsystems survey

  • What “tasks” (name it agents or algorithms if you prefer) will your subsystem need in Run 3 ?

  • For each of these task or group of tasks, tell us

    • Whether it already exist today and if so what is its performance.

    • What is the expected performance of such a task in Run 3.

    • How many plots are expected to be produced (for the shifter and for the experts).

    • Percentage of events needed to carry out the task online (minimum, optimal).

    • What is the input ? i.e. at which stage will it run ?

    • How fast the response has to be taken into account in the data flow ?

    • Whether the DQM/QA results have to become persistent and for how long ?

  • What does “Calibration QA” mean to you ?

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014


Survey status

Survey status

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014


Current and short term work

Current and short term work

  • Run 2 : Prepare a proposal for the QA in relation with the online reco, calibration and monitoring (QA tasks, validation, bridge to DQM, …)

    • Online Calibration

      • Mainly for TPC

      • Many open questions on

        • Requirements

        • General architecture

        • Implementation

        • CDB

      • Working on a proposal to meet requirements while minimizing work

        • Use analysis QA train within HLT for reco monitoring

        • Use analysis QA train within HLT for calib monitoring

        • Use AMORE for raw data monitoring

        • Use AMORE infrastructure for storage and visualization

  • Run 2 – Review and preparation

    • Detectors « interviews »

    • (DATE Monitoring update)

  • Run 3 : Prepare requirements of the future system following CWG1 input

  • For the TDR (2014)

    • Define requirements and general architecture and features of the QA-DQM-Viz for Run 3

    • Write it

  • Event Display

    • Decentralized model under implementation

      • Better stability

      • Split GUI and reconstruction

      • Possibility to switch between offline and HLT reco

    • Bookmarks (for users and for PR)

    • Involvement of the Warsaw group

    • Implemented by the end of 2013

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014


Cwg9 and event display

HCDB

OCDB

DCS, GRP

QC

(sort of)

Reco

Selection/filters

Calib

Calibparam

Raw

ESDs

QC

QC

Data on Castor

Reading Writing Producing

DBsdata procedures

PHYSICS Run

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014


  • Login