cwg9 and event display
Download
Skip this Video
Download Presentation
CWG9 and Event Display

Loading in 2 Seconds...

play fullscreen
1 / 22

CWG9 and Event Display - PowerPoint PPT Presentation


  • 105 Views
  • Uploaded on

CWG9 and Event Display. B. von Haller. 17.07.2014. CERN. CWG 9 in a nutshell. Started in May 2013 along with O 2 Group working on the Data Quality Monitoring the Quality Assurance the Visualization For Run 2 and Run 3 . CWG9 Members. Renu Bala Francesca Bellini Mihaela Gheata

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' CWG9 and Event Display' - marv


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
cwg9 and event display

CWG9 and Event Display

B. von Haller

17.07.2014

CERN

cwg 9 in a nutshell
CWG 9 in a nutshell
  • Started in May 2013 along with O2
  • Group working on
    • the Data Quality Monitoring
    • the Quality Assurance
    • the Visualization
  • For Run 2 and Run 3

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

cwg9 members
CWG9 Members
  • RenuBala
  • Francesca Bellini
  • MihaelaGheata
  • Lukasz Graczykowski
  • MalgorzataJanik
  • Andreas Morsch
  • MihaiNiculescu
  • JeremiNiedziela
  • Ankita Sharma
  • Maciej Szymanski
  • Barthélémy Von Haller

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

definitions of dqm and qa
Definitions of DQM and QA

Data Quality Monitoring & Quality Assurance

  • Feedback on the quality of data
  • Online (DQM)
    • Make sure to recordhigh quality data
    • Identify and solve problem(s) early
  • Offline (QA)
    • Make sure to analyze high quality data
    • Identify high quality runs
  • Involves
    • [Online gathering of data]
    • Analysis by user-defined algorithm
      • Production of monitoring objects such as histograms
      • Assessment of the quality of the data based on the objects
    • Storage of monitoring data
    • Visualization (+ human assessment of quality)

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

cwg9 tasks
CWG9 Tasks
  • Group focusing on the data Quality Assurance, online and offline, and the visualization of its results and the data itself
  • Run 3
    • Study how to monitor data efficiently and in plenty without interfering with the data taking
    • Discuss QA output and results, incremental QA and procedures to formalize if the results are acceptable or not
    • Determine the needs, and design the software, to access, visualize and interpret the results
    • Define and develop the software to visualize data, raw and reconstructed
    • Participate to the writing of the Technical Design Report and to the possible prototyping
  • Run 2
    • Production data taking period -> coordinate maintenance and improvements of software
    • Opportunity to test concepts and software for Run 3

Storage

Dataflow

Monitoring Object Generation

Automatic

QualityAssessment

Visualization

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

event display
Event Display

People currentlyworkingdirectly on ED

Maintenance and developmenttaken over by Warsaw group

  • JeremiNiedziela – PhDstudentat CERN
  • MaciejSzymański – Service Task on ED
  • Jakub Sala, Jakub Abelski, Adam Felis – Summerstudents
  • Warsaw group @ WUT for support
  • Mihai – Offline, former ED developer

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

event display1
Event Display

Offline recoEvent Display

Off.Reco

Off.

Reco

ED

ED

Run 2 tasks

  • Run 2 requirements
    • Split reconstruction and online ED
    • Allow multiple sources and unifyexistingEDs
    • Bookmarks
  • Under development by J. Niedziela and M. Szymanski
  • Workwill continue over the summer
  • Readyfor the commissioning and cosmics

HLT

HLT

HLT

HLTED

HLTED

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

future work
Future work
  • Event Display
    • Alternativeplatforms (mostly web and/or mobile)
      • SummerstudentsfromFaculty of Mathematics and Computer Science startedworking on it
      • Longer term collaboration on thistopicis possible in order to implement a full fledgeevent display
    • Possible use of earlyalfa components
      • Prototypingneededhere
  • DQM/QA
    • Prototype of merging QA objectsusingmap-reducewithZeroMQ
    • Web based ROOT objects browser

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

future work cwg10
Future work – CWG10

Control & Configuration

  • Control and Configuration of hierarchical distributed systems
    • Development of a zeroMQ-based prototype for configuration distribution and process control
    • Evaluation of performance based on number of nodes, hierarchical levels, processes and configuration size
  • Process Management
    • Evaluation of technologies for process execution in sandbox/container (Docker, etc.)
    • Evaluation of performance based on number of nodes, hierarchical levels, processes
  • Evaluation of ZooKeeper for possible usage in O2
    • Configuration management
    • Synchronization
    • Process control

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

conclusion
Conclusion
  • CWG9 is an important and dynamic working group in the O2 project
  • WUT is a key player in CWG9
    • In terms of people and responsibility
    • Crucial for the future of Visualization in ALICE
    • The ED is already profiting greatly from its involvement
  • There are many more tasks and opportunities for collaboration in CWGs

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

slide12

Backups

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

tdr and feasibility tests
TDR and feasibility tests
  • April 2014: First draft
  • August 2014: Submission
  • CWG9 participates to
    • Chapter 4 System architecture : Quality control and assessment
      • Table 1: detectors needs
      • Explain DQM/QA architecture and the choices made
      • Figure 1: DQM/QA architecture
      • Explain Event Display architecture and the choices made
      • Figure 2: Event Display architecture
    • Chapter 5 Technology survey, evaluations and prototypes
      • Mergers architecture and feasibility tests with 0MQ
      • Results of storage tests (e.g. DB technologies)
      • [Web gui architecture (ROOT JS lib + DABC)]
      • Event display design as tested (cf Run 2)
    • Chapter 6 System Design

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

past and current tasks
Past and Current tasks
  • Bring everyone aboard [done]
    • Make people aware of others’ work in the field
    • Give a picture of the current situation to everyone
  • Run 2
    • Event Display review and meetings, Warsaw involvement
      • Core refactoring
      • New features
      • Knowledge transfer

 Gain stability and support for Run 2

    • DQM/QA review and preparation
    • Proposal for the online reconstruction and calibration
  • Run 3
    • System requirements and system functionalities document [done]
    • Detectors needs survey
    • Definition of the future architecture and design
    • Prototypes and feasibility tests
    • Technical Design Report redaction

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

o 2 technical design report
O2 Technical Design Report

Schedule

  • October ‘13:
    • Define table of content
    • Establish editorial board
  • December ‘13:
    • System Requirement Document
    • High-level dataflow model
    • Computing platforms benchmarks
    • Networking benchmark
  • June ‘14
    • Software framework architecture
  • Sep ‘14
    • TDR

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

cwg9 tdr timeline
CWG9 TDR Timeline

Proposal

  • January 14
    • Define list of tables and figures
    • Draft of the architecture of the system
    • Launch subsystems exhaustive survey

 Submit list of tables and figures to TDR EC

  • February 14
    • Draft tables and figures
    • Skeleton of 4.2.4 and 5.6

 Submit skeleton to TDR EC

  • March 14
    • Finalize tables and figures, including subsystems input
    • Iterate on text using input of TDR EC

 Submit text and final tables and figures to TDR EC

  • April 14
    • Finalize text
    • Submit final text to TDR EC
  • May 14
    • Iterate over our sections using CWGs input
    • Review work of other CWGs (especially what concerns us!)

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

tdr skeleton tables figures
TDR – skeleton, tables & figures

Chapter 4 System architecture

  • 4.2 Data processing and computing model
    • 4.2.4 DQM and QA
      • « Quality control and assessment »
      • Table 1: detectors needs
      • Explain architecture
      • Figure 1: architecture
      • Explain the choices
      • Figure 2: Event display arch.

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

tdr skeleton tables figures1
TDR – skeleton, tables & figures

Chapter 5 Technology survey, evaluations and prototypes

  • 5.6 DQM and QA
    • « Quality control and assessment »
    • Technologies and design choices available concerning key points of our system
      • Storage
      • Access to results worldwide
      • Event display
    • Feasability tests & prototypes
      • Table 1: results of storage tests (e.g. DB technologies)
      • Figure 1: Web gui architecture (ROOT JS lib + DABC)
      • Figure 2: Event display design as tested (cf Run 2)

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

subsystems survey
Subsystems survey
  • What “tasks” (name it agents or algorithms if you prefer) will your subsystem need in Run 3 ?
  • For each of these task or group of tasks, tell us
    • Whether it already exist today and if so what is its performance.
    • What is the expected performance of such a task in Run 3.
    • How many plots are expected to be produced (for the shifter and for the experts).
    • Percentage of events needed to carry out the task online (minimum, optimal).
    • What is the input ? i.e. at which stage will it run ?
    • How fast the response has to be taken into account in the data flow ?
    • Whether the DQM/QA results have to become persistent and for how long ?
  • What does “Calibration QA” mean to you ?

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

survey status
Survey status

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

current and short term work
Current and short term work
  • Run 2 : Prepare a proposal for the QA in relation with the online reco, calibration and monitoring (QA tasks, validation, bridge to DQM, …)
    • Online Calibration
      • Mainly for TPC
      • Many open questions on
        • Requirements
        • General architecture
        • Implementation
        • CDB
      • Working on a proposal to meet requirements while minimizing work
        • Use analysis QA train within HLT for reco monitoring
        • Use analysis QA train within HLT for calib monitoring
        • Use AMORE for raw data monitoring
        • Use AMORE infrastructure for storage and visualization
  • Run 2 – Review and preparation
    • Detectors « interviews »
    • (DATE Monitoring update)
  • Run 3 : Prepare requirements of the future system following CWG1 input
  • For the TDR (2014)
    • Define requirements and general architecture and features of the QA-DQM-Viz for Run 3
    • Write it
  • Event Display
    • Decentralized model under implementation
      • Better stability
      • Split GUI and reconstruction
      • Possibility to switch between offline and HLT reco
    • Bookmarks (for users and for PR)
    • Involvement of the Warsaw group
    • Implemented by the end of 2013

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

slide22

HCDB

OCDB

DCS, GRP

QC

(sort of)

Reco

Selection/filters

Calib

Calibparam

Raw

ESDs

QC

QC

Data on Castor

Reading Writing Producing

DBsdata procedures

PHYSICS Run

B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

ad