parallel alice offline reconstruction with proof n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Parallel ALICE offline reconstruction with PROOF PowerPoint Presentation
Download Presentation
Parallel ALICE offline reconstruction with PROOF

Loading in 2 Seconds...

play fullscreen
1 / 21

Parallel ALICE offline reconstruction with PROOF - PowerPoint PPT Presentation


  • 141 Views
  • Uploaded on

Parallel ALICE offline reconstruction with PROOF. C. Cheshkov, P. Hristov on behalf of ALICE Core Offline Team 24/03/2009 CHEP09. Outline. Introduction to ALICE raw data and reconstruction Parallel reconstruction with PROOF Motivation Design and implementation Performance on ALICE CAF

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Parallel ALICE offline reconstruction with PROOF' - kirk-may


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
parallel alice offline reconstruction with proof
Parallel ALICE offline reconstruction with PROOF

C. Cheshkov, P. Hristov

on behalf of ALICE Core Offline Team

24/03/2009

CHEP09

outline
Outline
  • Introduction to ALICE raw data and reconstruction
  • Parallel reconstruction with PROOF
    • Motivation
    • Design and implementation
    • Performance on ALICE CAF
  • Conclusions & Outlook

Parallel ALICE Offline Reconstruction with PROOF

alice offline reconstruction
ALICE Offline Reconstruction
  • Raw data from ALICE detector
  • Local reconstruction (clusterization)
  • Vertex finding
  • Tracking
  • Particle identification
  • Reconstructed data -> Event Summary Data (ESD) ROOT tree

Parallel ALICE Offline Reconstruction with PROOF

alice offline reconstruction1
ALICE Offline Reconstruction
  • ALICE raw data:
    • Events are stored as entries in ROOT tree
    • Loading of various detectors data on demand
    • From few to ~106 events in one raw-data chunk
    • Size: from few KBs up to ~100 MB/ev. depending on active detectors, collision type, luminosity, etc.
  • Reconstruction:
    • Involves sophisticated algorithms
    • Recons. time varies from fraction of s to ~200s
  • Event Summary Data (ESD):
    • Contains all information relevant for physics analysis
    • Size: at least one order of magnitude smaller than raw-data size

Parallel ALICE Offline Reconstruction with PROOF

alice offline reconstruction2
ALICE Offline Reconstruction

From low to high track multiplicities

Parallel ALICE Offline Reconstruction with PROOF

parallel reconstruction motivation
Parallel Reconstruction: Motivation
  • Fast feedback from reconstruction
    • Understand ALICE detector and reconstruction software
    • Debug, tune and optimize reconstruction code
  • Not a replacement for central ALICE GRID (AliEn) based reconstruction

Parallel ALICE Offline Reconstruction with PROOF

parallel reconstruction design
Parallel Reconstruction: Design

CLIENT

WORKERS

ALICE GRID

Raw data

Conditions data

Initialization

MASTER

List of raw-data files

Conditions data

Reconstruction options

Broadcasting

Input data

To slaves

Reconstruction

Access to event packet

from raw-data files

Event-processing

Packetizer

ESD file

Merging ESD files

Termination

Parallel ALICE Offline Reconstruction with PROOF

parallel reconstruction implementation
Parallel Reconstruction: implementation
  • Fully based on PROOF (TSelector)
    • All platforms supported by ROOT can be used
    • No additional code/libraries are needed
  • Transparent
    • User does not notice a difference w.r.t to running locally
  • Minimal data flow between components:
    • Common (conditions and options) data accessed once from the client machine
    • Workers access raw-data events directly from AliEn
  • Minimal I/O on the workers
    • Diminishing number of intermediate files

Parallel ALICE Offline Reconstruction with PROOF

differences w r t to normal alien based reconstruction job
Differences w.r.t. to normal Alien based reconstruction job
  • Parallelization at event level (at file level in AliEn)
    • Allows faster execution in case of small number of big raw-data files
  • Conditions data/reconstruction options sent from client -> workers (in AliEn worker nodes access directly conditions data from file catalogue)
    • Allows user to test custom conditions data and/or reconstruction options
  • Output ESD file is sent back to the client machine (via xrootd)
    • Allows immediate access and check of the reconstructed data quality

Parallel ALICE Offline Reconstruction with PROOF

input data handling
Input-data handling
  • Contrary to the other 'normal' PROOF based processing/analysis, the raw-data reconstruction needs relatively big initialization data – field map, geometry, detector offline conditions DB (OCDB) entries
    • Size: from few to ~100 MB
  • Input data distributed a la ROOT “par” packages
    • Input objects assembled into an input file
    • Input file distributed on each unique file-system of the slaves
    • On the slaves input objects are extracted from the file

Parallel ALICE Offline Reconstruction with PROOF

performance on alice cern analysis facility caf
Performance on ALICE CERN Analysis Facility (CAF)

Parallel ALICE Offline Reconstruction with PROOF

performance initialization time
Performance – initialization time
  • As expected does not depend on # of slaves activated
  • Typical initialization time:~ 10s + 0.5 s/MB
  • Jump at ~3 MB due to geometry 'unpacking'

Parallel ALICE Offline Reconstruction with PROOF

performance processing rates
Performance – processing rates
  • Tested with various raw data (different participating detectors, raw-data file/event sizes)

Parallel ALICE Offline Reconstruction with PROOF

performance dependence on local i o
Performance – dependence on local I/O

No Quality Assurance (QA)

No QA,ESD-friend

No QA,ESD-friend,temp reco files

No QA,ESD-friend,temp reco,log

0

2

4

6

8

10

12

14

16

Processing rate in events/s (4 files, 2.2 GB, 1200 ev.)

Parallel ALICE Offline Reconstruction with PROOF

observations
Observations
  • Linear increase in the processing rate up to ~40 workers, “saturation effect” with more workers
    • The current CAF provides 26 workers for a user session, the speed-up in the processing is ~30 times, sufficient for fast feedback.
  • The “saturation” is more important for events with small size
  • The size of the output doesn’t affect significantly processing rate
  • => Investigation of the xrootd IO and the tree cache

Parallel ALICE Offline Reconstruction with PROOF

test w o read ahead in xrootd client and smaller tree cache
Test w/o read-ahead in xrootd client and smaller tree cache

Proc. data rate____ _ vs. number of slaves

Proc. event rate*event size

  • The xrootd read-ahead is essential:
    • The “saturation” effect is pronounced for all runs with small event size, if no read-ahead
  • The tree cash size may play significant role
    • Can be optimized
    • Some instabilities with too small cache size

<= Perfect

Parallel ALICE Offline Reconstruction with PROOF

conclusions outlook
Conclusions & Outlook
  • The parallel Proof based offline reconstruction is designed, implemented, tested and ready for the data taking
  • It permits fast feedback immediately after the storage of the raw data at Tier0
    • The current CAF provides ~30-fold speed-up in the processing rate for every user

Parallel ALICE Offline Reconstruction with PROOF

conclusions outlook1
Conclusions & Outlook
  • Investigate scalability
    • ROOT tree cache vs event-packet size (packetizer)
  • Benchmark with PROOF Lite on multicore machines
    • The code does not have to be changed at all
  • Try further optimization of input data handling and output file merging

Parallel ALICE Offline Reconstruction with PROOF

slide19

Many thanks for ALICE CAF and ROOT PROOF teams for their great help and support!

Parallel ALICE Offline Reconstruction with PROOF

slide20

SPARES

Parallel ALICE Offline Reconstruction with PROOF

exercise with local data set
Exercise with local data-set

4 files x 4 slaves (eff ~24 slaves)

4 files x 2 slaves (eff ~22 slaves)

4 files x 1 slave (only local slaves)

4 files x all slaves

via xrootd from castor

0

5

10

15

20

25

Processing rate in event/s

Parallel ALICE Offline Reconstruction with PROOF