Root project status major developments directions l.jpg
This presentation is the property of its rightful owner.
Sponsored Links
1 / 79

ROOT Project Status Major developments Directions PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

ROOT Project Status Major developments Directions. DESY 5 December 2005 Ren é Brun CERN. ROOT: a 10 years old project. Started in January 1995 in NA49 First public presentation in November 95 Cooperation with Masa Goto/CINT in April 96

Download Presentation

ROOT Project Status Major developments Directions

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Root project status major developments directions l.jpg

ROOT Project StatusMajor developmentsDirections


5 December 2005

René Brun


Overview, Major Developments, Directions

Root a 10 years old project l.jpg

ROOT: a 10 years old project

  • Started in January 1995 in NA49

  • First public presentation in November 95

  • Cooperation with Masa Goto/CINT in April 96

  • First implementation of ROOT with gAlice in December 97 (TGeant3)->VMC

  • FNAL chooses ROOT for I/O and data analysis

    • Announced at CHEP98 in Sept 98 in Chicago

    • RHIC experiments follow immediately

  • Hoffmann computing Review in 2000/2001

  • LCG project in 2002 (Blueprint RTAG)

  • ROOT: LCG project in 2005 :

Overview & Major Developments

Applications area organization in lcg phase i l.jpg

Applications Area Organizationin LCG Phase I








User - provider















Overview & Major Developments

Applications area organization in lcg phase ii l.jpg

Applications Area Organizationin LCG Phase II

Same organization as in Phase I.

ROOT and SEAL projects merge













ROOT project structured in work-packages

Overview & Major Developments

New root team structure l.jpg

New ROOT team structure

The work-packages


Lorenzo Moneta


Philippe Canal


Fons Rademakers


Lorenzo Moneta

I/O & Trees

Philippe Canal

2-D/3D graphics

Olivier Couet


Ilka Antcheva


Fons Rademakers


Andrei Gheata

Overview & Major Developments

Root version 5 l.jpg

ROOT version 5

  • pro release 4.04/02 3 May (new Users Guide)

  • 1st dev release 5.02 28 June

  • 2nd dev release 5.04 20 September

  • 3rd dev release 5.06 2 Nov

  • Pro release 5.12 15 December (New Guide)

See detailed Release Notes

Overview & Major Developments

Root 2005 workshop l.jpg

ROOT 2005 workshop

  • September 28-30 at CERN

  • 115 registered participants

  • 41 talks

  • 9 posters

  • See talks at ROOT web page

Overview & Major Developments

Root large user community l.jpg

ROOT: large user community


Overview & Major Developments

Base work package l.jpg

Base Work Package

New infrastructure features


Overview, Major Developments, Directions

Tarchivefile and tzipfile l.jpg

TArchiveFile and TZIPFile

  • TArchiveFile is an abstract class that describes an archive file containing multiple sub-files, like a ZIP or TAR archive.

  • The TZIPFile class describes a ZIP archive file containing multiple ROOT sub-files. Notice that the ROOT files should not be compressed when being added to the ZIP file, since ROOT files are normally already compressed. To create the file do:

  • The ROOT files in an archive can be simply accessed like this:

  • A TBrowser and TChain interface will follow shortly.

zip –n root multi file1.root file2.root

TFile *f = TFile::Open("")


TFile *f = TFile::Open("root://mymachine/")

Overview & Major Developments

Replica of a db subset l.jpg

Replica of a DB subset



local TZipFile

remote TZipFile

http, xrootd, castor, dcache..

Overview & Major Developments

Slac s new file server xrootd l.jpg

SLAC’s New File Server - xrootd

  • The file server xrootd (eXtended ROOT daemon) has been developed by Andy Hanushevsky of SLAC.

  • The server exploits a multithreaded architecture to provide high-performance file based access, focusing on scalability and fault tolerance. The server is being extensively used by BaBar, Star and Alice.

  • The xrootd file server will in the near future replace the current daemon rootd.

  • xrootd and its plugin facility is heavily used by PROOF

Overview & Major Developments

The new xrootd client txnetfile l.jpg

The New xrootd Client - TXNetFile

  • The new client class TXNetFile implements the xrootd protocol and is provided to open a file via the xrootd daemon.

  • TXNetFile can detect when it talks to on old rootd daemon and return a TNetFile.

  • To open a file via xrootd, just use the standard static method TFile::Open() as for opening via rootd.

Overview & Major Developments

Class tgrid abstract interface l.jpg

Class TGrid (abstract interface)

//--- General GRID

const char *GridUrl() const

const char *GetGrid() const

const char *GetHost() const

const char *GetUser() const

const char *GetPw() const

const char *GetOptions() const

Int_t GetPort() const

//--- Catalogue Interface

virtual TGridResult *Command(const char *command,

Bool_t interactive = kFALSE,

UInt_t stream = kFALSE)

virtual TGridResult *Query(const char *path, const char *pattern,

const char *conditions,

const char *options)

virtual TGridResult *LocateSites()

virtual TGridResult *ls(const char*ldn ="", Option_t*options ="")

virtual Bool_t cd(const char*ldn ="",Bool_t verbose =kFALSE)

virtual Bool_t mkdir(const char*ldn ="", Option_t*options ="")

virtual Bool_t rmdir(const char*ldn ="", Option_t*options ="")

virtual Bool_t register(const char *lfn , const char *turl , Long_t size,

const char *se, const char *guid)

virtual Bool_t rm(const char*lfn , Option_t*option ="")

//--- Job Submission Interface

virtual TGridJob *Submit(const char *jdl)

virtual TGridJDL *GetJDLGenerator()

//--- Load desired plugin and setup conection to GRID

static TGrid *Connect(const char *grid, const char *uid,

const char *pw, const char *options)

Overview & Major Developments

Slide15 l.jpg

Access to File Catalogues

eg Alien FC

Same style interface could be implemented for

Other GRID File Catalogues

Overview & Major Developments

Slide16 l.jpg

TGrid example with Alien

// Connect

TGrid alien = TGrid::Connect(“alien://”);

// Query

TGridResult *res =alien.Query



// List of files

TList *listf = res->GetFileInfoList();

// Create chain

TChain chain(“Events", “session");


// Start PROOF

TProof proof(“remote”);

// Process your query


Overview & Major Developments

Auto loading of plugins l.jpg

Auto Loading of Plugins

  • Support for auto-loading libraries when an unknown class is being referenced.

  • The auto-loading mechanism reads the files $ROOTSYS/etc/system.rootmap, ~/.rootmap and ./.rootmap (via TEnv) to try to map the unknown class to a library.

  • If the library is found it, and the libraries on which it depends, are loaded.

  • The rootmap files are created with the rlibmap tool when executing "make map".

  • Example: in an interactive session, one can do directly without having to do

TLorentzVector v;


Overview & Major Developments

Tmacro l.jpg


  • This class allows for storing a C++ macro in a ROOT file.

  • In addition to being stored in a ROOT file a TMacro can be executed, edited, etc.

TMacro m("Peaks.C"); //macro m with name "Peaks" is created

//from file Peaks.C

m.Exec(); //macro executed with default arguments

m.Exec("4"); //macro executed with argument


TFile f("mymacros.root","recreate");

m.Write(); //macro saved to file with name "Peaks"

Overview & Major Developments

Using pcre for reg exp s l.jpg

Using PCRE for Reg Exp’s

  • new class TPRegexp which uses the Perl Compatible Regular Expressions library.

  • Well know, rich, regular expression syntax.

  • It is interfaced to TString and other class and methods now using TRegexp.

  • TRegexp will of course stay for backward compatibility.

Overview & Major Developments

Dict work package l.jpg

Dict Work Package

New version of Reflex

New version of rootcint

rootcint  CINT

rootcint -> Reflex ->Cintex ->CINT

rootcint ->gccxml -> Reflex -> CINT

Adapt PyRoot to Reflex

Adapt CINT to Reflex

Overview, Major Developments, Directions

Dictionaries root only l.jpg

Dictionaries : root only

Root meta C++










Overview & Major Developments

Dictionaries situation today l.jpg

Dictionaries : situation today








Root meta C++












Overview & Major Developments

Dictionaries situation in the future l.jpg

Dictionaries : situation in the future



Root meta C++






rootcint -cint


rootcint -reflex


rootcint -gccxml

Overview & Major Developments

Io work package l.jpg

IO work-package

Consolidation, Consolidation, Consolidation

Support for STL collections

More cases in auto schema evolution

Better support for references

Bitmap index


Overview, Major Developments, Directions

Root i o stl collections l.jpg

ROOT I/O: STL Collections

  • ROOT now supports I/O of all STL containers

    • std::vector<T> std::list<T> std::set<T> std::deque<T>std::map<K,T> std::multimap<K,T>

    • And implicitly (through std::deque) std::queue<T> std::stack<T>

  • STL collections are saved in split mode

    • Objects are split (but: NOT if pointers)

    • Quick pre-selections on trees

    • Interactivity: Trees can be browsed

    • Save space (see $ROOTSYS/test/bench):std::vector<THit>:compression 5.38std::vector<THit*>:compression 3.37

Overview & Major Developments

Float double and space 1 l.jpg

Float, double and space…(1)

  • Math operations very often require doubleprecision, but on saving single precision is sufficient…

  • New data type: Double32_tIn memory: doubleOn disk: float or integer

Overview & Major Developments

Float double and space 2 l.jpg

Float, double and space… (2)

  • Usage (see tutorials/double32.C):Double32_t m_data; // [min,max<,nbits>]

    • No nbits,min,max:saved as float

    • min, max:saved as int 32 bits precisionexplicit values or expressions ofvalues known to Cint (e.g. “pi”)

    • nbits present:saved as int with nbit precisionhigher precision than float forsame persistent space

Overview & Major Developments

Float double and space 3 l.jpg

Float, double and space… (3)

Increase precision

Save space

Overview & Major Developments

File types access in 5 06 l.jpg

File types & Access in 5.06

























Overview & Major Developments

Bitmap indices l.jpg

Bitmap Indices

  • Bitmap indices are efficient data structures for accelerating multi-dimensional queries:

    • E.g. pT > 195 AND nTracks < 4 AND muonTight1cm > 12.4

  • Supported by most commercial database management systems and data warehouses

  • Optimized for read-only data

  • However, because an efficient index may be as big as the data, we think that it is only appropriate for things like event meta data catalogues

Overview & Major Developments

Query performance ttreeformula vs bitmap indices l.jpg

Query Performance - TTreeFormula vs. Bitmap Indices

Bitmap indices 10X faster than TTreeFormula

Overview & Major Developments

Data analysis with bitmap indices l.jpg

Data analysis with bitmap indices

Event catalogue

Bitmap index

Direct use by PROOF slaves to select events


Event list

Overview & Major Developments

Math work package l.jpg

Math work-package





Linear & Robust Fitter


Overview, Major Developments, Directions

Math work package news l.jpg

MATH work-package : News

  • MathCore library with basic Math functionality

    • Basic Special and statistical functions

    • Physics and geometry vectors

  • MathMore library

    • C++ interface to function and algorithm from GSL

      • Extra math functions, Adaptive integration, derivation, root finders

  • Minuit2

    • New OO implementation of Minuit

    • Interface to ROOT TVirtualFitter

  • Linear and Robust Fitter

  • sPlot

Overview & Major Developments

New math libraries organization l.jpg

New Math Libraries organization

Overview & Major Developments

Math work package plan l.jpg

MATH work-package : Plan

  • Complete MathCore with Random numbers

  • Adapt ROOT classes to MathCore

  • TF1,2,3, Fitting

  • Virtual Fitter extensions

    • corresponding changes in ROOT fitting and roofit

  • Fully integrate and extend new Minuit

  • Fitting GUI

  • Box plots, qqplots

  • Many new tools required for LHC Physics analysis (PHYSTAT05 Oxford)

Overview & Major Developments

Linear fitter 0 l.jpg

Linear Fitter (0)

  • To fit functions linear in parameters

    • Polynomials, hyperplanes, linear combinations of arbitrary functions

  • TLinearFitter can be used directly or through TH1, TGraph, TGraph2D::Fit interfaces

  • When used directly, can fit multidimensional functions

Overview & Major Developments

Linear fitter 1 l.jpg

Linear Fitter (1)

  • Special formula syntax:

    • Linear parts separated by “++” signs:

      • “1 ++ sin(x) ++ sin(2*x) ++ cos(3*x)”

      • “[0] + [1]*sin(x) + [2]*sin(2*x) + [3]*cos(3*x)”

    • Simple to use in multidimensional case

      • “x0 ++ x1 ++ exp(x2) ++ log(x3) ++ x4”

  • Polynomials (pol0, pol1…) and hyperplanes (hyp1, hyp2, …) are the fastest to compute

  • By default, polynomials in TH1, TGraph::Fit functions now go through Linear Fitter

  • Data to be used for fitting is not copied into the fitter

Overview & Major Developments

Linear fitter 2 l.jpg

Linear Fitter (2)

  • Advantages in separating linear and non-linear fitting:

    • Doesn’t require setting initial parameter values

    • The gain in speed

Overview & Major Developments

Robust fitting 0 l.jpg

Robust fitting (0)

  • Least Trimmed Squares regression – extension of the TLinearFitter class

  • Motivation: least-squares fitting is very sensitive to bad observations

  • Robust fitter is used to fit datasets with outliers

  • The algorithm tries to fit h points (out of N) that have the smallest sum of squared residuals

Overview & Major Developments

Robust fitting 1 l.jpg

Robust fitting (1)

  • Highbreakdown point - smallest proportion of outliers that can cause the estimator to produce values arbitrarily far from the true parameters

Graph.Fit(“pol3”, “rob=0.75”, -2, 2);

2nd parameter – fraction h of the good points

Overview & Major Developments

Multivariate covariance l.jpg

Multivariate covariance

  • Minimum Covariance Determinant Estimator – a highly robust estimator of multivariate location and scatter

  • Motivation: arithmetic mean and regular covariance estimator are very sensitive to bad observations

  • Class TRobustEstimator

  • The algorithm tries to find a subset of h observations (out of N) with the minimal covariance matrix determinant

Overview & Major Developments

Multivariate covariance43 l.jpg

Multivariate covariance

  • Left – covariance ellipses of a 1000-point dataset with 250 outliers

  • Right – distances of points from the robust mean, calculated using robust covariance matrix

  • High breakdown point

  • Indices of outlying points can be returned

Overview & Major Developments

Splot a statistical tool to unfold data distributions l.jpg

sPlot – A statistical tool to unfold data distributions

Left – Projection plot

cut on the likelihood ratio

Excess of events –

Signal? Background?

Right – sPlot – no cut

getting rid of background

by statistical methods


Overview & Major Developments

News in th1 and tf1 l.jpg

News in TH1 and TF1

  • TH1:

    • Chi2 test

    • Mean & RMS error, skewness and kurtosis

  • TF1:

    • Derivatives (1st, 2nd and 3rd)

    • Improved minimization – a combination of grid search and Brent’s method (golden section search and parabolic interpolation)

Overview & Major Developments

Graphics work package l.jpg

Graphics work-package

zillions of micro/mini features

reimplement (TGaxis)

GL with new GUI

GL for dynamic tracks

GL in Pad

Overview, Major Developments, Directions

Slide47 l.jpg

Overview & Major Developments

Timagedump l.jpg


  • Many extensions to the libAfterImage library to support line, marker and (filled) polygon drawing. Accessible via TASImage.

  • The new class TImageDump uses TASImage and derives from TVirtualPS to allow the saving of canvases in gif, jpg, png, tiff, etc., image formats in batch mode:

  • Or to display any gif, jpg, png, tiff in a canvas, do:

$ root –b

root [0] .x hsimple.C

root [1] c1->Print("c1.gif");

TCanvas *c1;

TImageDump *imgdump = new TImageDump("test.png");



Overview & Major Developments

Gl in pad l.jpg

GL in Pad

Overview & Major Developments

Slide50 l.jpg

Overview & Major Developments

Gl features clipping l.jpg

GL Features : Clipping

Overview & Major Developments

Gui work package l.jpg

GUI work-package

zillions of micro/mini features

GUI Builder completion

New Editor Widgets

Fit Panel widget

Overview, Major Developments, Directions

Gui work package plan l.jpg

GUI work-package : Plan

High Level Widgets

Editors, Browsers,

Building Blocks Widgets

Combos, scroll bars, dialogs, sliders, MDI, etc

Code generators


New features

Bug fixes


Very stable




Overview & Major Developments

Slide54 l.jpg

Overview & Major Developments

Graphics editor l.jpg

Graphics Editor

  • Object orientation of editor design

  • Manage GUI complexity by object editors

  • Presents the right GUI at the right time according to the selected object in the canvas

  • Easy-to-use

  • Capacity for growth

Overview & Major Developments

Style manager l.jpg

Style Manager

  • Top level interface

    • Manage a collection of TStyle objects

    • Create a new style

    • Delete a selected style

    • Import from a canvas / a C++ macro

    • Export to a C++ macro

    • Apply on all canvases or a selected object

    • Activate the style editor

  • Preview window

    • Show the predicted results

    • On line update or by request

    • Placed in front of the selected canvas

  • Style Editor

Overview & Major Developments

Gui builder l.jpg

// transient frame

TGTransientFrame *frame2 = new TGTransientFrame(gClient->GetRoot(),760,590);

// group frame

TGGroupFrame *frame3 = new TGGroupFrame(frame2,"curve");

TGRadioButton *frame4 = new TGRadioButton(frame3,"gaus",10);


frame2->SetWindowName(“Fit Panel");





GUI Builder

  • GUI Builder simplifies the process of designing GUIs based on the ROOT widget classes.

  • Using Ctrl+S orSaveAsdialog, users can generate C++ code in a macro that can be edited and executed via CINT interpreter:

    root [0] .x example.C

Overview & Major Developments

Geom work package l.jpg

GEOM work-package

Support for parameterized shapes. This will reduce the geometry size in memory for certain geometries defined in G3 style.

CAD geometry import

Geometry builder GUI

Overview, Major Developments, Directions

Lhc detectors in root tgeo l.jpg

LHC detectors in ROOT TGeo

Overview & Major Developments

The v irtual mc l.jpg


G3 transport

User Code



G4 transport

FLUKA transport



Geometrical Modeller



The Virtual MC

Overview & Major Developments

Virtual monte carlo and root geometry l.jpg

Virtual Monte Carlo and ROOTGeometry

  • TGeant3

    • Used in production – native GEANT3

    • New: TGeant3TGeo – interface to G3 using TGeo geometry

      • No modification required in the user code

      • See presentation from Ivana Hrivnacova

  • TGeant4

    • Used for Geant4 physics validation – G4 native geometry built after g3tog4 conversion

    • No interface yet between G4 and ROOT geometry

      • Few possible strategies for this implementation discussed in details at last VMC workshop

      • Expect an interface with G4 June06

  • TFluka

    • Old geometry interface using G4 geometry vis FLUGG

    • Currently a fully validated geometry interface based on TGeo

Overview & Major Developments

Proof work package l.jpg

PROOF work-package

Connect/Disconnect modes

Multiple queries in parallel

Feedback histograms

Multi-core CPUs

Overview, Major Developments, Directions

Slide63 l.jpg

PROOF – Parallel ROOT Facility

G. Ganis, ROOT05, 29 Sept 2005

  • System to export the ROOT analysis model on

    clusters of computers for interactive analysis of

    large data sets

  • Flexible multi-tier architecture

    • in GRID contexts adapts to cluster of clusters or wide area

      virtual clusters

  • Exploit inter-independence of entries in a tree or

    directory to achieve basic parallelism

    • data set split into packets assigned to worker nodes

      on demand

Overview & Major Developments

Slide64 l.jpg

PROOF – Multi-tier Architecture

good connection ?

less important

VERY important

G. Ganis, ROOT05, 29 Sept 2005

  • Optimize for data locality

  • If not possible, remote data via (x)rootd, rfiod, dCache …

Overview & Major Developments

Slide65 l.jpg

PROOF – data analysis

G. Ganis, ROOT05, 29 Sept 2005

Normal ROOT

  • TChain: collection of TTree

  • TSelector: :Begin(),Process(),Terminate()

TChain a("h42");

{// Define the data set





// Process the selector



Local processing


  • Same chain, same selector

{// Open PROOF

TProof proof(“master”);

// Process the selector



Remote processing

Overview & Major Developments

Proof and selectors l.jpg

Initialize each slave

Many Trees are being processed

No user’s control on the order

PROOF and Selectors




The same code works also without PROOF (of course!)




Overview & Major Developments

Slide67 l.jpg

PROOF – User Sandbox

G. Ganis, ROOT05, 29 Sept 2005

  • User’s have their own sandbox on each worker node

  • File transfers minimized

    • cache packages, selector

    • File integrity: MD5 checksums, timestamps

  • Package manager to upload files or packages

    • binary or source

    • PAR (PROOF Archive, like Java jar)

      • provides ROOT-INF directory,, SETUP.C

        to control setup in each worker

    • TProof API to handle all this

Overview & Major Developments

Slide68 l.jpg

Typical query-time distribution

G. Ganis, ROOT05, 29 Sept 2005


Blocking /



Overview & Major Developments

Analysis session example l.jpg

Analysis Session Example

AQ1: 1s query produces a local histogram

AQ2: a 10mn query submitted to PROOF1

AQ3->AQ7: short queries

AQ8: a 10h query submitted to PROOF2

Monday at 10h15

ROOT session

on my laptop

BQ1: browse results of AQ2

BQ2: browse temporary results of AQ8

BQ3->BQ6: submit 4 10mn queries to PROOF1

Monday at 16h25

ROOT session

on my laptop

Wednesday at 8h40

Carrot session

on any web browser

CQ1: Browse results of AQ8, BQ3->BQ6

Overview & Major Developments

Slide70 l.jpg

GUI manager

Allows fullon-clickcontrol on everything

  • define a new session

  • submit a query, execute a command

  • query editor

    • execute macro to define or pick up a TChain

    • browse directories with selectors

  • online monitoring of feedback histograms

  • browse folders with results of query

  • retrieve, delete, archive functionality

Overview & Major Developments

Slide71 l.jpg

Query processing

Processing information

Feedback histograms

Overview & Major Developments

Proof and xrootd l.jpg


  • XROOTD is already playing a very important role in PROOF. It will continue to play a growing role.

  • The PROOF and XROOTD teams are cooperating to get even more from XROOTD: caching, read ahead, new XROOTD services.

  • Still a lot to do to have a good integration of XROOTD with other services like CASTOR.

Overview & Major Developments

Proof test facility l.jpg

PROOF test facility

  • Since a few weeks we have access to a dedicated farm with 32 dual processor nodes. These machines are intended for testing, not production.

  • The nodes are slow machines (800 MHz), but are extremely useful (vital) to test our PROOF prototypes (Alice, CMS, Phobos).

  • We expect to have many more (100) and faster machines next year. CAF for Alice and CMS.

Overview & Major Developments

Multi core cpus l.jpg

Multi Core CPUs

This is going to affect the evolution of ROOT in many areas

Overview & Major Developments

Moore s law revisited l.jpg

Moore’s law revisited

Overview & Major Developments

Cpu node hierarchy l.jpg


100 nanos

100 micros

100 millis



100 Giga

100 Tera

10 Peta



> 1 Peta

> 1 Tera

> 100 Peta

CPU/Node hierarchy

Laptop node

1->32->??N cpus

Local cluster

1000xN cpus


100x1000 nodes

Overview & Major Developments

Many implications for root l.jpg

Many implications for ROOT

  • More and more multi-threaded applications

  • Will have to make many classes thread aware

  • ACLIC compilation in parallel

  • GL viewer could take advantage of multi cpus

  • Fitting too

  • I/O with threads for

    • Read ahead

    • Unzipping

  • And obviously TTree queries and PROOF

Overview & Major Developments

Summary l.jpg


  • After 10 years of development, ROOT is widely used in HEP and elsewhere.

  • The team has been extended with LCG2 and cooperates with many external developers.

  • Consolidation phase for I/O and Trees

  • Intensive developments in most packages

  • Pushing PROOF data analysis model

  • Pro release 5.12 (15 December)

  • Next Pro release 6.06 (June 2006)

Overview & Major Developments

Seal root transition l.jpg

SEAL ROOT transition


  • Adiabatic changes towards the experiments

  • SEAL functionality will be maintained as long as the experiments require


Expts S/W



Overview & Major Developments

  • Login