Status Report to the LHC Computing Board Jamie Shiers, IT/ASD, CERN

1. LHC++ Status Report to the LHC Computing Board Jamie Shiers, IT/ASD, CERN

2. Overview • Introduction • Objectives • Status of LHC++ Components • Projects using LHC++ • Licensing and Distribution • Relationship with other projects • Linux & Java • Future Activities • Summary and Conclusions

3. Introduction • CERNLIB • evolved over 30+ years • result of several hundred man-years effort • important part of common software environment for HEP • Fortran-based: ‘natural’ evolution towards Fortran 90? • Questions concerning MATHLIB, 64-bit architectures, graphics, … • LHC experiments • move towards OO and C++ (now also Java) • request for “the equivalent of the current CERNLIB” (TPs & CTPs) • use of standards(both for s/w & s/w engineering) • extremely long timescales • must plan for change!

4. Computing Model: Software (I) • SOFTWARE: The Key Challenge and the Solution to Complexity • A Modern, Engineered Software Framework • Object-Oriented Design • Modern Languages (C++, Java,...) and Tools (ODBMS, HPSS,...) • Use of Mainstream Commercial products wherever possible CMS Example

5. Primary LHC++ Objectives • Meetthe requirements of the LHC experiments for a “CERNLIB-like” environment • foundation level class libraries, • mathematical libraries, • graphical libraries, • visualisation tool-kits, • event generators (in collaboration with e.g. Lund), • detector simulation (through RD44), • object persistency (through RD45). • Collaborate closely with other experiments/labs working on similar solutions • Re-use existing work where-ever possible

6. LHC++ Methodology • Adopt standards • Standards make it easier to cope with change (e.g. graphics) • Gives people reusable skills - attracts good people • Exploitwidely-usedcommercial components • when appropriate • Pros • Often more reliable and better tested than home grown solutions • Often built on industry standards, e.g. OpenGL • Significantly reduced needs for installation, documentation & maintenance • Better use of limited manpower resources (don’t re-invent the wheel…) • Cons • Don’t have complete control of code • Effort required to match versions (e.g. OS, compiler) • vendor review boards

7. LHC++ Components

8. ‘Layered’ Approach • Basic functionality provided by a set of C++ class libraries • histograms, fitting, mathematical functions, graphics etc. • Facilitates migration of individual components • Objectivity/DB to provide persistence • HEPODBMS library • smart pointers, tags, clustering directives, event collections… • Histogram library • Fitting libraries • Graphics libraries • A Modular Visualization System based on the above • Already demonstrated with RW Tools.h++ to STL migration!

9. LHC++ Strategy? • Result of many meetings, workshops, discussions etc. • Strongly influenced by the work of otherprojects • CLHEP • started at CHEP ’92 - now used by many experiments around the world • GEANT-4 class library survey • Recommended Tools.h++ as an interim solution with migration to STL • OpenGL and OpenInventor • widely adopted within HEP community • OpenGL de-facto standard in industry • OpenInventor the basis for VRML • NAG C library • based on widely-used Fortran library - largely meets our requirements • IRIS Explorer • built on OpenInventor & NAG libraries; C++ and other languages; extensible • (RD44, RD45)

10. LHC++ Progress • CLHEP • Histogram class library • Fitting tools • Mathematical libraries • HEP database layer & applications • Graphics and visualisation • Results of RD44 and RD45 will not be discussed

11. CLHEP - Class Library for HEP • Originated at CHEP ’92 • Lack of official repository - led to many different versions • Large parts obsolete • advances in C++, e.g. STL • developments that replaced need for “wrappers” to Fortran code • First official release (V1.0) made prior to CHEP ’97 • Major projects now use the official version (e.g. G4) • avoids divergence, loss of effort in maintaining multiple copies • Feedback from many users has resulted in improved versions • Random number generators, System of Units, etc. • Planning for CLHEP 2.0 under way

12. The HistOOgram package • ‘Clean-sheet’ design for 1 to N dimensional histograms • Based on URD: http://wwwinfo.cern.ch/asd/lhc++/requirements/histoo.ps • Modular : e.g. clear separation between sampling and display • Extensible : e.g. open for user defined binning systems • Light weight: e.g. whole package is implemented in 5000 lines • Operations like adding, division, scaling, etc. • Lineprinter output, streaming to/from text files • HBOOK-style interface for 1D and 2D histograms • Fitting from C++ program (either Minuit or NAG) • Persistency via HEPODBMS; visualisation via HEPInventor/Explorer • A replacement package is under development (HTL) • makes use of TEMPLATES (not widely available previously) • improved performance, more extensible and maintainable • enhancements based on user-feedback from initial prototype

13. Fitting libraries • GEMINI • C++ classes for minimization and fitting • URD: http://wwwinfo.cern.ch/asd/lhc++/requirements/stable/URD/html/ • Two engines (Minuit or NAG minimizers), common interface • New features available with NAG minimizers • Linear and general non-linear constraints; multi-thread safe! • Error routine able to handle general contraints • Contour objects (also using Minuit engine) • HEPFitting • Fitting to binned (histograms) and unbinned data • Based on GEMINI and HistOOgram packages • Provides a set of elementary functions • can be used as building blocks for composed models • Very simple user interface

14. Mathematical Libraries • General agreement that the NAG C library provides the main functionality required by HEP • previously supplied by “MATHLIB” part of CERN Program Library • Does not address all HEP-specific requirements • random number generators, based on existing CERNLIB algorithms, provided in CLHEP • fitting requirements, including MINOS-type error estimation • CERNLIB special functions, and enhancements for minimisation, will be added to the NAG C Library • Target is Mark 6 release, Q1 2000 • See LHC++ requirements page • http://wwwinfo.cern.ch/asd/lhc++/requirements.html

15. HEP Database Layer • HEP requirements go beyond scope of ODMG standard • scalable event collections, • mechanisms to optimize data access for analysis phase, • applications such as calibration database etc • Solutions provided in HEPODBMS layer • which also acts as an “insulation layer” between different releases of ODBMS, extends ODMG compliance etc • This layer is thin! • Future versions will exploit e.g. ODMG-compliant interface to schema information

16. HEPODBMS: Tags • Generic Tags • Generic content: No need to define a new persistent class • May use predefined types: float, double, short, long, char • Additional attributes may be added later • ODMG-compliant Type Manager Interface will be used in future GenericTag simTag(“simulation tag”); TagAttribute<float> et (simTag,"Et particle1"); TagAttribute<short> pid (simTag,”id particle1"); while ( evt = geant->nextEvent() ) { simTag.newTag(); // create a new tag et = evt->getPart(1).et; pid = evt->getPart(1).pdg_code; }

17. HEPODBMS: Event Collections • Single interface for multiple implementations • STL-like interface, including forward iterator • Support for collections up to 109 - 1011 events • Parallel processing support • Set-style operations based on a unique event identifier • Implemented on persistent STL & Objectivity/DB containers • Successfully created event collection of 5M events • by reference • Successfullyused with simple STL operations • copy, for_each, find • Available on Objectivity/DB V5

18. HEPODBMS: Calibration DB • ODBMS-based calibration databases have been developed in both BaBar and CMS • functionality similar - calibrations retrieved on basis of validity “instant” • as in e.g. DBL3, HEPDB, OPCAL • BaBar version has been adopted by a number of other experiments • STAR, BELLE, under test in ATLAS etc. • Currently studying the feasibility of providing an experiment-independent version of the BaBar CalibDB as part of LHC++ • remove BaBar dependencies; enhance functionality to meet requirements of other experiments

19. Graphics & Visualisation • Basedon industry standard libraries such as • OpenGL: http://www.opengl.org/ • Hardware acceleration offered by many affordable graphics cards • Support for all main platforms • DEC, HP, IBM, Linux, SGI, Sun, NT • Extended with C++ class libraries • OpenInventor; 3DMasterSuite; HEPInventor • Used by many other HEP applications • e.g. GEANT-4, HEPVis • Basis for VRML • http://www.vrml.org/ • The building blocks for IRIS Explorer

20. IRIS Explorer • IRIS Explorer: a modular data analysis/visualization framework • Graphics built on OpenGL & OpenInventor • Mathematical algorithms from NAG libraries • Wide user community (1500 sites) • Visual Programming approach • users build an application (map) by wiring together basic blocks (modules) • Rich set of general purpose & public domain modules • Users can enhance the environment writing new modules • C, C++, Fortran(Java under study) • Script language (alternatives, e.g. Perl, Tcl, Python under study) • Distributed applications • e.g. execute I/O intensive modules close to the Event store

21. HEPExplorer • A small set of HEP specific modules for IRIS Explorer • complement the standard & public domain ones • Focuses on Interactive Analysis • histograms, Ntuple-like analysis, fitting • Based on LHC++ components • HistOOgrams, HEPODBMS, Fitting libraries,…

22. HEP Explorer Modules • HistOOgram handling • Create, clone, delete, plot, reset, read • Add, subtract, product, ratio • Fitting • currently based on Minuit • new version based on GEMINI under development • Transform to lattice, line printing • Interactive Data Analysis • Explorable Collection Browser • HistOOgram Filling, HepLoop • Shared Library Generator, uses native C++ Compiler • Other (DB Browser, etc.)

23. IRIS Explorer Graphics • State-of-the-art 3D graphics • OpenGL, OpenInventor, VRML • Geant4 detector geometry • Geant4 Magnetic field • HEPInventor: HEP presentation graphics library • Based on MasterSuite (OpenInventor extensions) • Available as a C++ class library • Page layout, grid, text size • Log/linear axis, line width, line shape, colours • Bar chart, errors, smooth curves, colour plot, box, circle, lego • Vector PostScript output

24. Analysis class libraries • Histograms • 1D, 2D histograms with fixed or variable length bins • n-D histograms, extensible binning system • HEPODBMS • Explorable Collections of Tags • Set of attributes kept e.g. per event (or per track, photon, etc.) • Still connected to main event data • Two flavours, one interface: • concrete (corresponding to specific ODBMS classes, meant to speed up queries) • generic (no specific ODBMS classes, “replacement” for personal n-tuples) • Event Collections • Fitting • Minimization, fitting of unbinned data and histograms

25. Current Status • Release 98a • HP 10.20 (CC) compiler, Solaris 2.5/2.6 (CC 4.2), Windows NT (VC++5.0) DEC-UNIX 4.0 (cxx 5.6) • Soon on AIX 4.2 • Used by CMS (X5 & H2 testbeams), Atlas (Tile testbeam, users in the simulation group), GEANT-4, NA45 • Next release : early 99 • http://wwwinfo.cern.ch/asd/lhc++

26. Projects Using LHC++ Projects underway in LEP/SPS/LHC experiments • ALEPH: test of analysis framework + data archiving • DELPHI: ditto • NA45: event storage, event display • COMPASS: event storage • ATLAS: numerous pilot projects • CMS: test beam activities - full production chain Projects starting/planned in others: • ALICE: • LHCb: See also LHC++ workshop of 29 Sep - 1 Oct

27. Licensing & Distribution • Requests for licenses handled via COCOTIME review • both on- and off-site • External institutes should sign and return license forms • No cost to institutes! • one per commercial component; once only per institute • need also account name for access to down-load area • HEP components (e.g. CLHEP) do not require license forms • Distribution kits can then be copied as for CERNLIB • Again as for CERNLIB, s/w installed centrally at CERN • NICE, AFS • CD distribution being prepared • 1999 COCOTIME requests call for licenses for some 120-150 external sites

28. Relationships with other projects (RD44, RD45) Software Development Process and Environment for LHC • Highest priority areas are considered to be: • verification and validation • testing • configuration management • Will adopt common solutions, e.g. SRT, also from other sub-projects Other laboratories and experiments • meetings now use VC; workshops; joint collaboration e.g. CLHEP • intent to increase level of collaboration where possible • Lack of VC in large conference rooms a problem

29. Linux Following requests from user community, Linux versions have been agreed with vendors: • Objectivity/DB: pre-ß exists; release scheduled end-’98 • ObjectSpace: existing version supports g++ • NAG C (& Fortran) libraries: port scheduled for end-’98 • OpenInventor & MasterSuite: port available as from end-’97 • IRIS Explorer: port scheduled for end-’98 • LHC++ release: 99A (Jan/Feb ’99)

30. Java • Java already used within RD44/RD45 • user interfaces in GEANT-4 • Database Management Tool for Objectivity/DB • Java bindings / solutions exist for LHC++ components: • Objectivity/DB: new language binding, installed in LHC++ tree • ObjectSpace: JGL collection classes, based on STL • OpenInventor & MasterSuite for Java • IRIS Explorer: project to incorporate Java modules • http://www.scs.leeds.ac.uk/iecoe/ • Do not expect Java to replace C++ in immediate future • Co-existence; interoperability • See also Java Grande Forum • http://www.javagrande.org/ • JLAPACK, Java Numerical Toolkit, f2j(!),

31. Future Activities • Continue to offer the highest levels of support, documentation and solutions that are possible • Ensure that requirements of user community met by new releases in a timely manner • Ensure that access to the s/w, both at CERN and outside, is as straight-forward as possible

32. Futures - Detail • Make a production release of LHC++ in early 1999 • including GEANT-4 • Release a first version of the Pythia-8 event generators, • Provide CD ROM-based distribution of LHC++ software + documentation etc. • Participate in training activities such as the CERN School of Computing, the proposed INFN workshop and so forth, • Follow developments in areas such as STABLE, mathematical libraries, Fahrenheit, the use of component-ware, the Java-based implementations of MasterSuite and OpenInventor, Java/C++ interoperability including the Java binding to Objectivity/DB, Java-based analysis tools.

33. Summary • A consistent set of widely-used, standards-conforming (influenced) products has been identified • license model proposed and adopted • acquired for CERN site and current needs for external institutes • HEP-specific extensions designed & implemented • in general, these represent a small amount of code • e.g. wrt CERNLIB • obvious exception is GEANT-4, (MCLIBS++) • Proposed solution is modular • migration to new products demonstrated • Compatibility & interoperability with Java foreseen • Pilot projects underway in SPS/LEP & LHC experiments

35. Conclusions • The milestones, outlined in the Project Execution Plan, have been achieved • The use of standard, off-the-shelf tools together with a small amount of HEP-specific code to solve problems typical of HEP community has been demonstrated • packages typically ~2-3K lines of code • HEPExplorer modules often few tens to few hundred lines • Migration between different components whilst preserving overall strategy has been successfully accomplished • e.g. Rogue Wave Tools.h++ to ObjectSpace STL • A production release (98A) has been made & distribution kits for outside labs are available

36. Current LHC++ strategy • HEP applications (GEANT-4, event generators etc.) • HEPExplorer • IRIS Explorer • MasterSuite, HEPVis, HEPInventor • OpenInventor • OpenGL • HEPMATH (special functions, minimiser, etc.) • NAG C library • CLHEP • HEPODBMS • Objectivity/DB • The Standard C++ libraries

37. The End