Utilizing CMT in LHCb: Package Management Workshop Overview

1. Use of CMT in LHCb CMT Workshop, LAL (Orsay) 28th February - 1st March 2002 P. Mato / CERN

2. Contents • History • Package Categories • Release Practices • Useful Tools • Current Problems Use of CMT in LHCb

3. History • We started using CMT in LHCb before CMT was called CMT (Methods) • All the new C++ code (Gaudi) developed using CMT • Also adopted afterwards for the legacy FORTRAN code • Currently order of 100 packages organized hierarchically (“hat/packagename”) • Used on Linux and Windows since the beginning • Used directly by the physicists and librarians Use of CMT in LHCb

4. Packages Categories • Program: is a package that contains a main routine and a list of dependent packages needed to link it. • Library: contains a list of classes and the list of dependent packages needed to compile it. • Policy package: defines macros, compilation & link flags, patterns, fragments, etc. • Package group: contains a list of other packages with their version number (e.g. GaudiSys) • Interface package: interfacing to packages not managed with CMT (e.g. CERNLIB, CLHEP, ROOT,…) Use of CMT in LHCb

5. Policy Package (GaudiPolicy) • Definition of “tags” • Standard configurations • General macros • Compilation and link flags • Patterns • Simple and uniform requirements files • Naming conventions, standardization,… • Fragments • DevStudio, RuleChecker, etc. Use of CMT in LHCb

6. Interface Packages ExternalLibs Version Platform Compiler CLHEP CMTSITE • Set of environment variables to define where the external packages are installedXXX_DIR • Compilation and link options • Environment variablesPATH, LD_LIBRARY_PATH • Path relative to XXX_DIR Use of CMT in LHCb

7. Interface Package (cont’d) package CLHEPversion v17r10branches doc cmt include_path none use ExternalLibs v3* set LHCXX_BASE "${LHCXX_DIR}" \ Linux-2.95.2 "${LHCXX_DIR}/gcc-2.95.2" include_dirs $(CLHEP_DIR)/include set CLHEP_DIR "${LHCXX_BASE}/CLHEP/1.7.1.0" macro CLHEP_linkopts "-L${CLHEP_DIR}/lib -lCLHEP" \LinuxStatic "${CLHEP_DIR}/lib/libCLHEP.a" \WIN32 "${CLHEP_DIR}/lib/CLHEP.lib" path_remove LD_LIBRARY_PATH "/CLHEP/" \ WIN32 "\CLHEP\" path_append LD_LIBRARY_PATH "${CLHEP_DIR}/lib" Use of CMT in LHCb

8. Package Group package GaudiSys version v9r0 #---CMT and Gaudi policy use GaudiPolicy v5r* #---Gaudi libraries - generic use GaudiKernel v11r* use GaudiSvc v7r* use GaudiAud v5r* use GaudiAlg v5r* use GaudiTools v5r* use GaudiNagC v6r* use GaudiDb v5r* use GaudiIntrospection v2r* #--- External libraries use CLHEP v17r10 use HTL v13r10 use AIDA v1r0 use ExternalLibs v3r* • Useful to “fix” the versions of a number of packages • To be used directly by users • For management purposes • Release tools • “broadcast” command Use of CMT in LHCb

9. Use Practices • CMTCONFIG • Default configuration “tag” • If “tag” not specified then use CMTCONFIG as tag • .cmtrc • The list of “official” release areas • CMTPATH • $HOME/mycmt [eventually DEV areas] • Using both optimized and debug versions > getpack MyPackage v1r1> cd MyPackage/v1r1/cmt> make [tag=$CMTDEB]> source setup.csh [–tag=$CMTDEB]> … Use of CMT in LHCb

10. Release Practices • Complete LHCb software release 3-4 times a year • Major new functionality • Incompatible changes allowed • Changes in external packages versions & dependencies • Change in the major version number • Partial/Incremental releases when convenient • Bug fixes • New functionality (changes in Algorithms/Services) • Major emphasis on source code/binary compatibility • Change in the minor version number Use of CMT in LHCb

11. Integration and Testing new releases • We do not have a nightly build system • Integration Area (DEV) • Quasi-continuous builds of “head” revision of the packages involved in the integration • Release by Layers • One after the other Private Area Public DEV Area Public RELEASE Area Private Area Private Area Applications Algorithms Event/Detector Framework (Gaudi) Use of CMT in LHCb

12. Extra Tools: DevStudio add-in • Integration of CMT in DevStudio • Regeneration of DevStudio project • Set environment • Show uses • … • Using exclusively the “cmt.exe” interface Use of CMT in LHCb

13. Extra Tools: GetPack • Python script to checkout/update packages in the LHCb CVS repositories • Easy to use (combines CVS and CMT commands) • Platform independent • We do not use “cmt co” command • Unpredictable result • Can not handle several repositories usage: getpack [OPTIONS]... package [ [version] ['tag'|'head'] ] getpack [OPTIONS]... -i ['Gaudi' [hat] |'LHCb' [hat] ] Use of CMT in LHCb

14. Extra Tools • Scripts for the creation of tar files for distributions • Prototype of a script for “Grouping” a number of packages into a single one • Useful for using the package • Improves scalability • Reduces the length of PATHs and commands • Not useful for package development Use of CMT in LHCb

15. Current Problems • Not able to use new CMT version • Our current practices do not work  big impact to the end-users • Need to understand new “private use” feature • Impact on “shareable libraries” and environment variables • Problems with “soft links”. Introduced to reduce the length of PATH and LD_LIBRARY_PATH • Not re-built when something has changed • Not re-built on “source setup” • No not work on Windows! Use of CMT in LHCb

16. Current Problems (cont’d) • Care with handling “wildcards” in version numbers • Gained experience but still making errors time to time • Missing standard Interface packages • Difficult to integrate other software systems following different “conventions” Use of CMT in LHCb

17. Summary • Using CMT since the beginning • Has taken some time to develop the current practices (package categories, use version wildcards, etc.) • Additional tools developed around CMT are necessary • Major concerns in future compatibility Use of CMT in LHCb