240 likes | 252 Views
Peter Hristov CERN, March 22, 2006. Installation of the ALICE Software. Packages. Simulation, reconstruction, analysis Root Geant3 Fluka AliRoot Grid/distributed computing: VO, etc. LCG AliEn. What use cases are you supporting?. Root, Geant3, Fluka, AliRoot:
E N D
Peter Hristov CERN, March 22, 2006 Installation of the ALICE Software
Packages • Simulation, reconstruction, analysis • Root • Geant3 • Fluka • AliRoot • Grid/distributed computing: VO, etc. • LCG • AliEn
What use cases are you supporting? • Root, Geant3, Fluka, AliRoot: • User installation: CVS download of recommended versions • Installation for Grid productions/applications: tarballs • AliEn: • User installation: AliEn installer • Production: VO installation
Installation: Environment Variables # ROOT export ROOTSYS=/home/mydir/root export PATH=$PATH:$ROOTSYS/bin export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ROOTSYS/lib # AliRoot export ALICE=/home/mydir/alice export ALICE_ROOT=$ALICE/AliRoot export ALICE_TARGET=`root-config --arch` export PATH=$PATH:$ALICE_ROOT/bin/tgt_${ALICE_TARGET} export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ALICE_ROOT/lib/tgt_${ALICE_TARGET} # GEANT 3 export PLATFORM=`root-config --arch` export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ALICE/geant3/lib/tgt_${ALICE_TARGET}
Root Download (CVS) • Login to the ROOT CVS repository if you haven't done it yet: • cvs –d :pserver:cvs@root.cern.ch:/user/cvs login • CVS password: cvs • Download (check out) the needed ROOT version (v5-10-00 in the example): • cvs -d :pserver:cvs@root.cern.ch:/user/cvs co -r v5-10-00 root
Root Configuration: Pythia6 • AliRoot supplies libpythia6.so • This is a modified version from standard for use with ALICE code. • ROOT checks for libPythia6.so and the a common block before the compilation of TPythia6 interface. • AliRoot needs ROOT with TPythia6 to compile successfully. • => Circular dependence
Root Configuration with Pythia6 • Configure and “make” Root: no Pythia6 • “make all-pythia6” in AliRoot (see below) • Create symbolic link between libpythia6.so and libPythai6.so • Re-configure Root: libPythia6.so is found • Do “make” in $ROOTSYS: obtain the “final” version of Root • Do “make” for AliRoot
Root: Compilation, Tests • After the configuration optionally edit MyConfig.mk. For example add there OPT += -g • Do “make; make map” • Do “cd test; make” • Optionally add “.” (dot) to LD_LIBRARY_PATH • Run “stress”, ”stressgeom”, ”stressLinear”, “stressVector”, etc. • Now you have fully operational Root
Installation of Geant3 • Geant3 compiles with minimal problems. Needs Root to make TGeant3 interface • Geant3 must be accessible from $ALICE/geant3 • cd $ALICE • cvs -d :pserver:cvs@root.cern.ch:/user/cvs co -r v1-4 geant3 • cd geant3 • make
Fluka • Register and get the Fluka library from http://www.fluka.org • Unpack the library in $ALICE/fluka
AliRoot • Login to the AliRoot CVS repository if you haven't done it yet: • cvs –d :pserver:cvs@alisoft.cern.ch:/soft/cvsroot login • CVS password: cvs • cd $ALICE • Download (check out) the needed AliRoot version (v4-04-01 in the example): • cvs -d :pserver:cvs@alisoft.cern.ch:/soft/cvsroot co -r v4-04-01 AliRoot • cd AliRoot; make
Questions I • How the tarballs (rpms) are prepared? • By the program librarian. We also include the relevant gcc libraries. • Do you have a common repository of tarballs? • They are registered/stored in AliEn • What different types and categories of tarballs are you supporting? • Linux (SLC3) on Pentium, Itanium, Opteron
Questions II • What optimizations are put in place to minimize the size of files to download? • Removal of the field maps which are not used • Support for concurrent versions? • Yes, stored in AliEn and described in the JDL file • How do you handle the dependences? • Simple Web page with the recommended sets of Root/Geant3/AliRoot tags
Questions III • What tools are used to simplify the task of end-user physicists? • No special tools. The support is via mailing list. Manuals and tutorials recently put in place • Are the users satisfied? What are the problems? – No major problems
Questions IV • Connection with the experiment configuration management system • AliEn packman used to register the tarballs • Plans to use AliEnBITS for nightly builds and preparation of tarballs • What is currently used from what is provided by AA/SPI? • What could be provided to simplify your task?
Package manager (PackMan) • Definition of a Package • Group of libraries and executables that have to be installed in the worker node • Do not require root access to be installed • Defined either by a VO or by a user • Dependencies with other packages • Packages configure the environment variables before job execution • PackMan was supposed to be the EGEE/gLite package manager
(Possible) deployment scenarios Common Packages (ROOT, POOL,..) VO & user Packages Site package cache Worker node cache
Creating a package with PackMan • Creating tar file • tar czf ROOT.tar.gz • Registering the package in the catalogue: • (from the alien prompt) packman define ROOT 5.2.0 file://myhost/ROOT.tar • In the JDL of a job, require the package • Submit a JDL like: Executable=“myExec.”; Packages=“ROOT::5.2.0”; InputFile=….
Additional package definition • There is also user defined metadata about the package. The metadata includes: • Size, dependencies, configuration script, pre- and post-installation scripts, installation script, pre- and post- remove scripts • To define any metadata the user has to : • Create the metadata structure for that directory packman define AliROOT v4.03.04 file:/tmp/aliroot.tar.gz \ Dependencies=‘ROOT:5.2.0’
AliEn BITS: Motivation • Automate the process of building AliEn components • Have a current, up to date, installable binary image of AliEn for different architectures • Better understand the dependencies between the packages • Perform functional and unit tests at the build time and receive alerts immediately (4 different platforms!) • Provide software that works to the end-users
AliEn BITS: Building Environment • Based on the Konstruct framework used to build KDE and dependencies which is itself based on BSD ports system • Each package has a directory in a hierarchy • For each package there is a Makefile defining a set of variables: • Name, version, author, web page • License information • Build and runtime dependencies • Master sites for downloading the package source • Additional patches for AliEn specifics • Checksums for verifying the integrity of the files
Building Process [1] • Get the available Releases • Select the one to be (re)built • Fetch the latest changes for this release • Do a (rsync and) cvs update • Select the packages that have to be rebuilt • Based on modifications received on cvs update • Based on last build status • Always rebuild and test AliEn packages • Determine the build order • Analyze the dependencies between packages
Building Process [2] • For each package • Clean its environment • Fetch source archive from master sites • Generate SLOC statistics • Build, install and run tests • Create the binary image for binary installations • Generate web pages • Build / SLOC / Graph dependencies • Generate the pages for AliEn tests • Cleanup environment • Send notification email when status changes
AliEnBITS: Summary • AliEn BITS is a tool for automatizing big projects build process (AliEn -- 184 pkgs) • Intelligently select packages that have to be rebuilt • Offer different views for packages dependencies - very useful in understanding and managing large projects • Provide SLOC statistics • Save and present relevant logs on components testing • Building and continuous testing is important for delivering high quality software to end-users