1 / 15

The Computing Environment

The Computing Environment. Outline. Software Management CVS CMT Tag-Collector Savannah Data Structures Grid & Local cluster Athena Projects & Packages Versions Python / C++ Algorithms & Tools HyperNews

mbonds
Download Presentation

The Computing Environment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Computing Environment

  2. Outline • Software Management • CVS • CMT • Tag-Collector • Savannah • Data Structures • Grid & Local cluster • Athena • Projects & Packages • Versions • Python / C++ • Algorithms & Tools • HyperNews • Atlas Computing twiki page:https://twiki.cern.ch/twiki/bin/view/Atlas/AtlasComputing

  3. CVS - Concurrent Version System • The software environment in Atlas (and in the whole HEP community) is always changing. • CVS is a program the keep track of the evolution of the software. That way one can always return to a previous stable version, or advance to an exciting new version. • The bookkeeping of the version is done using “tags”. Each version is associated with a specific tag, and it is identified by it. • ViewVC: http://atlas-sw.cern.ch/cgi-bin/viewcvs-atlas.cgi/offline/

  4. CMT - Configuration Management Tool • An environment for packaged based development framework. • The framework is divided into packages. Each package has a specific goal, and compiled into its own library. • The CMT provide a simple way for configuring and building (gmake) the environment using simple configuration files (requirements) • It is installed over the CVS system, and the check-out/in of the packages is done inside the cmt environment. • LXR: http://alxr.usatlas.bnl.gov/Doxygen: http://atlas-computing.web.cern.ch/atlas-computing/links/nightlyDocDirectory/allpackages.html

  5. Tag Collector • A web interfaced database application which is an essential part of the management of ATLAS software releases. • Sites:https://twiki.cern.ch/twiki/bin/view/Atlas/TagCollectorInAtlashttp://atlastagcollector.in2p3.fr

  6. Savannah • Savannah is a tracking system used to track bugs and tasks • Sites:https://twiki.cern.ch/twiki/bin/view/Atlas/SoftwareDevelopmentWorkBookSavannahhttps://savannah.cern.ch/search/?type_of_search=soft&words=%%%&type=7

  7. Data Structures • HepMC – A C++ event record for Monte-Carlo Generators. http://lcgapp.cern.ch/project/simu/HepMC/ • BS / PRD(PrepRawData) / RDO (Raw Data Object) – Low level encoding of detector information. Has the complete information of the hits in the detector, but has no reconstructed information. Size ~2MB/evt • ESD (Event Summary Data) – Contains reconstructed objects together with hit clusters in each detector. Size ~500kB/evt • AOD (Analysis Object Data) – A reduced sized files. Contains only reconstructed objects and their associated hit clusters and tracks in the detector. Size ~100kB/evthttps://twiki.cern.ch/twiki/bin/view/Atlas/AODClassSummary • DnPD (Derived Physics Data) – Reduced size file. Contains derived objects (like Z/H) and “pre-analysis step” like overlap-removal, Slimming, thinning etc. Each group defines their own DPDs. Most of the DPDs are usually pool files. However the last DPD can be flat ntuple. (will be discussed on the advanced tutorial) • TAG – List of identifiers are extracted to index the events. Size ~1kB/evt (?) • Formats – • Ntuple – The well known flat ntuple. Several enhancement were made to include back navigation to AOD (Athena Aware Ntuple – AAN). Advantage – very fast analysis. Disadvantage – not very flexible. • Pool files – Containers of C++ objects. Advantage – very flexible. Disadvantage – complex and slow.

  8. Grid & Local Cluster • Local Cluster (TAU, Tech, WIS): • ~140 cores @ a site • ~36TB @ a site • Half of the resources are for the Grid, and half is for local use. • Use it for small production, simple analysis and small tasks. • Grid: • Institutes all over the world share their computing power and storage facilities in a big cloud called the Grid (see later talk by Lorne) • Use it to run your analysis on real data or centrally produced MC, heavy analysis jobs and long tasks that can be divided into parallel jobs

  9. Athena • See: https://twiki.cern.ch/twiki/bin/view/Atlas/WorkBookAthenaFramework • A framework for Atlas software • A skeleton of an application into which developers plug in their code • Provides most of the common functionality and communications between different components • Embodies the underlying design and philosophy of the software • Encourages a common approach • Factors out common functionality for re-use • Athena is a hybrid beast of python and C++. Its engine is python, and it is controlled by python scripts (also known as “JobOptions”), and it is capable of running C++ algorithms and tools. • Divided into projects and packages. Each is associated with an overall release tag. • Atlas Work Bookhttps://twiki.cern.ch/twiki/bin/view/Atlas/WorkBook

  10. Athena Projects • https://twiki.cern.ch/twiki/bin/view/Atlas/WorkBookPackages • Some of the projects are: • AtlasAnalysis - This contains Tools, Algorithms and Services associated with physics analysis, monitoring and the event display. It depends upon the AtlasTrigger project. • AtlasCore - This contains core components and services (e.g. Athena and StoreGate). It depends upon the Gaudi framework project, and the LCGCMT project which provides links to LCG supported external software packages such as POOL and ROOT. • AtlasOffline - This is the project that provides the default entrypoint into the project hierarchy for interactive use. It depends upon the AtlasAnalysis and AtlasSimulation projects via explicit dependencies. By default it contains essentially no packages, but is a placeholder whereby bugfix releases may be created. The release number of this project is the ATLAS offline release number. • AtlasProduction - This is a top-level project for production use of the ATLAS offline release. It depends upon all other projects . Only a few packages are assigned to this project, in particular those that allow global testing of the complete release in a production enironment (e.g. KitValidation). By convention the release number of this project is the same as the ATLAS offline release number, although a forth digit (i.j.k.l) denotes a patch that is applied for production use only. See also WorkingWithPatchedReleases

  11. Athena Package • A package is a building block in Athena. • Each package is created under a specific project, and has a certain goal. • A package has the following directory structure: • cmt : contains requirements + setup files. This is where you build your code. • <PackageName> : contains the C++ header files. • src : contains the C++ source files. • share : contains job option (.py) files. • run : where you run your jobs. • python : any additional python source.

  12. Athena Versions • Athena has versions for releases and nightly. • Nightly – Each night pre-defined versions of the code are build and the latest development is tested. The nightlies are rotated between rel_0 – rel_6, so you can always return to up to a seven days old nightly. Nightlies can be installed on the local cluster if necessary (I think???), however – I have no experience with it, and I think it is better to work with the local builds on lxplus.https://twiki.cern.ch/twiki/bin/view/Atlas/NightlyBuildSummary • Releases – from time to time a new “stable” version of Athena is out to the market. In order to see the content of each release take a look at:http://atlas-computing.web.cern.ch/atlas-computing/projects/releases/status/

  13. Python / C++ • Athena is handled in a python environment using jobOptions python scripts. • Each C++ packages is compiled and a dynamic link library is created (.so files). The C++ inherits from a common interface (Tool, Algorith,…) and is represented using a python class defined automatically in genConf • “some” things will become clearer after the tutorial later today

  14. Algorithms & Tools • Algorithm – Inherits from Algorithm. Has: initialize, execute and finalize. The method execute is called once per events • Tool – Inherits from AlgTool. Can be instantiated many times. • Service – Provides common services like StoreGate, message services, histograms etc.

  15. How To Get Help • 1st advantage in big collaborations – Someone knows the answer so: • Ask your friend • 2nd advantage in big collaboration – there is always someone that is dumber than you. Search for his dumb questions on the Hypernews, Savannah or google it.https://hypernews.cern.ch/HyperNews/Atlas/top.pl • If you can’t find the answer, put your question on the Hypernews. Usually, the problem will be solved.

More Related