slide1 l.
Skip this Video
Loading SlideShow in 5 Seconds..
Automated Tests in NICOS Nightly Control System PowerPoint Presentation
Download Presentation
Automated Tests in NICOS Nightly Control System

Loading in 2 Seconds...

play fullscreen
1 / 4

Automated Tests in NICOS Nightly Control System - PowerPoint PPT Presentation

  • Uploaded on

Automated Tests in NICOS Nightly Control System. Alexander Undrus Brookhaven National Laboratory, Upton, NY 11973. Abstract.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Automated Tests in NICOS Nightly Control System' - finley

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Automated Tests in NICOS Nightly Control System

Alexander Undrus

Brookhaven National Laboratory, Upton, NY 11973


Software testing is a difficult, time-consuming process that requires technical sophistication and proper planning. This is especially true for the large-scale software projects of High Energy Physics where constant modifications and enhancements are typical. The automated nightly testing is the important component of NICOS, NIghtly COntrol System, that manages the multi-platform nightly builds based on the recent versions of software packages. It facilitates collective work in collaborative environment and provides four benefits to developers: repeatability (tests can be executed more than once), accumulation (results are stored and reflected on NICOS web pages), feedback (automatic e-mail notifications about test failures), user friendly setup (configuration parameters can be encrypted in the body of test scripts). The modular structure of NICOS allows plugging in other validation and organization tools, such as QMTest and CppUNIT. NICOS classifies tests according to their granularity level and purpose. The low level structural tests reveal compilation problems, inconsistencies in package configuration, such as circular dependencies, and simple isolated bugs. The results for these three groups of tests are published for each package of the software project. The integrated (or behavioral) tests find bugs at levels of users scenarios and NICOS generates the special web page with their results. The NICOS tool is currently used to coordinate the efforts of more than 100 developers for the ATLAS project at CERN and included in the tool library of the LHC computing project.

CHEP 2004






Tag Collector


Control System







Automatic e-mails

Build Results



NICOS - NIghtly COntrol System

  • NICOS Control System [1] has a modular structure. Each module is responsible for a certain step (such as code checkout, release build, testing, error analysis) and independently described in the NICOS configuration file. This organization allows to plug in build, test, validation, and other external tools.
  • NICOS currently manages the nightly releases of ATLAS software comprising about 1000 packages. It creates the nightly build framework that also uses:
    • ATLAS Tag Collector [2] – web interfaced database application. Developers are able to interactively select the tags from ATLAS CVS repository for the nightly releases
    • CMT [3] configuration management tool that defines the conventions for structuring software releases and describes the package properties, constituents, and dependencies

CHEP 2004 - Alexander Undrus - Automated Tests in NICOS Nightly Control System - page 2


Test Levels and Tools in ATLAS Nightly Builds


Verification of





Coding Standards)


Unit Tests

Integration Tests


  • NICOS Control Tool provides a framework for tests of different manners and levels of focus. The tests are automatically applied to each nightly release.
  • The compilation and linking results are verified automatically and published for each package.
  • Quality Assurance tests verify the descriptions of package properties (in CMT requirements files). The code checking tools are under development and will be added to the nightly tests in the future.
  • Unit tests check individual software components (such as C++ classes) or a collection of components (such as the packages of the ATLAS software release). A helper tool for unit testing of C++ programs, CppUNIT [4], is integrated in CMT environment of ATLAS releases. For testing individual packages the special make target, make check, is provided. It allows to run the test jobs focused on the package functionalities in Athena, ATLAS control framework.
  • Integration tests show that the major systems of the software release work well and communicate with other systems successfully. QMTest [5] tool is used for the ATLAS integration tests organization and validation. It supports regression tests that compare the output of the current test with previous (or known) values. The tests are arranged in suites. The suite usually includes the tests related to the specific system of a release. QMTest saves the test results for further inspection with the graphical or command-line interface.

CHEP 2004 - Alexander Undrus - Automated Tests in NICOS Nightly Control System - page 3


Delivering Test Results to Developers

Precise sw release info in header




Sorting options


for e-mail


Build, QA and Unit tests results

Problems highlighted

NICOS web pages show the results of QA and unit tests for each package of a release. There are two levels of problem seriousness: warning and error. The results for integrated tests and suites are published on a separate web page.

  • References
  • The NICOS home page is
  • S. Albrand “The Tag Collector – A Tool for Atlas Code Release Management”, CHEP04 contribution
  • CMT home page is
  • CppUNIT home page is
  • QMTest home page is

CHEP 2004 - Alexander Undrus - Automated Tests in NICOS Nightly Control System - page 4