1 / 15

Tool for Automated Testing

Tool for Automated Testing. P. Sivera European Southern Observatory. Garching – January 2007. ESO. tat. tat - Tool for Automated Testing run multiple tests with one command produce reproducible output automatic setup compare output against reference file(s). How tat works.

ppoulin
Download Presentation

Tool for Automated Testing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Tool for Automated Testing P. Sivera European Southern Observatory Garching – January 2007 ESO

  2. tat • tat - Tool for Automated Testing • run multiple tests with one command • produce reproducible output • automatic setup • compare output against reference file(s) Tool for Automated Testing

  3. How tat works • Looks for <mod_name>/test/TestList • Redefine LD_LIBRARY_PATH, PYTHONPATH, PATH • Executes “make clean all” • Source a tcl script for additional env vars • Executes a prologue script • Executes the tests suite and compare output with the reference • Executes an epilogue script • Executes “make clean” Tool for Automated Testing

  4. TestList directives • SOURCE <tcl_script_name> • PROLOGUE <prog_name> [<arg>] • <test_program_name> (1 or more) • EPILOGUE <prog_name> [<arg>] The 3rd directive is the only mandatory one Tool for Automated Testing

  5. TestList syntax • Blank lines are allowed • Comment lines start with “#” • A test directive can be split on multiple lines => continuation character is “\” Tool for Automated Testing

  6. Test case syntax • <testname> • <testNum> <testName> <proc 1> [<proc 2>..<proc n>] Ex: 1mySimpleTest checkCompStatus 2myComplexTest sendMsg checkCompStatus setParam makeCrash 3myComplexTest sendMsg“@SLEEP 10”checkCompStatus“@SLEEP 10”“setParam3” “@SLEEP 10”makeCrash Tool for Automated Testing

  7. Cookbook - 1 • getTemplate for MODROOT • getTemplate for Makefile under test • getTemplate for script/program • update Makefile • write/try the script/program • prepare TestList Tool for Automated Testing

  8. Cookbook – 2 • create reference file: $ tat –g testscript Reference file acstHello.sh.ref generated Reference file generated. • run the test: $ tat testscript TEST acstHello.sh PASSED. PASSED. Tool for Automated Testing

  9. How to deal with the CDB • Create directory: test/CDB • Put your CDB files • Redefine environment variable: ACS_CDB=`pwd` (in the SOURCE script) Tool for Automated Testing

  10. How to run tat • Run as • tat(make all, source, prologue, test suite, compare with ref, epilogue, make clean) • or • tat makeEnv(make all, source, prologue) • tat or tat <testitem> • tat cleanEnv(epilogue, make clean) Tool for Automated Testing

  11. Some options • -v to run tat in verbose mode • -nc (not clean) to run “make all” and not “make clean all” • -f <filename>to use another different testlist besides the TestList Tool for Automated Testing

  12. Files and log files • TestList.grep → grep –v –f TestList.grep <output> • TestList.sed → sed –f TestList.sed <output> • test/tatlogs/run$$ → .out, out.orig and .diff (.diff is the unix diff between .ref and .out) Tool for Automated Testing

  13. Example of SOURCE • SOURCE needs a tcl scriptunder test dir • Add a line in TestList.lite: SOURCE <tclscript_name>.tcl • Write the file: set env(VARIABLE) WHATEVER Tool for Automated Testing

  14. Example of prologue #! /bin/bash acsStart > $ACS_TMP/acsStart.log 2>&1 & sleep 3 acsStartContainer bilboContainer >$ACS_TMP2/bilbo.log 2>&1 & sleep 3 See standard prologue: $ACSROOT/bin/acsutilTATPrologue Tool for Automated Testing

  15. Example of epilogue #! /bin/bash acsStopContainer bilboContainer >bilboStop.log 2>&1 sleep 3 acsStop >acsStop.log 2>&1 sleep 3 See standard epilogue: $ACSROOT/bin/acsutilTATEpilogue Tool for Automated Testing

More Related