ARC tools for revision and nightly functional tests

1. ARC tools for revision and nightly functional tests JozefCernak, MarekKocan, Eva Cernakova (P. J. Safarik University in Kosice, Kosice, Slovak Republic)

2. Outline • Testing strategy in ARC • ARC-EMI testing tools • Build testing • Nightly (every night) • Revision (after each commit of developers) • Automatic functional testing of nightly builds • Future plans • Increasing number of automatic functional tests • Automatic performance testing • Conclusion Use Insert Header & Footer to set this field

3. Testing practice in ARC • Several roles in project: developers, testers and ARC release manager • Adopted tests: • Unit tests(developers) • Build tests (testers) • Revision (Kosice) • Nightly (Copenhagen) • EMI-RC (ETICS) • Functional tests (testers, developers) • Manual • Automatic (Kosice) • Regression testing (testers, developers) • Performance testing (testers) • Manual • Automatic in development phase • Large scale testing (in preparation phase) Use Insert Header & Footer to set this field

4. ARC-EMI testing tools Relation between ARC and EMI testing tools Use Insert Header & Footer to set this field

5. Block diagram of automatic revision and Functional tools Simple database server with PHP and Python scripts Use Insert Header & Footer to set this field

6. Technical details • Infrastructure (local cluster) • 2 SUN 2200M2 servers (2x4GB RAM, 250GB disk, AMD 64 bits) • 2 desktops (2x 1GB RAM, 500GB disk 32 bits) • 1 MAC mini (2GB RAM, 160GB, 32 bits) • Software • MySQL, PHP, Python • Code is free • Distributed data processing • SVN Use Insert Header & Footer to set this field

7. Revision automatic testing • Features • Supported are several platforms of OS: • CentOS (32,64), • Debian(32,64), • and MAX OS (64) • Detailed identification of errors during building process • Autogen • Configuration • make • make dist • Build Use Insert Header & Footer to set this field

8. …continued. • User friendly interface to define search criteria • Daily • Weakly • Monthly • Custom • Revision number, etc (see next examples) • Direct connection with SVN • Warning messages for developers if code is broken • CCCC metric • Quick mechanisms to identify breaking event Use Insert Header & Footer to set this field

9. Example of www interface for automatic testing tools Home page: http://arc-emi.grid.upjs.sk/ (Revision database is slow) Use Insert Header & Footer to set this field

10. Several criteria to search results of revision testing Use Insert Header & Footer to set this field

11. Example of the results of daily revisions Use Insert Header & Footer to set this field

12. Automatic functional tests • Features • Automatic download code from SVN based on several criteria: • trunk, • revision code, • nightly builds • Automatic build • Automatic deployment • User friendly interface for: • submitting proposals of test cases • to search the results of functional tests • Tests are grouped into two main groups: • functional tests of server, • functional tests of client. Use Insert Header & Footer to set this field

13. Interface to enter search criteria Use Insert Header & Footer to set this field

14. The results of automatic functional tests Use Insert Header & Footer to set this field

15. Functional test scenaria • client_arcsub_to_arex_gridmap_simplejob_jsdl • Tries to submit a simple JSDL job to a grid-map secured A-REX service. • client_arcsub_to_grid-manager_gridmap_simplejob_jsdl • Tries to submit a simple JSDL job to a grid-map secured ARC0 CE. • client_arcsub_to_GIIS_simplejob_jsdl • Submission of a simple JSDL job to a GIIS service. • client_arcsub_to_ISIS_simplejob_jsdl • Submission of a simple JSDL job to an ISIS cloud. • client_arcsub_random_broker_isis_simplejob_jsdl • Submission of a simple JSDL job to an ISIS cloud using random broker. • client_arcsub_random_broker_giis_simplejob_jsdl • Submission of a simple JSDL job to an GIIS cloud using random broker. • client_arcsub_fastestqueue_broker_isis_simplejob_jsdl • Submission of a simple JSDL job to an ISIS cloud using fastest queue broker. • client_arcsub_fastestqueue_broker_giis_simplejob_jsdl • Submission of a simple JSDL job to a GIIS service using fastest queue broker. • Failed • There was some error before we managed to start testing. The error could have been in code retrieval, installation or somewhere else. Use Insert Header & Footer to set this field

16. Automatic performance tests • Requirements: • several servers and clients • Monitoring of several distributed processes: • resources: • Memory • CPU • Network usage • Performance • Scalability • Number of concurrent requests • Reliability of services: • Ratio between successful tasks and total tasks. Examples of tasks: job submission, file transfer and etc. Use Insert Header & Footer to set this field

17. … continued. • Proposed infrastructure: • Grid servers and clients, we plan to use existing infrastructure • Communication layer • XML-RPC client-server architecture • Central database server • Common Python library on each server and client (for example start stop service) • Central control program Use Insert Header & Footer to set this field

18. Conclusions • Automatic testing covers: • Build process (ARC, ETICS), • Functional, • Performance tests (in progress). • Utilization of automatic test tools increases probability to find weak part of the software. On the other hand, manual tests are important. Use Insert Header & Footer to set this field

19. References • Testing in ARC • http://wiki.nordugrid.org/index.php/Testing • Revision tests • http://download.nordugrid.org/builds/ • http://arc-emi.grid.upjs.sk/revisionTests.php • Functional tests • http://arc-emi.grid.upjs.sk/functionalTests.php • Code: • http://svn.nordugrid.org/trac/workarea/browser/ARCTestScriptshttp://svn.nordugrid.org/trac/workarea/browser/ARCTestScripts Use Insert Header & Footer to set this field

20. Thank you EMI is partially funded by the European Commission under Grant Agreement INFSO-RI-261611 Use Insert Header & Footer to set this field