Component Testing

1. CBSE 2005 Component Testing Component-based Software Engineering Christian Schanes LIACS – Leiden University Fall 2005

2. Introduction • Importance of testing and problems • Testing methods • Testing activities • Component based software testing • Component testability • Test automation Component Testing

3. Case study: Ariane 5 (1996) • Ariane 5 explodes during startup • Reused software (component) • Software was successful for Ariane 4 • Problem with new hardware • Analyses shows • NO software testing error • Software reuse error (Sommerville 2004) Component Testing

4. Testing problems • Costs of testing • About 30%-50% • Last phase in project • Dijkstra: “Testing can only show the presence of errors, not their absence” (Vincenzi et al. 2003, Gao 1999) (Sommerville 2004) Component Testing

5. Characteristics • Test effectiveness • Find smallest test set which finds most faults • Test adequacy • How much was tested • Is it sufficient (Vincenzi et al. 2003) Component Testing

6. Test strategies • Validation testing • Software fulfills the requirements • Successful when system correct • Demonstrate that software meets requirements • Defect testing • Discover faults or defects • Successful when system incorrect • Use problematic inputs (Sommerville 2004) Component Testing

7. T est T est T est T est cases da ta r esults r epor ts Design test Pr epar e test R un pr o g r am Compar e r esults cases da ta with test data to test cases Software testing process (Sommerville 2004) Component Testing

8. Black box testing • Source code not used • Behavior determined with inputs and outputs • Input data • Equivalence partitions • Output data (Sommerville 2004) Component Testing

9. 3 1 1 4 7 1 0 Less than 4 Betw een 4 and 1 0 Mor e than 1 0 Equivalence partitions • Different classes with common characteristics • e.g. • Positive/negative numbers • Valid/invalid inputs • Identify test cases for each partition (Sommerville 2004) Component Testing

10. White box/Structural testing • Source code/structure known • Better input data • Execute each statement • Path testing • Execute each independent path • Combinations not tested • When a loop -> infinite combinations • Therefore errors can still occur (Sommerville 2004) Component Testing

11. Control Flow Graph // Statement 1 while (bottom <= top) { if (elemArray[mid]==key) break; else if (elemArray[mid]<key) // Statement 6 else // Statement 5 // Statement 7 return; } // Statement 8 // Statement 9 (Sommerville 2004) Component Testing

12. Testing activities • Unit testing/Component testing • Performed by developers • Goal: discover defects • System testing • Performed by independent testing team • Integration testing • Release testing (Sommerville 2004) Component Testing

13. Unit testing • Test a unit independent • Driver: inputs data to the test unit • Stubs: simulates behavior of used units (Vincenzi et al. 2003) Component Testing

14. Interface testing • Access component through interface • Test if interface behaves as specified • Errors • Interface misuse • e.g. wrong number of parameters • Interface misunderstanding • Timing errors • Real time applications (Sommerville 2004) Component Testing

15. Integration testing • Integration • Building system from components • Write glue code • Check if • Components work together • Called correctly • Transfer right data at right time • Difficult to locate errors • Use incremental integration (Sommerville 2004) Component Testing

16. Incremental integration testing • Test each time a new component is integrated • Regression testing • Basic principle of eXtreme Programming • Error can occur • In the new component • In tested integrated system • e.g. new component breaks systems • Not possible without tools • e.g. JUnit (Sommerville 2004) Component Testing

17. Incremental integration testing (Jeffries 1999) Component Testing

18. Release testing • Goal: meets requirements • Functional • Non-functional • Usually black-box testing • Problems • Difficult to understand system behavior • Difficult error isolation, debugging • High cost on performance testing • No information provided by vendor (Sommerville 2004) Component Testing

19. Unit testing (Vincenzi et al. 2003) Component Testing

20. Integration testing (Vincenzi et al. 2003) Component Testing

21. System testing (Vincenzi et al. 2003) Component Testing

22. Component developer/provider • Single component • No context knowledge • Testing with source code • High costs when find error after release • see Ariane 5 • Testing increases user’s confidence • Reduces costs for users (Vincenzi et al. 2003, Gao 1999) Component Testing

23. Component user • Component based software • Quality based on used components • No knowledge about implementation and testing • No source code • No test reports/coverage • More functionality as required • Should all be tested • How get coverage (Vincenzi et al. 2003, Gao 1999) Component Testing

24. Component testability • Two aspects • Observability • Traceability • Incoming and outgoing interfaces • Controllability • Inputs/Outputs • Operations • Behaviors • Conflict with component definition • No externally observable state (Vincenzi et al. 2003, Gao 1999) (Szyperski 2002) Component Testing

25. Component testability • Component presentation • Documentation, specification • Component test suite • Suites which can be used by user • Configuration management • Manage test resources • Component test support • Interfaces • Component test bed (Gao 1999) Component Testing

26. Component testability • What we would need • Tracking interface • Interface • To execute unit tests • Check test result • Report errors • Built in tests • Standardized (Gao 1999) Component Testing

27. Possible solutions • Use the byte code format • e.g. Java and .NET • Insert a layer to perform test and coverage • Auto-testable components • Built in testing capabilities • Enable/Disable testing (Vincenzi et al. 2003) Component Testing

28. Test automation • Provides regression testing • Easy to test complete code • Supporting tools • Test case generators • Drivers • Stubs • Simulators • Component test bed (environment) • Reporting (Vincenzi et al. 2003) Component Testing

29. Conclusion • Component reuse benefits will be maximized when components fulfill • Clients requirements in terms of quality and reliability • Easily tested in client application context (Vincenzi et al. 2003) Component Testing

30. References • Gao, J. (1999), Testing Component-Based Software. • Jeffries, R.E. (1999), eXtreme Testing. Software Testing and Quality Engineering. • Sommerville, I. (2001), Software Engineering, Pearson Education Limited, USA, ISBN: 0-201-39815-X, 6th edition • Vincenzi, A.M.R., Maldonado, J.C., Delamaro, M.E., Spoto, E.S. and Wong W.E. (2003), Component-Based Software: An Overview of Testing. In: Component-Based Software Quality, pp. 99-127. Component Testing

31. Questions? Component Testing