Chapter 18 Software Testing Strategies

1. Chapter 18Software Testing Strategies

2. Testing Strategy unit test integration test system test validation test

3. Unit Testing module to be tested results software engineer test cases

4. Unit Testing module to be tested interface local data structures boundary conditions independent paths error handling paths test cases

5. Unit Test Environment driver interface local data structures Module boundary conditions independent paths error handling paths stub stub test cases RESULTS

6. Integration Testing Strategies • Options: • • the “big bang” approach • • an incremental construction strategy

7. Top Down Integration A top module is tested with stubs B F G stubs are replaced one at a time, "depth first" C as new modules are integrated, some subset of tests is re-run D E

8. Bottom-Up Integration A B F G drivers are replaced one at a time, "depth first" C worker modules are grouped into builds and integrated D E cluster

9. Sandwich Testing A Top modules are tested with stubs B F G C Worker modules are grouped into builds and integrated D E cluster

10. High Order Testing validation test system test alpha and beta test other specialized testing

11. Debugging: A Diagnostic Process

12. The Debugging Process test cases results new test cases regression tests suspected causes Debugging corrections identified causes

13. Debugging Effort time required to diagnose the symptom and determine the cause time required to correct the error and conduct regression tests

14. Symptoms & Causes symptom and cause may be geographically separated symptom may disappear when another problem is fixed cause may be due to a combination of non-errors cause may be due to a system or compiler error cause may be due to symptom assumptions that everyone cause believes symptom may be intermittent

15. Consequences of Bugs infectious damage catastrophic extreme serious disturbing annoying mild Bug Type Bug Categories: function-related bugs, system-related bugs, data bugs, coding bugs, design bugs, documentation bugs, standards violations, etc.

16. Debugging Techniques brute force / testing backtracking induction deduction

17. Debugging: Final Thoughts 1. Don't run off half-cocked, think about the symptom you're seeing. 2. Use tools (e.g., dynamic debugger) to gain more insight. 3. If at an impasse, get help from someone else. 4. Be absolutely sure to conduct regression tests when you do "fix" the bug.

18. Testing approaches • Architectural validation • Top-down integration testing is better at discovering errors in the system architecture • System demonstration • Top-down integration testing allows a limited demonstration at an early stage in the development • Test implementation • Often easier with bottom-up integration testing • Test observation • Problems with both approaches. Extra code may be required to observe tests

19. Interface testing • Takes place when modules or sub-systems are integrated to create larger systems • Objectives are to detect faults due to interface errors or invalid assumptions about interfaces • Particularly important for object-oriented development as objects are defined by their interfaces

20. Interfaces types • Parameter interfaces • Data passed from one procedure to another • Shared memory interfaces • Block of memory is shared between procedures • Procedural interfaces • Sub-system encapsulates a set of procedures to be called by other sub-systems • Message passing interfaces • Sub-systems request services from other sub-systems

21. Interface errors • Interface misuse • A calling component calls another component and makes an error in its use of its interface e.g. parameters in the wrong order • Interface misunderstanding • A calling component embeds assumptions about the behaviour of the called component which are incorrect • Timing errors • The called and the calling component operate at different speeds and out-of-date information is accessed

22. Interface testing guidelines • Design tests so that parameters to a called procedure are at the extreme ends of their ranges • Always test pointer parameters with null pointers • Design tests which cause the component to fail • Use stress testing in message passing systems • In shared memory systems, vary the order in which components are activated

23. Stress testing • Exercises the system beyond its maximum design load. Stressing the system often causes defects to come to light • Stressing the system test failure behaviour.. Systems should not fail catastrophically. Stress testing checks for unacceptable loss of service or data • Particularly relevant to distributed systems which can exhibit severe degradation as a network becomes overloaded