Theory and Practice of Software Testing

1. Theory and Practice of Software Testing Chapter 13 (Pressman) Software Testing Strategies

2. Testing To check the quality of something by performing experiments in controlled way

3. Definition Myers “Testing is the process of executing a program with the intent of finding errors.” [ISO] “Testing is a technical operation that consists of the determination of one or more characteristics of a given product, process, or service according to a specified procedure.” [Pol] “Testing is a process of planning, preparation and measurement aimed at establishing the characteristics of an information system and demonstrating the difference between the actual and the required status.”

4. spec construction accept exploit & use destroy deliver Why are we Testing ? • Testing as measurement for software quality • Software quality as a measure to predict future operation: • reliability, dependability • expected value, expected costs, probability of failure • Risk • probability of failure  expected cost of failure systemtest acceptancetest quality?test it !

5. A Strategic Approach to SW Testing Verification and validation (V&V) • Verifications: Are we building the product right? • Validation: Are we building the right product?

6. More on V & V ValidationThe process of checking whether the system being built is what the customer expects or needs VerificationThe process of checking any deliverable of the software process meets defined needs or requirements

7. Verification: Formal manipulation prove properties performed on the model Testing: experimentation show error Performed on actual system Verification vs. Testing Verification is only as good as the validity of the model on which it is based Testing can only show the presence of errors, not their absence

8. Organizing for SW Testing • A software testing strategy for conventional software architectures • Spiral model • V model

9. System Testing validation Testing Integration Testing Unit Testing Code Design Requirements System eng. SW Testing Strategy : Spiral Model

10. V-Model

11. Types of Testing Level of detail system integration module unit Accessibility security white box black box robustness performance usability reliability functional behaviour Characteristics

13. Unit testing • interface, • local data structures, • boundary conditions, • independent paths, • error handling

14. Unit Testing Process From Textbook: Ian Sommerville, Software Engineering, 6th Edition

15. Goals of Unit Testing • Verify code against interface and functional specifications • Check both “good” and “bad” inputs against design and code implementation • Check all possible paths of execution through code

16. Integration testing • Tests complete systems or subsystems composed of integrated components • Integration testing should be black-box testing with tests derived from the specification • Main difficulty is localising errors • Incremental integration testing reduces this problem

17. Incremental integration testing

18. Integration testing strategies • Top-down integration • Bottom-up integration • Regression testing • Smoke testing • An initial set of tests that shows if a new version of the application performs well enough for further testing • Used for time-critical projects • Assess the project on a frequent basis

19. Approaches to Integration Testing • Top-down testing • Start with high-level system and integrate from the top-down replacing individual components by stubs where appropriate • Depth-first vs. breadth-first • Bottom-up testing • Integrate individual components in levels until the complete system is created • In practice, most integration involves a combination of these strategies

20. Top-down testing

21. Bottom-up testing M

22. Top-down Vs. Bottom-up • Architectural validation • Top-down integration testing is better at discovering errors in the system architecture • System demonstration • Top-down integration testing allows a limited demonstration at an early stage in the development • Test implementation • Often easier with bottom-up integration testing • Test observation • Problems with both approaches. Extra code may be required to observe tests

23. Guideline for Integration testing • Integrate the component that implement the most frequently used functionality • Perform regression testing for existing and new functionality

24. Validation testing • Validation test criteria • All functional, • all nonfunctional • Configuration review (auditing) • Ensures that all system elements have been properly documented • Alpha/beta testing

25. System testing (1) • Recovery testing • Forces the software to crash and shows the adequacy of recovery • E.g., use the back button after submitting the credit card information • Security testing • Tests adequacy of system protection mechanisms • E.g., how does the website authenticate users?

26. System testing (2) • Stress testing/load testing • Subjecting the system to varying and maximum loads to evaluate the resulting performance • E.g., how can you overwhelm the system? • Performance testing • Used to evaluate system performance under normal and heavy usage • E.g., how long should a task take?

27. More on Stress testing • Exercises the system beyond its maximum design load. • Stressing the system often causes defects to come to light • Stressing the system test failure behaviour. • Systems should not fail catastrophically. • checks for unacceptable loss of service or data • Particularly relevant to distributed systems • exhibit severe degradation as a network becomes overloaded