System Testing in a Hurry: Practical Strategies for Last-Minute Testing Projects

1. System Testing In a Hurry British Computer Society Specialist Interest Group In Software Testing 6th December 1999 Graham Thomas Wall Street Systems

2. Abstract “Hi Graham, we have a project which finished development last Friday and starts system testing today (Monday).” “Unfortunately all of our testers are pre-occupied with Year 2000 projects and we can’t spare them, so we thought that we would ask the development team to carry out the system testing. “ “Can you talk to the team for a couple of hours and tell them everything they need to do ‘system testing in a hurry’? . . . .How long do you need to prepare? . . . .You have a day!”

3. Contents • The Presentation • Testing Progress • Feedback from Development • A surprising conclusion

4. The Presentation • Planning • Strategy • Test Design • Techniques • Monitoring • Feedback

5. People Accommodation Equipment PC’s, printers, modems Support; Development Operations Business Environment provision CICS Regions DASD (Disk space) Requirements How many Skills Availability (courses, holidays, leavers, joiners, training) Need to generate a Plan Requires a Test Manager role Planning

6. Strategy Unit System Accp. Regulatory Trials Retro-Fit Integration Testing Lifecycle

7. Strategy Requirements Acceptance Analysis System Design Integration Build Unit Testing V Model

8. Analysis System Strategy Requirements Acceptance Functional Design Spec. Design Integration Build Unit Testing V Model

9. Function (logical) Condition Script Network Test Data (physical) Case Test Design Functional Design Spec. Accp. Criteria Test Methodology

10. Techniques • Boundary Value Analysis • Equivalence Partitioning • Error Guessing 0 5 16 60 Free Child Adult OAP Tube Ticket Example

11. Step Action Test Case Ref. Exp. Results Act. Results Scripting • Write high level scripts to save time • Make scripts object based e.g. • Select Contract, Print Contract • Include reference to test case for traceability Script Template

12. Execution • Common Test Faults • The Test Data • The sequence of instructions (script) • Less likely to be a code problem !!! • Problem Recording • Record the scene of the crime • Don’t waste time trying to diagnose fault • Just record instance of fault via Post-it note

13. Monitoring • Don’t get bogged down measuring • Test Case generation • Script generation • Function coverage • Recommended 2 simple ways to monitor • S - Curve • Outstanding Faults

14. S - Curve

15. Faults Outstanding

16. Presentation Feedback • Developers • Not very structured • Management • Relieved to have a structured testing method and limited support • Client Testing Management • Very keen to adopt the proposed method

17. Interlude

18. Testing Progress • Assigned 1 member of testing team to assist project on a part time basis • Developers found it “a bit boring” generating the test cases • Development Team Leader welcomed a structured testing method • All went very quiet . . .

19. Feedback from Development • The testing support was welcomed • The ‘method’ hung together • The test manager was in control of testing • Actually gave the developers a different perspective on testing • Took the pressure off what could have been a disastrous project

20. A Surprising Conclusion 4 • The project was successful • Testing finished on time • Good quality product handed-over • Minimal impact to Y2K program • But was it really? • “We now test the way Graham showed us” No early testing feedback No advanced test planning No metrics No repeatability 8

21. Start Testing Early Requirements Acceptance Analysis System Design Integration Build Unit Testing V Model

22. Contact Details Graham Thomas Testing Manager Wall Street Systems Phone: 020 7827 0430 e-mail: graham.thomas@wallstreetsystems.com