8. UNIT TESTING

1. 8. UNIT TESTING

2. Software Engineering Roadmap: Chapter 8 Focus Identify corporate practices Test units (parts) separately - use implementations - apply discipline - gain coverage Plan project Maintain Analyze requirements Integrate & test system Design Test units Implement Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

3. Learning Goals of This Chapter • Understand meaning of unit testing • Distinguish black box vs. white box testing • Attain proper test coverage • Learn a testing standard • Inspect a unit test plan Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

4. 1. Introduction to unit testing

5. Golden Rules of Testing Goal of testing: maximize the number and severity of defects found per dollar spent … thus: test early Limits of testing: Testing can only determine the presence of defects, never their absence • use proofs of correctness to establish “absence” Who should test: Someone other than the developer. • Why? Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

6. Testing: the Big Picture 3. System tests Include use-cases 2. Integration tests OO: Packages of classes Module combination 1. Unit tests Combinations of methods in class Module Methods Function

7. Unified Process Jacobson et al: USDP Requirements Analysis Design Implemen- tation Test Inception Elaboration Construction Transition Unit Tests Integration tests ... System tests Prelim. iterations Iter. #1 Iter. #n Iter. #n+1 Iter. #m Iter. #m+1 Iter. #k ….. …..

8. RoadMap for Unit Testing Requirements 1. Plan for unit testing Identify largest trouble spots Detailed design Unit test plan 2. Design test cases and acquire test I/O pairs Generate I/O pairs (often products of prior testing) Test set 3. Execute unit test Test results Code under test IEEE, 1986

9. 2. Test types

10. Input determined by... Black-, Gray-, & White-box Testing Result Actual output compared with required output Black box … requirements Gray box As for black- and white box testing … requirements & key design elements White box Confirmation of expected behavior …design elements Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

11. Black-box testing

12. Equivalence Partitioning • Input data and output results often fall into different classes where all members of a class are related • Each of these classes is an equivalence partition where the program behaves in an equivalent way for each class member • Test cases should be chosen from each partition

13. Equivalence Partitioning

14. Test Input Possibilities Infinitely many illegal values: choose a finite sample. principal $100 $100M 20% Infinitely many legal values: choose a finite sample. inflation estimate 25% interest rate 1% 0% Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

15. Test Input Partitioning and Boundaries An illegal region principal Boundaries $100 $100M 20% inflation estimate Range of valid inputs 25% interest rate 1% 0% Equivalence partitions Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

16. Testing Ranges: Elementary Cases 1. within range 2. at the boundaries of the range 3. outside the range (“illegal”) range Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

17. Equivalence Partitioning – Example • Requirement: The system must accept a 5-digit integer between 10000 and 99999 and perform various functions on the values based on the following equivalence partitions: <10000, 10000 - 99999 and >= 100000 • Which test cases should be chosen? Consider the boundaries of these sets … 00000, 09999, 10000, 99999, 100000, <max bin>

18. Equivalence Partitions 9 9 9 9 1 0 0 0 0 0 1 0 0 0 0 5 0 0 0 0 9 9 9 9 9 L e s s t h a n 1 0 0 0 0 B e t w e e n 1 0 0 0 0 a n d 9 9 9 9 9 M o r e t h a n 9 9 9 9 9 I n p u t v a l u e s

19. White Box Testing • Every statement of code should be covered by at least one test • However, this is not sufficient since the correct opeation of a unit of code depends upon sequences of statements

20. Covering Every Statement is Not Sufficient (Myers) Required program u>1 and v==0 Yes x = x/u No u==2 or x>0 Yes ++x No

21. Covering Every Statement is Not Sufficient (Myers) • Code attempt to implement flowchart • if( (u>1) && (v==0) ) (1) • x = x/u; (2) • if( (u==2) || (x>3) ) (3) • ++x; (4) • u=2, v=0 and x=3 • executes every line (1) - (4) • gives the correct output x= 2.5 • However, line (3) is wrong • SO THIS IS NOT A THOROUGH TEST Required program u>1 and v==0 Yes x = x/u No u==2 or x>0 Yes ++x No

22. Paths to be Checked Parameter & settings make sense? N Y Set _name to “defaultName" Parameter name too long? N Y Set _name to parameter Truncate name

23. Paths to be Checked Parameter & settings make sense? N Y Set _name to “defaultName" Parameter name too long? N Y Decision Coverage Set _name to parameter Truncate name Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

24. Decision Coverage via Path testing • The objective of path testing is to ensure that the set of test cases is such that each path through the program is executed at least once • The starting point for path testing is a program flow graph that shows nodes representing program decisions and arcs representing the flow of control • Statements with conditions are therefore nodes in the flow graph

25. Program flow graphs • Describes the program control flow. Each branch is shown as a separate path and loops are shown by arrows looping back to the loop condition node • Used as a basis for computing the cyclomatic complexity • Cyclomatic complexity = Number of edges - Number of nodes +2

26. Cyclomatic complexity • The number of tests to test all control statements equals the cyclomatic complexity • Cyclomatic complexity equals number of conditions in a program • Useful if used with care. Does not imply adequacy of testing. • Although all paths are executed, all combinations of paths are not executed

27. Binary search (Java)

28. Binary search flow graph

29. Independent paths • 1, 2, 3, 8, 9 • 1, 2, 3, 4, 6, 7, 2 • 1, 2, 3, 4, 5, 7, 2 • 1, 2, 3, 4, 6, 7, 2, 8, 9 • Test cases should be derived so that all of these paths are executed • A dynamic program analyser may be used to check that paths have been executed

30. Grey Box Testing • Combination of Black and White box testing Gray box Correct output and expected behaviour … requirements & key design elements

31. Grey Box Testing ExampleBinary search - equiv. partitions • Pre-conditions satisfied, key element in array • Pre-conditions satisfied, key element not in array • Pre-conditions unsatisfied, key element in array • Pre-conditions unsatisfied, key element not in array • Input array has a single value • Input array has an even number of values • Input array has an odd number of values

32. Binary search equiv. partitions

33. Binary search - test cases

34. 3. Planning unit tests

35. Plan for Unit Testing One way to ... 1. Decide on the philosophy for unit testing • individual engineer responsible (common)? • reviewed by others? • designed & performed by others? 2. Decide what / where / how to document • individual’s personal document set (common)? • how / when to incorporate into other types of testing? • incorporate in formal documents? • use tools / test utilities? 3. Determine extent of unit testing (i.e., in advance). • do not just “test until time expires” • prioritize, so that important tests definitely performed 4. Decide how and where to get the test input 5. Estimate the resources required • use historical data if available 6. Arrange to track time, defect count, type & source Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

36. 4. Checklists and examples for Method testing

37. Perform Method Testing (Humphrey) 1/2 One way to ... 1. Verify operation at normal parameter values (a black box test based on the unit’s requirements) 2. Verify operation at limit parameter values (black box) 3. Verify operation outside parameter values (black box) 4. Ensure that all instructions execute (statement coverage) 5. Check all paths, including both sides of all branches (decision coverage) 6. Check the use of all called objects 7. Verify the handling of all data structures 8. Verify the handling of all files Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

38. Perform Method Testing (Humphrey) 2/2 One way to ... 9. Check normal termination of all loops (part of a correctness proof) 10. Check abnormal termination of all loops 11. Check normal termination of all recursions 12. Check abnormal termination of all recursions 13. Verify the handling of all error conditions 14. Check timing and synchronization 15. Verify all hardware dependencies Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

39. Method Test Case Examples • See pages 409-414 and Case Study at end of chapter • Each Example layouts Input, Execution and Expected Output • Test code is developed to exercise the methods and output the specifics of the tests (as above) and the actual results to a file for analysis

40. 5. Checklists and examples for class testing

41. Perform ClassUnit Tests One way to ... 1. Exercise methods in combination • 2-5, usually • choose most common sequences first • include sequences likely to cause defects • requires hand-computing the resulting attribute values 2. Focus unit tests on each attribute • initialize, then execute method sequences that affect it 3. Verify that each class invariant is unchanged • verify that the invariant is true with initial values • execute a sequence (e.g., the same as in 1.) • verify that the invariant still true 4. Verify that objects transition among expected states • plan the state / transition event sequence • set up the object in the initial state by setting variables • provide first event & check that transition occurred . etc. Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

42. Encounter State-Transition Test Sequence 1 of 2 test step 1 Preparing Verify that the game is initially in Preparing state (by checking on the class membership of gameStateI) Player dismisses qualities menu Waiting

43. Encounter State-Transition Test Sequence 1 of 2 test step 1 test step 2 Preparing Verify that the game is initially in Preparing state (by checking on the class membership of gameStateI) Dismiss the quality menu, and verify that the game is in Waiting state. Player dismisses qualities menu test step 3 Move the player character to an adjacent area, and verify that the game is still in Waiting state. Waiting Move to adjacent area Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

44. Complete Encounter State-Transition Test 1 2 Preparing Player dismisses qualities menu Reporting Player dismisses encounter report menu 6 3 5 Waiting Encounter completed Move to adjacent area 4 Character enters area inhabited by an opponent Engaging Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

45. Testing and debugging • Defect testing and debugging are distinct processes • Inspection and testing is concerned with establishing the existence of defects in a program • Debugging is concerned with locating and repairing these errors • Debugging involves formulating a hypothesis about program behaviour then testing these hypotheses to find the error

46. The debugging process

47. 6. Summary

48. Unit Testing: Summary • Unit testing is about “pieces” • Other testing is about “assemblies” • Black box: input / output only • White box: verifies processing • Several ways , Ensure completeness • Grey box: a bit of Black and White • Test planning - earlier the better • helps clarify requirements Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

49. Case StudyEncounterCharacter.java

50. Listing page 1 /** To test this class. * @param argsP destination of method test log, class test log respectively */ public static void main( String[] argsP ) { // Default files on which to write test output & run tests String methodOutputFileNameM = "methodOutput.txt"; String classOutputFileNameM= "classOutput.txt"; if( argsP != null && argsP.length == 2 ) // use defaults if input improper { methodOutputFileNameM = argsP[0]; classOutputFileNameM = argsP[1]; } // 1. EXECUTE TESTS WHICH DO NOT REQUIRE HUMAN INTERVENTION // Test methods individually, then test class try { testEncounterCharacterMethods( methodOutputFileNameM ); testEncounterCharacterClass( classOutputFileNameM ); } catch( IOException eP ) { System.out.println( eP ); } Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.