CIS 4932 Software Testing White-Box Testing

1. CIS 4932Software TestingWhite-Box Testing

2. Software under Test WHITE-BOX TESTING Stimuli Response(s) Testing to ensure that software does not do what is not supposed to do. Test ALL of it! No-Surprise Software!! Tick -- Tick -- Tick Unit Test Concepts

3. Software under Test WHITE-BOX TESTING • Focus is thorough execution of program elements during the testing process. • Warning: Tests only what is built, not what was intended! Stimuli Response(s) Unit Test Concepts

4. WHITE-BOX TESTING • Concept of coverage. Numeric measure of thoroughness of testing, relative to • Statements • Branches • Conditions • Paths Unit Test Concepts

5. CONTROL FLOW GRAPH • Defines the flow of control through unit. 1 2 4 3 5 Unit Test Concepts

6. 1 2 4 3 5 CONTROL FLOW GRAPH TERMINOLOGY • 5 NODES • sequential blocks of code terminated by a branch • 3 PATHS: • [1,2,3,5], [1,2,5], [1,4,5] • 2 BRANCHES • 1 and 2 are decision nodes Unit Test Concepts

7. 1 2 4 3 5 CONTROL FLOW GRAPH COVERAGE • One test case forces execution of one path (red) • Paths are determined by branches (decision nodes) • A thorough test set forces execution of all paths (red,green, blue). Unit Test Concepts

8. COVERAGE LEVELS (%) • Statement -- a statement has been executed at least once during testing • Branch -- each outcome of a branch has been performed at least once during testing • Path -- a path through the code has been executed at least once during during testing • Condition -- a condition has evaluated to true and to false at least once during testing Unit Test Concepts

9. CONTROL FLOW GRAPH COVERAGE MEASUREMENT • For 2 test cases (red, green) • Node (statement) cov = 4/5 • Branch cov = 1/2 [2] • Path cov = 2/3 • Acceptable coverage levels • Statement cov = 90% • Branch cov = 80% • Path cov = 70% 1 2 4 3 5 Unit Test Concepts

10. BRANCH vs CONDITION COVERAGE • Code example 1 if (x<1 && y>1) x = x + y; else y = y - x; 2 4 • 100% Branch coverage [1] • (x=0,y=2), (x=1,y=2) [TT,FT] • But not 100% Condition coverage • Need case TF (x=0,y=1) Unit Test Concepts

11. 1 2 3 1 2 5 3 4 THE PROBLEM WITH COMPOUND CONDITIONS • Makes complex logic appear simpler than it really is • Test cases may be omitted • Logic results in 3 paths, not 2!! if (x<1) {if (y>1) x=x+y; else y=y-x; } else y=y-x; Unit Test Concepts

12. 1 2 3 4 5 THE PROBLEM WITH PATH COVERAGE • Not all paths are feasible • No test case can force path [1,2,3,4,5]. Consecutive decisions mutually exclusive. if (x<1) y=2; if (x >= 1) y=3; z=y; Unit Test Concepts

13. Measuring Path Testing Difficulty • McCabe metric -- logical code complexity • Formula: 1 + #decisions in control flow graph • Test Significance: #basis paths through code • Design use: complexity of code • Test use: min #test cases for 100% path coverage • McCabe measures test (development) difficulty Unit Test Concepts

14. How To Design White Box Tests • Test cases must execute different paths • Decision tables • Rows -- elementary conditions in the code • Columns -- combinations of conditions in the code • Column based test case forces flow through different logic paths in the code • Decision table built from code reflects what was built versus intended (from spec) • Decision analysis for white-box testing. Unit Test Concepts

15. WHITE-BOX TESTING TOOLS • Tools instrument source code and gathers coverage data when tests • Compiled languages • Script languages -- coming to market • Some provide test case design assistance • List of paths not covered • Data conditions to force branch/path • Graphic depiction of graph coverage Unit Test Concepts

16. Age>80 | Y N N N N N Weight>300 | - Y N N N N Age<=12 | Y N N N Age>65 | Y N N Weight<120 | Y N Pills = | 0 0 1 2 2 C ------------------------------ Note: C: 2+(Weight/120)/50 Problem P1Code (v1) & Decision Table if (Age>80 || Weight>300) return 0; if (Age <= 12) return 1; if (Age > 65) return 2; if (Weight < 120) return 2 else return 2+(Weight-120)/50; McCabe = 6 Unit Test Concepts

17. Age>80 | Y N N N N N N Weight>300 | - Y N N N N N Age<=12 | Y N N N Age>65 | Y N N Weight<120 | - Y N Pills = | 0 0 0 1 2 2 C Problem P1Code (v2) & Decision Table if (Age>80 || Weight>300) return 0; if (Age <= 12) return 1; if (Age > 65 || (Age<=65 && Weight<120)) return 2; return 2+(Weight-120)/50; McCabe = 7 Unit Test Concepts

18. ____________ | ____________ | ____________ | ____________ | ____________ | Pills = | - - - - - - - --------------------------- Note: C: 2+(Weight/120)/50 Your Turn -- White-Box Testing(1) Construct Decision Table pills=0; if (Age < 80 && Weight <300) { pills=1; if (Age >= 65) pills=2; else if (Age > 12) pills=2+(Weight-120)/50; } return pills; Unit Test Concepts

19. Your Turn -- P1(2) Derive White-Box Test Cases Case 1 2 3 4 5 6 7 8 9 Age ___ ___ ___ ___ ___ ___ ___ ___ ___ Weight ___ ___ ___ ___ ___ ___ ___ ___ ___ Pills ___ ___ ___ ___ ___ ___ ___ ___ ___ Unit Test Concepts

20. OBSERVATIONS -- WHITE-BOX TEST CASES • Code may not be complete with respect to input combinations from the specification • Decision table constructed from code is simpler -- subset of black-box table • Claim: black-box test cases force coverage of logic • Unless the code implements the wrong (a different) function Unit Test Concepts

21. MAIN POINTSWHITE-BOX TESTING • White-box = logic testing • Limitation: can't tell what's missing • Don't forget exceptions -- throwing, catching, propagating (debugger) • Perform decision analysis of code • Coverage tools help. • Use black-box test cases. Unit Test Concepts