White Box Testing and Symbolic Execution

1. White Box Testing and Symbolic Execution Written by Michael Beder

2. Agenda • What is White Box Testing? • Flow Graph and Coverage Types • Symbolic Execution: • Formal Definition • Examples • Questions White Box Testing and Symbolic Execution

3. What is White Box Testing? • Software testing approach that uses inner structural and logical properties of the program for verification and deriving test data • Also called: Clear Box Testing, Glass Box Testing and Structural Testing • Manual: Inspection, Walkthrough • Automatic: Syntax Parser, Symbolic Execution White Box Testing and Symbolic Execution

4. Pros and Cons Pros: • Usage of more information on the tested object (than BlackBox) • Inference of real Equivalence Partitioning • Structural Coverage Assurance Cons: • Expensive • Limited Semantic Coverage White Box Testing and Symbolic Execution

5. Example I Sin(x) { if (|x| < eps) return x; if (|x| < 5*eps) return x – (x^3)/6; … } • Black Box Testing: Test 0, Pi/2, -Pi/2 • White Box Testing: Test 0.5*eps, eps, 3*eps, 5*eps, 7*eps, … Usage of logical properties makes better coverage White Box Testing and Symbolic Execution

6. Example II: Unit Testing f(x) g(x) { h(x) { { … … if (x > 0) return g(x); } } return h(x); } • Black Box Testing: Test f(x), g(x), h(x) for every x • White Box Testing: Test g(x) for x > 0, h(x) for x <= 0 and verify f(x) Usage of structural properties makes fewer, qualitative tests White Box Testing and Symbolic Execution

7. Flow Graph • Abstraction of the program • Defines the data and control flow in the program • Uniform representation of the program, language independent • Simple basic elements: assignment and condition • Further analysis is performed using graph algorithms White Box Testing and Symbolic Execution

8. start 1, 2 6 3 4 5 end Flow Graph – cont. G = (V, E) where - V is a set of basic blocks. start, end in V - E is a set of control branches Example: 1 a = Read(b) 2 c = 0 3 while (a > 1) { 4 If (a^2 > c) 5 c = c + a 6 a = a - 2 } F T Input: b = 2 F Output: a = 0, c = 2 T White Box Testing and Symbolic Execution

9. Basic Path Set • An execution path is a set of nodes and directed edges in a flow graph that connects (in a directed fashion) the start node to a terminal node. • Two execution paths are said to be independent if they do not include the same set of nodes and edges. • A basic set of execution paths for a flow graph is an independent maximum set of paths in which all nodes and edges of the graph are included at least once. White Box Testing and Symbolic Execution

10. White Box Coverage Types • Statement Coverage: Every statement is executed • Branch Coverage: Every branch option is chosen • Path Coverage: Every path is executed • Basic Path Coverage: Every basic path is executed Loops? White Box Testing and Symbolic Execution

11. start 1, 2 6 3 4 5 end Basic Path Coverage • An upper bound on the number of Basic paths needed is E – N + 2 (Linear Complexity) • Each path is called “basic path” • Example: p1 = start – 1,2 – 3 – end p2 = start – 1,2 – 3 – 4 – 6 – 3 – end p3 = start – 1,2 – 3 – 4 – 5 – 6 – 3 – end E – N + 2 = 8 – 7 + 2 = 3 F T F T White Box Testing and Symbolic Execution

12. Path Function • A function , when D is the working domain • Represents the current values of the variables as function of their initial values • Each variable X is represented by a projection function • Function composition: • For example: White Box Testing and Symbolic Execution

13. start 1, 2 6 F 3 T F 4 5 T end Path Condition For Example: p = start – 1,2 – 3 – end. 1 a = Read(b) 2 c = 0 3 while (a > 1) { 4 If (a^2 > c) 5 c = c + a 6 a = a - 2 } The path condition is B <= 1, when B is b’s value at start • A condition that should be fulfilled for going along the path • A constraint on the initial values of the variables White Box Testing and Symbolic Execution

14. Symbolic Execution • A method for deriving test cases which satisfy a given path • Performed as a simulation of the computation on the path • Initial path function = Identity function, Initial path condition = true • Each vertex on the path induce a symbolic composition on the path function and a logical constraint on the path condition: If an assignment was made: If a conditional decision was made: path condition path condition branch condition Output: path function and path condition for the given path White Box Testing and Symbolic Execution

15. Example: Symbolic Composition • x = x + y • y = y + x • end • The final path function represents the values of X, Y, Z after both assignments as a function of their initial value White Box Testing and Symbolic Execution

16. V1 Vn Vk Symbolic Execution is a special case when Concatenation and Associativity If is the path function of path and is the path function of path then is the path function of path The composition is associative: White Box Testing and Symbolic Execution

17. start 1, 2 6 F 3 T F 4 5 T end Example: Symbolic Execution 1 a = Read(b) 2 c = 0 3 while (a > 1) { 4 If (a^2 > c) 5 c = c + a 6 a = a - 2 } Find test case for path: p = start – 1,2 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – end input = b output = c White Box Testing and Symbolic Execution

18. start 1, 2 6 3 4 5 end Example: Symbolic Execution 1 a = Read(b) 2 c = 0 3 while (a > 1) { 4 If (a^2 > c) 5 c = c + a 6 a = a - 2 } p = start – 1,2 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – end vertex path function path condition start: (A, B, C) true 1,2 (A, B, C) true 3 (B, B, 0) true 4 (B, B, 0) (true Λ B>1) ↔ B>1 5 (B, B, 0) (B>1 Λ B^2>0) ↔ B>1 input = b output = c F T F T White Box Testing and Symbolic Execution

19. start 1, 2 6 3 4 5 end Example: Symbolic Execution 1 a = Read(b) 2 c = 0 3 while (a > 1) { 4 If (a^2 > c) 5 c = c + a 6 a = a – 2 } p = start – 1,2 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – end vertex path function path condition 6 (B, B, B) B>1 3 (B-2, B, B) B>1 4 (B-2, B, B) (B>1 Λ B-2>1) ↔ B>3 5 (B-2, B, B) (B>3 Λ (B-2)^2>B) ↔ B>4 6 (B-2, B, 2B-2) B>4 3 (B-4, B, 2B-2) B>4 end (B-4, B, 2B-2) (B>4 Λ B-4<=1) ↔ B=5 input = b output = c F T F T White Box Testing and Symbolic Execution

20. start 1, 2 6 F 3 T F 4 5 T end Example: Symbolic Execution 1 a = Read(b) 2 c = 0 3 while (a > 1) { 4 if (a^2 > c) 5 c = c + a 6 a = a – 2 } p = start – 1,2 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – end end (B-4, B, 2B-2) B=5 Hence the test case is B = 5 and the expected result is 2B-2 = 8. input = b output = c Is there a test case for p = start – 1,2 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – end ? White Box Testing and Symbolic Execution

21. Question (from exam) • 1 d = b + c; • 2 if (d > 20) • 3 a = 3 * a + d; • if (b < a) { • 5 a = 1; • 6 if (d < 2 * b) • 7 b = 2; • 8 } • Draw program’s Flow Graph • Find minimal number of test cases for the following coverage types: • Statement Coverage • Path Coverage • Branch Coverage • Basic Path Coverage White Box Testing and Symbolic Execution