Black-Box Testing Techniques III

# Black-Box Testing Techniques III

## Black-Box Testing Techniques III

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
##### Presentation Transcript

1. Black-Box Testing Techniques III Software Testing and Verification Lecture 6 Prepared by Stephen M. Thebaut, Ph.D. University of Florida

2. Black-Box Test Case Design Techniques Considered • Partition testing • Combinatorial Approaches • Boundary Value Analysis • Intuition & Experience

3. Another Cause-Effect Example: Symbol Table Storage Specification The conditions for storing an identifier in one of two symbol tables are: (a) must be from 2 to 8 characters in length; (b) first character must be a letter or “\$”; (c) other characters must be a letter or digit. If the first character is a letter, the identifier will be stored in symbol table A. If the first character is “\$”, it will be stored in symbol table B. If the first character is neither a letter nor “\$”, or if condition (c) is not satisfied, error message J11 is output. If condition (a) is not satisfied, error message J12 is output.

4. What are the “Effects”? The conditions for storing an identifier in one of two symbol tables are: (a) must be from 2 to 8 characters in length; (b) first character must be a letter or “\$”; (c) other characters must be a letter or digit. If the first character is a letter, the identifier will be stored in symbol table A. If the first character is “\$”, it will be stored in symbol table B. If the first character is neither a letter nor “\$”, or if condition (c) is not satisfied, error message J11 is output. If condition (a) is not satisfied, error message J12 is output.

5. What are the “Effects”? The conditions for storing an identifier in one of two symbol tables are: (a) must be from 2 to 8 characters in length; (b) first character must be a letter or “\$”; (c) other characters must be a letter or digit. If the first character is a letter, the identifier will be stored in symbol table A. If the first character is “\$”, it will be stored in symbol table B. If the first character is neither a letter nor “\$”, or if condition (c) is not satisfied, error message J11 is output. If condition (a) is not satisfied, error message J12 is output.

6. What are the “Causes”? The conditions for storing an identifier in one of two symbol tables are: (a) must be from 2 to 8 characters in length; (b) first character must be a letter or “\$”; (c) other characters must be a letter or digit. If the first character is a letter, the identifier will be stored in symbol table A. If the first character is “\$”, it will be stored in symbol table B. If the first character is neither a letter nor “\$”, or if condition (c) is not satisfied, error message J11 is output. If condition (a) is not satisfied, error message J12 is output.

7. What are the “Causes”? The conditions for storing an identifier in one of two symbol tables are: (a) must be from 2 to 8 characters in length; (b) first character must be a letter or “\$”; (c) other characters must be a letter or digit. If the first character is a letter, the identifier will be stored in symbol table A. If the first character is “\$”, it will be stored in symbol table B. If the first character is neither a letter nor “\$”, or if condition (c) is not satisfied, error message J11 is output. If condition (a) is not satisfied, error message J12 is output.

8. Causes and Effects Causes:Effects: (1) 2  no. chars  8 (31) store in table A (2) 1stchar is letter (32) store in table B (3) 1stchar is \$ (33) output msg J11 (4) other charsonly letters/digits (34) output msg J12 only (35) output msgs J11 and J12

9. Boolean Graphs [2,8] let \$ (1) (2) (3) (4) (31) (32) −> A −> B E others let/dig

10. Boolean Graphs [2,8] let \$ (1) (2) (3) (4) (31) (32) −> A −> B E others let/dig

11. Another Cause-Effect Example: Symbol Table Storage Specification The conditions for storing an identifier in one of two symbol tables are: (a) must be from 2 to 8 characters in length; (b) first character must be a letter or “\$”; (c) other characters must be a letter or digit. If the first character is a letter, the identifier will be stored in symbol table A. If the first character is “\$”, it will be stored in symbol table B. If the first character is neither a letter nor “\$”, or if condition (c) is not satisfied, error message J11 is output. If condition (a) is not satisfied, error message J12 is output.

12. Boolean Graphs (cont’d) [2,8] let \$ (1) (2) (3) (4) Л (31) (32) −> A −> B E others let/dig

13. Boolean Graphs [2,8] let \$ (1) (2) (3) (4) (31) (32) −> A −> B E others let/dig

14. Another Cause-Effect Example: Symbol Table Storage Specification The conditions for storing an identifier in one of two symbol tables are: (a) must be from 2 to 8 characters in length; (b) first character must be a letter or “\$”; (c) other characters must be a letter or digit. If the first character is a letter, the identifier will be stored in symbol table A. If the first character is “\$”, it will be stored in symbol table B. If the first character is neither a letter nor “\$”, or if condition (c) is not satisfied, error message J11 is output. If condition (a) is not satisfied, error message J12 is output.

15. Boolean Graphs (cont’d) [2,8] let \$ (1) (2) (3) (4) (31) (32) −> A −> B E Л others let/dig

16. Boolean Graphs (cont’d) [2,8] let \$ (1) (2) (3) (4) Л (31) (32) −> A −> B E Л others let/dig

17. Boolean Graphs (cont’d) [2,8] let \$ (1) (2) (3) (4) (33) (34) (35) J11 only J12 only J11 & J12 E others let/dig

18. Boolean Graphs (cont’d) [2,8] let \$ (1) (2) (3) (4) (33) (34) (35) J11 only J12 only J11 & J12 E others let/dig

19. Another Cause-Effect Example: Symbol Table Storage Specification The conditions for storing an identifier in one of two symbol tables are: (a) must be from 2 to 8 characters in length; (b) first character must be a letter or “\$”; (c) other characters must be a letter or digit. If the first character is a letter, the identifier will be stored in symbol table A. If the first character is “\$”, it will be stored in symbol table B. If the first character is neither a letter nor “\$”, or if condition (c) is not satisfied, error message J11 is output. If condition (a) is not satisfied, error message J12 is output.

20. Boolean Graphs (cont’d) [2,8] let \$ (1) (2) (3) (4) (33) (34) (35) J11 only J12 only J11 & J12 Л E (A) others let/dig

21. Another Cause-Effect Example: Symbol Table Storage Specification The conditions for storing an identifier in one of two symbol tables are: (a) must be from 2 to 8 characters in length; (b) first character must be a letter or “\$”; (c) other characters must be a letter or digit. If the first character is a letter, the identifier will be stored in symbol table A. If the first character is “\$”, it will be stored in symbol table B. If the first character is neither a letter nor “\$”, or if condition (c) is not satisfied, error message J11 is output. If condition (a) is not satisfied, error message J12 is output.

22. Boolean Graphs (cont’d) [2,8] let \$ (1) (2) (3) (4) (33) (34) (35) J11 only J12 only J11 & J12 Л E (A) V (B) others let/dig

23. Another Cause-Effect Example: Symbol Table Storage Specification The conditions for storing an identifier in one of two symbol tables are: (a) must be from 2 to 8 characters in length; (b) first character must be a letter or “\$”; (c) other characters must be a letter or digit. If the first character is a letter, the identifier will be stored in symbol table A. If the first character is “\$”, it will be stored in symbol table B. If the first character is neither a letter nor “\$”, or if condition (c) is not satisfied, error message J11 is output. If condition (a) is not satisfied, error message J12 is output.

24. Boolean Graphs (cont’d) [2,8] let \$ (1) (2) (3) (4) (33) (34) (35) J11 only J12 only J11 & J12 Л E (A) V (B) others let/dig

25. Boolean Graphs (cont’d) [2,8] let \$ (1) (2) (3) (4) (33) (34) (35) J11 only J12 only J11 & J12 Л E (A) V (B) others let/dig

26. Another Cause-Effect Example: Symbol Table Storage Specification The conditions for storing an identifier in one of two symbol tables are: (a) must be from 2 to 8 characters in length; (b) first character must be a letter or “\$”; (c) other characters must be a letter or digit. If the first character is a letter, the identifier will be stored in symbol table A. If the first character is “\$”, it will be stored in symbol table B. If the first character is neither a letter nor “\$”, or if condition (c) is not satisfied, error message J11 is output. If condition (a) is not satisfied, error message J12 is output.

27. Boolean Graphs (cont’d) [2,8] let \$ (1) (2) (3) (4) Л (33) (34) (35) J11 only J12 only J11 & J12 Л E (A) V (B) others let/dig

28. Boolean Graphs (cont’d) [2,8] let \$ (1) (2) (3) (4) Л (33) (34) (35) J11 only J12 only J11 & J12 Л E (A) V (B) others let/dig

29. Another Cause-Effect Example: Symbol Table Storage Specification The conditions for storing an identifier in one of two symbol tables are: (a) must be from 2 to 8 characters in length; (b) first character must be a letter or “\$”; (c) other characters must be a letter or digit. If the first character is a letter, the identifier will be stored in symbol table A. If the first character is “\$”, it will be stored in symbol table B. If the first character is neither a letter nor “\$”, or if condition (c) is not satisfied, error message J11 is output. If condition (a) is not satisfied, error message J12 is output.

30. Boolean Graphs (cont’d) [2,8] let \$ (1) (2) (3) (4) Л (33) (34) (35) J11 only J12 only J11 & J12 Л E (A) V (B) others let/dig

31. Boolean Graphs (cont’d) [2,8] let \$ (1) (2) (3) (4) Л (33) (34) (35) J11 only J12 only J11 & J12 Л E (A) V (B) others let/dig

32. Boolean Graphs (cont’d) [2,8] let \$ (1) (2) (3) (4) Л (33) (34) (35) J11 only J12 only J11 & J12 Л Л E (A) V (B) others let/dig

33. Boolean Graphs (cont’d) [2,8] let \$ (1) (2) (3) (4) Л (33) (34) (35) J11 only J12 only J11 & J12 Л Л E (A) V (B) others let/dig

34. Boolean Graphs (cont’d) [2,8] let \$ (1) (2) (3) (4) Л (33) (34) (35) J11 only J12 only J11 & J12 Л Л E (A) V (B) Л others let/dig

35. A Variation on Test Case Selection Strategy #3 • Test case selection “Strategy #3” considers ALLfeasible combinations of connected Cause values that result in each Effect being True. • For complex specifications, this can be impractical. • We now consider a variation on this strategy which “culls” all but the combinations “of greatest interest”.

36. A Variation on Test Case Selection Strategy #3 • Test case selection “Strategy #3” considers ALLfeasible combinations of connected Cause values that result in each Effect being True. • For complex specifications, this can be impractical. • We now consider a variation on this strategy which “culls” all but the combinations “of greatest interest”.

37. A Variation on Test Case Selection Strategy #3 • Test case selection “Strategy #3” considers ALLfeasible combinations of connected Cause values that result in each Effect being True. • For complex specifications, this can be impractical. • We now consider a variation on this strategy which “culls” all but the combinations “of greatest interest”.

38. Test Case Selection Strategy #3 Plus “Culling Rules” REPEAT Select the next (initially, the first) Effect. Tracing back through the graph (right to left), find all feasible combinations of connected Cause values that result in the Effect being True, subject to the following culling rules: • When encountering an nth-degree OR-node that must be True, consideronly those n combinations for which exactly one incoming edge is True.

39. Test Case Selection Strategy #3 Plus “Culling Rules” REPEAT Select the next (initially, the first) Effect. Tracing back through the graph (right to left), find all feasible combinations of connected Cause values that result in the Effect being True, subject to the following culling rules: • When encountering an nth-degree OR-node that must be True, consideronly those n combinations for which exactly one incoming edge is True.

40. Test Case Selection Strategy #3 Plus “Culling Rules” (cont’d) • When encountering an nth-degree AND-node that must be False, consider only those n combinations for which exactly one incoming edge is False. For each new such combination found: Determine values of all other Effects, and Enter values for each Cause and Effect in a new column of the test case coverage matrix. UNTIL each Effect has been selected.

41. Test Case Selection Strategy #3 Plus “Culling Rules” (cont’d) • When encountering an nth-degree AND-node that must be False, consider only those n combinations for which exactly one incoming edge is False. For each new such combination found: Determine values of all other Effects, and Enter values for each Cause and Effect in a new column of the test case coverage matrix. UNTIL each Effect has been selected.

42. Rationale for these Culling Rules? • Number of combinations decreases by a factor of O(2) to O(n) at each true OR node and each false AND node. • Idea: cover only the minimally sufficient conditions for the desired result. n ( ( V Л T F

43. Applying Strategy #3 Plus Culling Rules [2,8] let \$ (1) (2) (3) (4) Л (31) (32) −> A −> B E Л others let/dig

44. Applying Strategy #3 Plus Culling Rules [2,8] let \$ (1) (2) (3) (4) Л (31) (32) −> A −> B E Л others let/dig

45. Coverage Matrix

46. Applying Strategy #3 Plus Culling Rules (cont’d) [2,8] let \$ (1) (2) (3) (4) Л (31) (32) −> A −> B E Л others let/dig

47. Coverage Matrix (cont’d)

48. Applying Strategy #3 Plus Culling Rules (cont’d) [2,8] let \$ (1) (2) (3) (4) Л (33) (34) (35) J11 only J12 only J11 & J12 Л Л E (A) V (B) Л others let/dig

49. Applying Strategy #3 Plus Culling Rules (cont’d) (33)  1, B

50. Applying Strategy #3 Plus Culling Rules (cont’d) [2,8] let \$ (1) (2) (3) (4) Л (33) (34) (35) J11 only J12 only J11 & J12 Л Л E (A) V (B) Л others let/dig