Dependence Analysis in Reduction of Requirement Based Test Suites

1. Dependence Analysis in Reduction of Requirement Based Test Suites Boris Vaysburg Luay Tahat Bogdan Korel Computer Science Department Bell Labs Innovations Computer Science Department Illinois Institute of Technology Lucent Technologies Illinois Institute of Technology Chicago, IL USA Naperville, IL USA Chicago, IL USA

2. Outline • Model-based testing • Selective model-based testing • Selective test suite reduction using EFSM analysis • EFSM dependencies • Test Reduction based on EFSM analysis • Static approach • Dynamic approach • Conclusions

3. Model Based Testing • Model based testing is used to test software systems based on a system model • Test cases are generated from the system model • Testing state-based software systems

4. Model Based Testing • Systems can be modeled using: • EFSM: Extended Finite State Machine • SDL: Specification Description Language

5. Model Based Testing • Systems can be modeled using: • EFSM: Extended Finite State Machine • SDL: Specification Description Language

6. Extended Finite State MachineEFSM • EFSM consists of: • States • Transitions • The following elements are associated with each transition: • Event • Enabling condition • Action(s)

7. State 1 State 2 EFSM Transition Event(p)[Condition]/Action

8. EFSM Model of the ATM System

9. Model Based Testing Strategies • State coverage • Transition coverage • Path coverage • Constrained path coverage: each transition can be traversed at most “n” times in a test

10. Model-Based Testing • These strategies are used to test the whole system • Model-based testing strategies may generate large test suites

11. Selective Model-Based Testing • Testers are frequently interested in testing only selected functionality (requirement) • In a system model requirements frequently correspond to transitions • The problem of selective testing of a requirement can be converted to a problem of testing a transition in the model (selective testing of a transition)

12. Selective Model-Based Testing

13. Selective Model-Based Testing

14. Selective Testing Strategies with respect to a transition • State coverage • Transition coverage • Path coverage • Constrained path coverage Each test must contain a transition (requirement) under test

15. Selective Model-Based Testing • The size of a selective model-based test suite is frequently significantly smaller than the size of a complete system test suite • However, the size of selective test suites still may be large

16. Reduction of Selective Test Suites • The goal is to develop an approach of reduction of selective test suites • EFSM dependence analysis is used to reduce the size of selective test suites

17. Test Suite Reduction Two tests are considered to be “equivalent” with respect to a transition under test if, during traversal of the EFSM model, these tests exhibit the same “pattern” of interactions with respect to the transition under test.

18. Test Suite Reduction Interactions? Dependencies between EFSM elements.

19. EFSM dependence analysis • Dependencies between transitions (“active” elements of EFSM) • Two types of dependencies between transitions in the EFSM are identified: • Data dependence • Control dependence

20. Data Dependence Data dependence captures the notion that one transition defines a value to a variable and another transition may potentially use this value.

21. Data Dependence

22. Control Dependence • Control dependence captures the notion that one transition may affect traversal of another transition • It is defined based on the concept of post-dominance

23. Control Dependence

24. Control Dependence

25. EFSM Dependence Graph • Nodes represent transitions • Arcs represent control and data dependencies

26. T5 T6 T8 T1 T4 T9 T2 T3 T7 Static EFSM Dependence Graph data dependence control dependence

27. Test Suite Reduction Two tests are considered to be “equivalent” with respect to a transition under test if, during traversal of the EFSM model, these tests exhibit the same “pattern” of interactions with respect to the transition under test.

28. Test suite reduction Interactions: • Data dependencies • Control dependencies Pattern of interactions: • Dependence sub-graph • Static interaction pattern • Dynamic interaction pattern

29. Test #1: T1, T4, T6, T8, T6, T8, T5, T8, T9 Test #2: T1, T2, T4, T6, T8, T6, T8, T5, T8, T9

30. Selective test suite reductionStatic Interaction Pattern • All dependencies that occurred during traversal of the test (path) are mapped to the EFSM dependence sub-graph • Dependencies that do not “influence” a transition under test are removed • The resulting dependence sub-graph is called Static Interaction Pattern • If two tests have the same Static Interaction Pattern with respect to a transition under test, one of these tests is removed from the test suite

31. Test #1: T1 T4 T6 T8 T6 T8 T5 T8 T9

32. T5 T6 T8 T1 T4 T9 Dependence Sub-graph

33. T6 T8 T1 T4 T9 Dependence Sub-graph T5

34. T6 T8 T1 T4 T9 Dependence Sub-graph T5

35. T6 T1 T4 Static Interaction Patternfor Test #1 T5

36. Test #2: T1 T2 T4 T6 T8 T6 T8 T5 T8 T9

37. T5 T6 T8 T1 T4 T9 T2 Dependence Sub-graph

38. T6 T8 T1 T4 T9 T2 Dependence Sub-graph T5

39. T6 T8 T1 T4 T9 T2 Dependence Sub-graph T5

40. T6 T1 T4 Static Interaction Patternfor Test #2 T5 Test #1 and Test #2 have the same Static Interaction Patterns

41. Test suite reduction • Significant test reduction can be achieved using Static Interaction Patterns

42. n Size of Selective Test Suite Size of Reduced Test Suite Using Static Interaction Pattern 2 18 3 3 100 7 4 450 14 5 1,806 16 6 6,762 16 7 24,300 16 Test Suite Size Reduction

43. Selective test suite reduction • Test reduction using Static Interaction Patterns is relatively “coarse” • Static Interaction Patterns are not able to capture intricate interactions • Dynamic Interaction Pattern is used

44. Dynamic EFSM dependencies • Dynamic dependencies are dependencies that occur during traversal of the test (path) • Dynamic EFSM dependence graph

45. Dynamic EFSM Dependencies for Test #2 T1 T2 T4 T6 T8 T6 T8 T5 T8 T9 Dynamic EFSM Dependence Graph

46. Selective test suite reductionDynamic Interaction Pattern • All dependencies that occurred during traversal of the test (path) are used to create a dynamic EFSM dependence graph. • Dependencies that do not “influence” a transition under test are removed. • The resulting dependence sub-graph is called Dynamic Interaction Pattern. • If two tests have the same Dynamic Interaction Pattern with respect to a transition under test, one of these tests is removed from the test suite.

47. Dynamic EFSM Dependence graph for Test #2 T1 T2 T4 T6 T8 T6 T8 T5 T8 T9

48. Dynamic EFSM Dependence graph for Test #2 T1 T2 T4 T6 T8 T6 T8 T5 T8 T9

49. Dynamic EFSM Dependence graph for Test #2 T1 T2 T4 T6 T8 T6 T8 T5 T8 T9

50. Dynamic Interaction Pattern for Test #2 T1 T2 T4 T6 T8 T6 T8T5T8 T9 If two tests have the same Dynamic Interaction Pattern with respect to a transition under test, one of these tests is removed from the test suite