Loading in 2 Seconds...

Department of CS and Mathematics, University of Pitesti

Loading in 2 Seconds...

91 Views

Download Presentation
## Department of CS and Mathematics, University of Pitesti

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**TAIC PART 2007**State-based Testing is Functional Testing ! Florentin Ipate, Raluca Lefticaru University of Pitesti, Romania Department of CS and Mathematics, University of Pitesti**TAIC PART 2007**• Background • Over the last 10-15 years we have developed specification based test generation techniques proven to determine system correctness under certain “design for test” conditions. • State-based specifications – stream X-machine (SXM). • Industrial evidence of the power and practicality of these techniques. • Joint research with the Verification & Testing Research Group of the University of Sheffield, UK (Mike Holcombe). Department of CS and Mathematics, University of Pitesti**TAIC PART 2007**• Class testing • State based languages (e.g. UML statecharts) are widely used to model class behaviour. • New challenge: extend our SXM based techniques to class testing. • Gather empirical evidence about their effectiveness. Department of CS and Mathematics, University of Pitesti**TAIC PART 2007**• Empirical evidence • Empirical studies show that state-based testing is more effective than random testingbut it is not likely to be sufficient unless the state diagram is straightforward. • They advocate the use of functional methods to complement state-based testing. • Our experiments (in which we evaluated our own SXM based techniques) also back these findings. Department of CS and Mathematics, University of Pitesti**TAIC PART 2007**• Questions • Why ? • Can functional testing and state-based testing be separated ? Department of CS and Mathematics, University of Pitesti**TAIC PART 2007**• Our answers • The modest fault detection of state-based class testing is actually due to the inappropriate state diagram used. • Functional testing of a class can be reduced to state-based testing, provided that the right state model is produced. Department of CS and Mathematics, University of Pitesti**push[0 < L < k-1]**pop[1 < L < k] popErr[L = 0] pushErr[L = k] push[L = 0] push[L = k-1] 0 1 2 pop[L = 1] pop[L = k] TAIC PART 2007 • How is a state diagram constructed ? • Usually: ad-hoc process, using conventional wisdom. • Example: a stack is associated with a 3 state diagram. Department of CS and Mathematics, University of Pitesti**push[0 < L < k-1]**pop[1 < L < k] count popErr[L = 0] count pushErr[L = k] count push[L = 0] push[L = k-1] 0 1 2 pop[L = 1] pop[L = k] TAIC PART 2007 • How is a state diagram constructed ? • Suppose we add a new method, count(l) that finds the • number of distinct elements among the last l. • A 3-state diagram is no longer appropriate. Department of CS and Mathematics, University of Pitesti**TAIC PART 2007**• Our approach • Construct a diagram for each class method to be tested. • The construction of the diagram – based on a partitioning of the domain of the class method. • Thus, state-based testing is functional testing ! Department of CS and Mathematics, University of Pitesti**TAIC PART 2007**• State diagram construction • Partition the domain according to some functional criterion. • Derive (not necessarily disjoint) data sub-domains by “hiding” the inputs. • Derive the states of the diagram by removing the overlapping of data sub-domains. • Construct the transitions by considering each pair of states and checking whether the pair is related to a method. • Split the transitions according to the original partitioning criterion. Department of CS and Mathematics, University of Pitesti**TAIC PART 2007**• State diagram construction (example) • push :3 state diagram • pop : 3 state diagram • count • domain partition • count(l) = 0 • count(l) = 1 • 1 <count(l) < L • count(l) = L Department of CS and Mathematics, University of Pitesti**count[p2]**count[p1] push[0→1] push[1→1] pop[1→1] 2 1 pop[1→0] push[1→2] pop[2→1] push[i→i,2≤i≤k-1] push[i→i+1,2≤i≤k-2] pop[i→i,2≤i≤k-1] pop[i→i-1,3≤i≤k-1] pop[k→k-1] 4 3 push[k-1→k] count[p2] count[p3] count[p4] count[p2] count[p3] TAIC PART 2007 • State diagram construction (example) Department of CS and Mathematics, University of Pitesti**TAIC PART 2007**• Previous approaches • Construct a composite (all-in-one) diagram rather than a separate diagram for each method. • A method may induce at most one transition between a given pair of states Original partitioning criterion is not fully used. • Stack example: • 9-state composite diagram but • One loop-back count transition for each state, so important test cases can be missed out. Department of CS and Mathematics, University of Pitesti**TAIC PART 2007**• Test generation • Adaptation of the W-method. • Transition cover: only the transitions labeled by the method in question are considered. • Characterization set: identifies each state of the diagram. Alternative : an abstract oracle associates with each state a predicate rather than a sequence of transitions. Department of CS and Mathematics, University of Pitesti**TAIC PART 2007**• Experimental results • A series of experiments to evaluate the cost effectiveness of state-based testing. • Java implementation of several classes, e.g. bounded stack, book from a library, vending machine. • Test suites were executed against mutant programs automatically generated by MuJava. • Test cost measure = number of mutants killed / total length Department of CS and Mathematics, University of Pitesti**TAIC PART 2007**• Experimental results • Bounded stack: approximately 90 LOC, 219 mutants. Our approach Composite (all-in-one) diagram Department of CS and Mathematics, University of Pitesti**TAIC PART 2007**• Experimental results • Book class (library): approximately 110 LOC, 136 mutants. Our approach Full predicate Department of CS and Mathematics, University of Pitesti**TAIC PART 2007**• Mutants left undetected • Equivalent mutants. • For which stronger criteria are needed. • Caused by attributes (dueDate) for which no obvious partitioning criterion exists: precise oracle needed. • Class mutants (JSI): simple tests in which more than one instance is used. Department of CS and Mathematics, University of Pitesti**TAIC PART 2007**• Further work • Evolutionary methods for automatically converting sequences of methods into actual test data. • Deriving the state model using evolutionary methods ? Department of CS and Mathematics, University of Pitesti**TAIC PART 2007**Thank you for your attention ! Department of CS and Mathematics, University of Pitesti