Prioritizing Test Cases for Regression Testing Based on Boundary Value Coverage Tao Xie and David Notkin Computer Science & Engineering, University of Washington. What test cases are most likely to expose faults ?. Introduction. Standard characterizations of test cases include:.
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
Prioritizing Test Cases for Regression Testing Based on Boundary Value CoverageTao Xie and David NotkinComputer Science & Engineering, University of Washington
What test cases are most likely to expose faults ?
Standard characterizations of test cases include:
Problem: If resources are constrained, how can we still perform regression testing effectively?
Structural coverage: exercising structural entities (e.g., statements, edges) that encode faults is essential to exposing those faults try to choose test cases that achieve high coverage of those entities.
Data Coverage: exercising entities with special values (e.g., boundary values) increases the ability to expose many faults try to choose special values that achieve high coverage of those entities.
Test Case Selection selects a subset of test cases that exercise those changed entities
Test Case Prioritization orders the test cases with the intent of executing the test cases most likely to expose faults earlier
What special values are dangerous?
Boundary values tend to be dangerous.
Extensively discussed in practitioner and researcher's testing handbooks:
“The Art of Software Testing” by G. J. Meyers, 1979
“Software Testing Techniques” by B. Beizer, 1990 …… and research literatures ….
Related Work:Boundary Value Coverage
Related Work: Test Case Prioritization
Most existing applications of boundary value coverage are in test case generation.
Most existing test case prioritization techniques are based on structural coverage.
is based on
Usually the specification or an assertion-like annotation is required to infer boundary value. Few programs are equipped with them.
Lack of tool support to collect boundary value coverage information without a priori specification.
Previous empirical studies showed data coverage techniques, e.g. boundary value coverage, outperformed structural coverage ones in testing effectiveness
Complementing structural coverage techniques with data coverage ones are suggested in practice.
Our Solution: Test Case Prioritization Based on Boundary Value Coverage without a priori specification
Boundary predicates (used to infer boundary values) are extracted from invariants dynamically detected by Daikon, an invariant detection tool. The boundary value coverage for a test case execution is collected based on the data trace of that execution and the boundary predicates. Then test case prioritization can incorporate this boundary value coverage information to improve performance.
Our initial experiments show promise for boundary value coverage; in particular, boundary value coverage dominated structural coverage for these small programs.
This work was supported in part by the National Science Foundation under grant ITR 0086003.