Analyzing and Testing Polymorphism in Java: A Fragment Class Analysis Approach

1. Fragment Class Analysis for Testing of Polymorphism in Java Software Atanas (Nasko) Rountev Ohio State University Ana Milanova Barbara Ryder Rutgers University ICSE 2003

2. Testing of Polymorphism • Testing of object-oriented software • Inheritance, polymorphism, object state, etc. • Polymorphism • A variable may refer to objects of different classes • Existing work on testing of polymorphism • Basic idea: exercise all possible polymorphic bindings • Our focus: bindings at polymorphic calls ICSE 2003

3. Two Coverage Criteria for Calls • RC criterion: exercise each call site withall possible classes of the receiver object • At x.m(), x may refer to A, B, or C objects • TM criterion: exercise each call site withall possibletarget methods • x.m() may invoke A.m or C.m • Goal: build a tool for Java that measures RC/TM coverage achieved by test suites • The approach is also applicable to other criteria for testing of polymorphism ICSE 2003

4. Coverage Tool for Testing of Polymorphism • Step 1: analyze the tested code • What are all possible receiver classes and target methods at polymorphic call sites? • Need class analysis • Step 2: insert instrumentation • Step 3: run tests and report coverage • What were thereceiver classes and target methods actually observed while running the tests? compare ICSE 2003

5. Outline • Class analysis • Class analysis of partial programs • Contribution 1: General approach that enables existing analyses to handle partial programs • Analysis imprecision • Experimental results • Contribution 2: Determining the imprecision of four popular class analyses • Conclusions and future work ICSE 2003

6. Class Analysis • Essential static analysis for object-oriented languages; large body of existing work • Given a reference variable x, what kinds of objects may x refer to? • e.g. “x may refer to instances of A, B, or C” • Uses of class analysis • Compiler optimizations: e.g. devirtualization • Software understanding and restructuring • Testing: e.g. to compute the RC/TM criteria ICSE 2003

7. Problem 1: Partial Programs • Many existing class analyses • Typically defined as whole-program analyses • Cannot be used directly when testing is done on partial programs • Analysis input • Component under test (CUT): set of classes • CUT interface that is being tested • Methods and fields against which tests have been written • Server classes of CUT classes ICSE 2003

8. Fragment Class Analysis • Goal: adapt whole-program class analyses • Solution: fragment class analysis • Step 1: Create additional code that “simulates” the possible effects of unknown external code • Located inside an artificial main • Step 2: Run a whole-program class analysis starting from main • Use the resulting solution to determine RC and TM at polymorphic call sites in the CUT ICSE 2003

9. Analysis Solution • If it is possible to write a test that executes x.m() with a receiver of class Y, the analysis solution must report Y • Need to “simulate” all possible (infinitely many) tests for the component • This property provably holds for a large category of whole-program class analyses • Flow insensitive • Context insensitive and context sensitive ICSE 2003

10. Placeholder Variables • main contains placeholders for all reference variables from unknown external code CUT Placeholder variables main() { A ph_a; B ph_b; Y ph_y; Z ph_z; … } class A { public A() {...} public Y m(Z z) {...} } class B extends A { public B m2(A a) {…} } ICSE 2003

11. Placeholder Statements Placeholder statements main() { … ph_a = new A(); ph_y = ph_a.m(ph_z); ph_b = ph_b.m2(ph_a); … ph_a = ph_b; ph_b = (B) ph_a; … } class A { public A() {...} public Y m(Z z) {...} } class B extends A { public B m2(A a) {…} } ICSE 2003

12. Placeholder Statements Placeholder statements main() { … ph_a = new A(); ph_y = ph_a.m(ph_z); ph_b = ph_b.m2(ph_a); … ph_a = ph_b; ph_b = (B) ph_a; … } class A { public A() {...} public Y m(Z z) {...} } class B extends A { public B m2(A a) {…} } ICSE 2003

13. Placeholder Statements Placeholder statements main() { … ph_a = new A(); ph_y = ph_a.m(ph_z); ph_b = ph_b.m2(ph_a); … ph_a = ph_b; ph_b = (B) ph_a; … } class A { public A() {...} public Y m(Z z) {...} } class B extends A { public B m2(A a) {…} } ICSE 2003

14. Problem 2: Analysis Imprecision • Among all classes reported by the analysis, which ones are impossible at run time? • Coverage requirements that cannot be satisfied • Goal: determine the imprecision of four fragment class analyses, derived from: • Class Hierarchy Analysis (CHA) • Rapid Type Analysis (RTA) • 0-CFA • Andersen’s points-to analysis (AND) • No such data is available in previous work ICSE 2003

15. Experiments • Considered 8 CUTs • From publicly available Java libraries • CUT size: between 8 and 24 classes • Wrote tests that achieve as much RC/TM coverage as possible • Two people wrote tests independently • Compared differences and merged • Metric of imprecision: coverage reported by the tool for these tests • Report of 100% means no analysis imprecision ICSE 2003

16. RC Coverage ICSE 2003

17. Results • Bad news: CHA and RTA have significant imprecision and should be avoided • Good news: 0-CFA and AND have much lower imprecision • Close to 100% for 5 of the 8 components • Good candidates for future investigations • Open questions • Will we see the same trend for other CUTs? • What is the improvement for analyses that are theoretically more precise than 0-CFA and AND? ICSE 2003

18. Future Work • More datapoints • More class analyses (e.g. context-sensitive) • When do analyses fail? • Common sources of imprecision • Generalize to flow-sensitive analyses • Apply to existing test suites • Measure achieved coverage • Identify weaknesses and add more test cases ICSE 2003