1 / 19

JMLAutoTest and Its Double-phase Testing Way

JMLAutoTest and Its Double-phase Testing Way. Guoqing Xu Com Sci., East China Normal Univ. Shanghai 200062, PRC. FATES 2003, Montreal, Canada, Oct.6th 2003. Introduction to the current JML testing framework. The main idea in the JML and JUnit framework: specification as test oracle.

abigail-puckett
Download Presentation

JMLAutoTest and Its Double-phase Testing Way

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. JMLAutoTestand Its Double-phase Testing Way Guoqing Xu Com Sci., East China Normal Univ. Shanghai 200062, PRC FATES 2003, Montreal, Canada,Oct.6th 2003

  2. Introduction to the current JML testing framework • The main idea in the JML and JUnit framework: specification as test oracle. • It can generate codes from formal specifications for unit test by translating pre- and post-conditions of a method into test oracles. • It has solved the problem that it is hard to write unit test code by hand.

  3. Problems • Writing codes for generating test cases, especially those with a linked data structure is still a labor-intensive job. • Meaningless test cases can not be avoided in the test, thus much time might be used to explore meaningless test inputs.

  4. Problems (more) • Some formal approach based testing frameworks require programmer to write special predicates to check the validity of test cases (i.e. Korat). However they can not be used in a black-box test since testers may know little about the details in the program to be tested.

  5. Goals of our new framework • Generate test cases automatically including those with a linked data structure. • Prevent meaningless test cases from being processed in the final test instead of only identifying them.

  6. The test case generation • Set Value Domain for Fields of the Input class. • Make a finitization of all cases by combining each value in the domain specified by testers. • Verify the validity of a test case by checking the run-time state of instance invariant specified in its class.

  7. Double-phase testing • Goals: 1. Prevent meaningless test cases being processed in the final test, therefore saving the time. 2. Do not require testers to know the details of the program to be tested, thus making the black-box test possible.

  8. Key steps in double-phase testing

  9. Operational Profile • Operational Profile is a standard made by the tester to divide the generated test case space into several partitions. • the validity of double-phase testing is based on the quality of the operational profile. • it is a good idea to start out with several models and evaluate their predictive accuracy before settling on one.

  10. The first phase • Take a relatively small number (e.g. 10%) of test cases out from each partition according to the average distribution. • Run these groups of cases respectively. • Make statistics on the number of meaningless test cases appeared in each group.

  11. The second phase • Calculate and determine the probability of meaningless test cases existing in each partition. • Reorganize the test cases according to the inverse proportion of meaningless cases in each partition. (e.g. take 80% of cases from the partition which produces 20% of meaningless ones in the first phase.) • Run the final test.

  12. Experimental results Target Method to be tested: /*+@ public normal_behavior @ requires parentTree!=null && thisNode!=null && @ (\exists Node n; parentTree.toObjectSet().has(n); @ n.ID== thisNode.ID); @ assignable \nothing; @ ensures \result.root.ID == thisNode.ID ; @+*/ BinaryTree findSubTree(BinaryTree parentTree, Node thisNode)

  13. Test case generation • The pre-condition of the method says that neither of its arguments can be null and there must be a node in parentTree whose ID equals the ID of thisNode. • We generate the test case space of type BinaryTree with a few nodes. We also generate the case space of type Node which contains 12 nodes whose IDs are from 0 to 11.

  14. Divide the test case space • For the test case space of type BinaryTree, we do not divide it and leave it as the only partition. • For the space of type Node, We divide it into two partitions. The first one contains nodes whose ID varies from 0 to 5 and the second one contains the rest.

  15. Test Results • Table 1. Performance of JMLAutoTest for testing the method findSubTree with arguments “-pre 0.25”.

  16. Test Results • Table 2. Performance comparison between the double-phase testing in JMLAutoTest and the conventional way in JMLUnit testing framework.

  17. Conclusions • JMLAutoTest provides interfaces for testers to automatically generate test cases, even those with a linked data structure. a) Set value domain for fields b) Finitization c) Verify validity of a test case by checking the instance invariant specified in its class

  18. Conclusions (more) • Based on statistical principle, JMLAutoTest uses the double-phase testing way to filter out meaningless test cases before the final test is run. a) Make an Operational Profile b) Determine the proportion of meaningless test cases of each partition in the first phase. c) Reorganize the test cases according the distribution if meaningless cases and run the final test

  19. Acknowledgement We would like to thank Professor Gary T. Leavens in Iowa State University who gave us help on both JML and the testing framework.

More Related