1 / 36

Rigorous Software Development CSCI-GA 3033-011

Rigorous Software Development CSCI-GA 3033-011. Instructor: Thomas Wies Spring 2012 Lecture 7. Programming Project: Minesweeper. Project Goal. Implement a m inesweeper game together with a perfect minesweeper solver

eris
Download Presentation

Rigorous Software Development CSCI-GA 3033-011

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Rigorous Software DevelopmentCSCI-GA 3033-011 Instructor: Thomas Wies Spring 2012 Lecture 7

  2. Programming Project: Minesweeper

  3. Project Goal • Implement a minesweeper game together with a perfect minesweeper solver • perfect solver: never guess the next move unless the current board configuration does not allow to deduce whether some unrevealed cell contains a mine or not. • in particular: a perfect solver can decide whether a given board configuration can be solved without guessing.

  4. Project Outline • Phase 1: Alloy Model • Phase 2: Java Implementation

  5. Project Phase 1: Alloy model • Objectives: • develop abstract minesweeper model • simulate board configurations and plays of the game • specify key invariants of board configurations • identify pre and post-conditions of operations on boards • implement simple minesweeper solver • Deadline: Tuesday, April 10, 2012

  6. Project Phase 2: Java Implementation • Objectives: • Transfer Alloy model to JML specifications • Implement • simple parser for board configurations • game • solver • Check JML specifications using JML tools • Deadline: Tuesday, May 1, 2012 • Competition: • submitted solvers will compete against each other

  7. Alloy Model • Abstract Minesweeper: game is played on an undirected graph rather than a grid-like board • nodes represent cells of the board • edges indicate which cells are neighbors 2

  8. Invariants of Board Configurations • Marked cells are not revealed. • The number of marked cells is not larger than the number of mines. • If x is a revealed cell that has no neighboring mines, then all its neighbors are also revealed. • ...

  9. Solving Minesweeper Games • Solving minesweeper games is closely related to the Minesweeper Consistency Problem: • given a board configuration • does there exist an assignment of mines to unrevealed cells that is consistent with the conf.?

  10. Minesweeper Consistency: Example Is this configuration consistent? No!

  11. Minesweeper Consistency: Example and this one? Yes!

  12. Minesweeper Consistency • The Minesweeper Consistency Problem is NP-complete. • There is a simple brute-force algorithm to solve minesweeper games. • If you want a solver that works fast in practice, you will have to do something more clever.

  13. Programming Project • Phase 1: Alloy Model • Deadline: Tuesday, April 10, 2012 • Phase 2: Java Implementation • Deadline: Tuesday, May 1, 2012 More detailed information will be available on the course webpage.

  14. Today’s Topic:Automated Test Case Generation

  15. How to Test Effectively? public class Factorial { /*@ requires n >= 0; @ ensures \result > 0; @*/ public static int factorial (int n) { int result = n; while (--n > 0) result *= n; return result; } public static void main (String[] param) { int n = Integer.parseInt(param[0]); intfact_n = factorial(n); System.out.println("n: " + n + ", n!: " + fact_n); } } Writing a main method for each test case does not scale.

  16. How to Test Effectively? Faulty implementation of enqueue on binary heap: public void enqueue(Comparable o) { if(numElems >= elems.length) grow(); intpos = numElems++; intparent = pos/ 2; while(pos > 0 && elems[parent].compareTo(o) > 0) { elems[pos] = elems[parent]; pos= parent; parent = pos/ 2; } elems[pos] = o; } Writing all test cases manually does not scale.

  17. Automated Testing • Unit Testing: write code to automatically test your code. • A unit test is a test suite for a unit (class/module) of a program and consists of • setup code to initialize the tested class; • tear down code to clean up after testing; • test cases that call methods of the tested class with appropriate inputs and then check the results. • Once test suites are written, they are easy to run repeatedly (regression testing).

  18. Unit Testing in Java: JUnit • A popular framework for unit testing in Java • Frameworks are libraries with gaps • Programmer writes classes following particular conventions to fill in the gaps • Result is the complete product • JUnit automates • the execution and analysis of unit tests; • generation of tests cases from parameterized test oracles and user-provided test data.

  19. JUnit Example import static org.junit.Assert.*; importorg.junit.*; ... public classPriorityQueueTest { privatePriorityQueuepq; @Beforepublic void setUp () { pq = new Heap(); } @Afterpublic void tearDown () { pa = null; } @Test public void enqueueTest () { Integer value = new Integer(5); pq.enqueue(value); assertEquals(pq.removeFirst, value); } ... }

  20. Drawbacks of JUnit • Low degree of automation • Programmer still needs to write all the test cases • Redundant specification • Duplication between checks in test oracles and formal specification (e.g. provided as JML annotations)

  21. JMLUnit: Unit Testing for JML JMLUnit is a unit testing framework for JML built on top of JUnit User: • writes specifications • supplies test data of each type JMLUnitautomatically: • constructs test cases from test data • assembles test cases into test suites • executes test suites • decides success or failure • reports results

  22. Test Cases and Suites • A test case (o,x) consists of: • anon-null receiver object o • a sequence xof argument objects • A test suite for method mis a set of test cases with: • receiver of m’s receiver type • arguments of m’s argument types

  23. Test Suites are Cross Products • For method enqueue:{ (pq, v) | pq∈ PriorityQueueTestData, v∈ IntegerTestData} • Default is to use all data for all methods • Filtered automatically by preconditions • Users can filter manually if desired • Factory method allows user control of adding test cases to test suite.

  24. Errors and Meaningless Test Cases When testing method m: entry precondition violation receiver.m(arg1, ...) internal precondition violation check m’s precondition check f’s precondition { ... x.f(...); } { ... } check f’s postcondition check m’s postcondition other violation Entry precondition violation )test case rejected Internal or other violation) error reported

  25. Supplying Test Data • Programmer supplies data in form of strategies • A strategyfor type T: • has method that returns iterator yielding T • Strategies allow reuse of test data • JMLUnit provides framework of built-in strategies • Strategies for built-in types • Allow for easy extension, composition, filtering, etc.

  26. Strategies for Test Data • Standard strategies: • Immutable: iterate over array of values; • Cloneable: iterate over array, clone each; • Other: create objects each time. • Cloning and creating from scratch can prevent unwanted interference between tests. • JMLUnittries to guess appropriate strategy.

  27. Example Strategies importorg.jmlspecs.jmlunit.strategies.*; importjunit.framework.*; public abstract classHeap_JML_TestDataextendsTestCase { publicIntIteratorvCompIter(String methodName, intargNum) { returnvComparableStrategy.ComparableIterator(); } privateStrategyTypevComparableStrategy = newImmutableObjectAbstractStrategy() { protected Object[] addData() { return newInteger[] {10, -22, 55, 3000}; } }; ...

  28. Example Strategies ... publicIndefiniteIteratorvHeapIter (String methodName, intargNum) { returnvPointStrategy.iterator(); } privateStrategyTypevHeapStrategy = newNewObjectAbstractStrategy() { protectedObject make(int n) { switch(n) { case0: return new Heap(); case1: return new Heap(new Integer {1, 2, 3}); default: break; } throw new NoSuchElementException(); } }; }

  29. Using JMLUnit • JML-compile the class to be tested jmlc Factorial.java • generate the test suite and test data templates jmlunit Factorial.java • supply the test data $EDITOR Factorial_JML_TestData.java • compile the test suite export CLASSPATH=.:$JMLDIR/bin/jmlruntime.jar:$JMLDIR/specs javacFactorial_JML_Test*.java • execute the test suite jmlracFactorial_JML_Test

  30. Drawbacks of JMLUnit • High memory consumption: • JUnit test suite with all test cases is constructed in memory before it is executed. • Limited degree of automation: • only test data for primitive types is generated automatically • Limited degree of granularity: • fine-grained filtering of test data for individual methods is difficult

  31. Alternatives to JMLUnit • JMLUnitNG • similar feature set as JMLUnit • lazy generation of test suites and lazy result aggregation (better memory footprint) • improved filtering of test data • not based on JUnit • Korat, TestEra, UDITA • automated generation of test data for complex data types

  32. Automated Test Case Generation with Korat • Provides test case generation for complex data types. • Supports checking of JML specifications. • User provides for each complex data type • a Java predicate capturing the representation invariant of the data type; • a finitization of the data type. • Korat generates test cases for all instances that satisfy both the finitization constraints and the representation predicate.

  33. Example: Binary Trees importjava.util.*; classBinaryTree { private Node root; privateint size; static class Node { private Node left; private Node right; } ...}

  34. Representation Predicate for BinaryTree publicbooleanrepOK() { if (root == null) return size == 0; Set visited = newHashSet(); visited.add(root); LinkedListworkList = newLinkedList(); workList.add(root); while (!workList.isEmpty()) { Node current = (Node) workList.removeFirst(); if (current.left != null) { if (!visited.add(current.left)) return false; worklist.add(current.left); } if (current.right!= null) { ... } } returnvisited.size () == size; }

  35. Finitization for BinaryTree public static FinitizationfinBinaryTree (intNUM_Node) { IFinitization f = newFinitization(BinaryTree.class); IObjSet nodes = f.createObjSet(Node.class, NUM_Node, true); // #Node = NUM_Node f.set(“root”, nodes); // root in null + Node IIntSet sizes = f.createIntSet(Num_Node); f.set(“size”, sizes); // size = NUM_Node f.set(“Node.left”, nodes); // Node.left in null + Node f.set(“Node.right”, nodes); // Node.right in null + Node return f; }

  36. Finitization for BinaryTree Instances generated for finBinaryTree(3) left right right left right left right right left left

More Related