Lazy Annotation for Program Testing and Verification

1. Lazy Annotation for Program Testing and Verification Speaker: Chen-Hsuan Adonis Lin Advisor: Jie-Hong Roland Jiang

2. Paper Information • Title • Lazy Annotation for Program Testing and Verification • Author • Kenneth L. McMillan • Publication (2010) • Computer Aided Verification, 22nd International Conference, Edinburgh, UK, pp. 104-118, 2010

3. Outline • Introduction • Overview of algorithm • Case study • Experimental results • Conclusion • Some ideas

4. Introduction • An interpolant-based approach to test generation and model checking for sequential programs • Sequential programs ≠ sequential circuits • The method generates Floyd/Hoare style annotations of the program on demand • Inspired from DPLL approach to Boolean satisfiability which combining search and deduction • Focuses deduction where the search becomes blocked • Deduce facts that guide the search away from the failure

5. Hoare Triple • A triple describes how the execution of a piece of code changes the state of the computation • where P and Q are assertions and C is a command • {P} C {Q} • P is named the precondition and Q the postcondition: • when the precondition is met, the command establishes the postcondition • Example • Empty statement axiom: Quick sort !!

6. Outline • Introduction • Overview of algorithm • Case study • Experimental results • Conclusion • Some ideas

7. Preliminary of algorithm • Designate a set of program locations as goals to be reached • the goals are calls to a function error • The vertices (locations) and edges (statements) of the program’s control flow graph will be labeled with formulas • A label represents a condition under which no goal can be reached • Don’t forget goal = error

8. Flow of algorithm • Initially, there are no labels (no annotation being equivalent to false) • Execute the program symbolically along some chosen path • Each input to the program is represented by a symbolic value pi • In the symbolic state, each program variable is evaluated as a symbolic expression over these parameters

9. Flow of algorithm (con’t) • A constraint is also maintained, which accumulates the conjunction of the branch guards along the chosen path • Block-action • We say state is blocked if the current vertex label is implied, meaning we cannot reach a goal from this state • When we are blocked, we will backtrack along the edge we just executed, annotating it with a new label that blocks that edge

10. Flow of algorithm (con’t) • Decide-action (met branch) • When choosing a branch to execute, we are guided by these edge labels • A blocked edge cannot lead to a goal, so we always continue along an unblocked edge if there is one • When all outgoing edges are blocked in the current state • label the current location with the conjunction of the conditions that block the outgoing edges, thus blocking the current state and causing us to backtrack

11. Outline • Introduction • Overview of algorithm • Case study • Experimental results • Conclusion • Some ideas

12. Case study: simple • Enter this code at l1 • Symbolic state is x = p0 with constraint T (true) • Branching from l1 → l2 (one choice) • Constraint p0 = 0 • At l2, branch to l3 or l6 ? • Neither edge is labeled, choose arbitrarily l6 • At l6, only one choice, branch to l7 • Obtain the UNSAT constraint p0=0 ∧p0<0 x = p0 p0 = 0 p0 = 0 p0=0 ∧p0<0

13. Case study: simple (con’t) • At l7, we are blocked, since F (false) holds in current state • Current annotation is F • Backtrack, annotate edge l6 → l7 • Label l6: x ≥ 0, and we are blocked now since current state: x = 0 (x = 0 implies x ≥ 0) • Backtrack to l2, labeling edge l2 → l6 : x ≥ 0 x ≥ 0 x ≥ 0 F

14. Case study: simple (con’t) • Since the edge l2 → l3 is still unblocked, and follow it (Annotation forced the search in a different direction) • Move on to l4 • Have y = p1 (new input) • At l5, we have constraint p1 ≥ 0 • Arrive at l6 in the state x=p0+p1 with constraints p0 = 0 and p1 ≥ 0 y = p1 p1 > 0 x=p0+p1, (constraints p0=0 and p1≥0) x ≥ 0 F

15. Case study: simple (con’t) • Since “x = p0+p1 & p0 = 0 and p1≥0” implies previous annotation l6: x≥0, we are blocked • The fact we previously learned tells us • There is no path to the goal from current state • Backtrack to l5 and take weakest precondition of x≥0, then gives us l5: x+y≥0 x+y ≥ 0 x ≥ 0 F

16. Case study: simple (con’t) • Backtrack to l4, however, we observe a slight problem • The weakest precondition of x + y ≥ 0 with respect to the (assertion y ≥ 0) is y < 0∨x+y ≥ 0 • The variable y is irrelevant here, and we could just as well block the state with x ≥ 0 • x ≥ 0 is also a precondition, not the the weakest ? x ≥ 0 x+y ≥ 0 x ≥ 0 F

17. Compute precondition with interpolant • It can be computed by interpolant • The precondition: l4: x ≥ 0 (not weakest) • assume y ≥ 0 • l5: x + y ≥ 0 • When the weakest precondition would yield l3 : ∀y. (y < 0 ∨ x+y ≥ 0) • We can simplify this to x ≥ 0, but this requires quantifier elimination, which can be very expensive • By computing preconditions with interpolants, we avoid the need for quantifier elimination

18. Case study: simple (con’t) • Backtrack to l2, labeling the edge (l2 → l3) with x ≥ 0 • At l2, both edges from l2 are blocked • label it with the conjunction of the blocking labels along these edges, yielding l2 : x ≥ 0 • Finally, we label l1 : T, proving that the goal error cannot be reached from l1 T x ≥ 0 x ≥ 0 x+y ≥ 0 x ≥ 0 F

19. Interpolants from Proofs • Given a pair of FOL formulas (A,B), such that A ∧ B is inconsistent, an interpolant for (A,B) is a formula Ā with the following properties: • A implies Ā, • Ā ∧B is unsatisfiable, and • Ā is in L(A) ∩ L(B) • For a given formula ϕ, L(ϕ) will denote the well-formed formulas (wff’s) over the uninterpreted vocabulary of ϕ • Interpolant always exists for inconsistent formulas in FOL (Craig’s lemma)

20. Something about Interpolant • A(X,Y) Λ B(Y,Z) ≡ FALSE • There exists I(Y) such that • A(X,Y)  I(Y) • I(Y) Λ B(Y,Z) ≡ FALSE • The “best” interpolant: • Interpolantion is an Existential Quantification

21. Outline • Introduction • Overview of algorithm • Case study • Experimental results • Conclusion • Some ideas

22. Experimental Results • Comparison of test generation with and without learning • Y axis: # of coverage goals reached • X axis: # of backtracks • Compare # of backtracks • Saturation (all reachable locations reached) • Completion The effect is more pronounced in the large program, with learning reducing backtracks to completion by factor 84

23. Outline • Introduction • Overview of algorithm • Case study • Experimental results • Conclusion • Some ideas

24. Conclusion • Deduce program annotations in response to search failure, much in the way that a DPLL SAT solver learns conflict clauses • Without learning, there are long plateaus during which many paths are explored but no new locations are reached • Learning clearly acts to push the search away from these regions, allowing the search to make steady progress

25. Outline • Introduction • Overview of algorithm • Case study • Experimental results • Conclusion • Some ideas

26. Some Ideas • Apply the concept of learning method of DPLL SAT solving to prune the searching space of other research topic, such as QBF, model checking, and so on • Using interpolants can avoid • the expense of quantifier elimination • Predicate image computation

27. Thanks for your attention