1 / 44

# Verifying Properties of Well-Founded Linked Lists - PowerPoint PPT Presentation

Verifying Properties of Well-Founded Linked Lists. Shuvendu K. Lahiri Shaz Qadeer. Software Reliability Research Microsoft Research. Motivation for analyzing linked lists. Verify control, memory, and API safety of low-level systems code Integers Arrays Singly linked lists

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

## PowerPoint Slideshow about 'Verifying Properties of Well-Founded Linked Lists' - clyde

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Well-Founded Linked Lists

Shuvendu K. Lahiri

Shaz Qadeer

Software Reliability Research

Microsoft Research

• Verify control, memory, and API safety of low-level systems code

• Integers

• Arrays

• Singly linked lists

• Doubly linked lists (acyclic and cyclic)

• Verify control, memory, and API safety of low-level systems code

• Integers

• Arrays

• Singly linked lists

• Doubly linked lists (acyclic and cyclic)

• Establish properties about linking structure and content

• Absence of null dereference, memory leaks

• All elements of a list have data value 0

• List1 and List2 are disjoint

//@ ensures v  R(hd): v.data = 0

void acyclic_simple(Cell hd) {

Cell iter = hd;

while (iter != null) {

iter.data = 0;

iter = iter.next;

}

}

Example: Acyclic linked list iteration

• Existing program analyses either lack scalability or precision for such programs/properties

• Can support many theories important for program verification

• Uninterpreted functions, linear arithmetic, arrays, quantifiers

• Reason about programs with a mix of scalar variables, arithmetic, arrays

• Powerful analysis engines

• Pioneering work by Nelson-Oppen[’79]

• Recent advances in SAT-based theorem provers

• Automated software verification tools

• SLAM, BLAST, MAGIC,…

• ESC/JAVA, Boogie,..

• Perform symbolic reasoning for first-order logic

• Theorem provers to discharge verification conditions

• Operations for abstract interpretation (predicate abstraction, join, ..)

• Automatic abstraction-refinement

x

• Class Cell {

• int data;

• Cell next;

• };

R(x)

R(u) = Set of cells reachable from u using next field

= {u, u.next, u.next.next,…}

//@ requires hd != null

//@ ensures v  R(hd): v.data = 0

void acyclic_simple(Cell hd) {

Cell iter = hd;

while (iter != null) {

iter.data = 0;

iter = iter.next;

}

}

hd

iter

Visited = R(hd)\ R(iter)

Example

Loop invariant

u Visited: u.data = 0

• Need to reason about reachability predicate

• e.g. u R(x): u.data = 0

• Need axioms to relate the field next and R

• However, reachability can’t be modeled in first-order logic

• Finite first-order axiomatization of reachability impossible

• Simple axioms may suffice for many examples

• Provide a first-order axiomatization of Reach

• Necessarily incomplete

• First investigated by Nelson [POPL’83]

• Enable list manipulating programs (also containing integers, arrays etc.) to be analyzed uniformly

• Can leverage first-order reasoning

• Predicate abstraction,…

• Abstraction refinement

• //@ requires hd != null

//@ ensures v  R(hd): v.data = 0

void acyclic_simple(Cell hd) {

Cell iter = hd;

while (iter != null) {

iter.data = 0;

iter = iter.next;

}

}

hd

iter

Visited = R(hd)\ R(iter)

Example

Loop invariant

u Visited: u.data = 0

Axiom for reach:

u, v : v R(u)

(v = u  (u.next  null  v R(u.next)))

//@ requires hd != null

//@ ensures v  R(hd): v.data = 0

void acyclic_simple(Cell hd) {

Cell iter = hd;

while (iter != null) {

iter.data = 0;

iter = iter.next;

}

}

hd

iter

Visited = R(hd)\ R(iter)

Example

Loop invariant

u Visited: u.data = 0

Axiom sufficient to prove the example

Axiom for reach:

u, v : v R(u)

(v = u  (u.next  null  v R(u.next)))

• How to

• Handle cyclic lists

• Handle destructive updates

• Generate first-order axioms for Reach

• Well-founded linked lists

• How it makes the above tasks amenable

• Results

• Deciding ground fragment with Reach predicate

//@ requires hd points to a cyclic list

//@ ensures v  R(hd): v.data = 0

void cyclic_simple(Cell hd) {

hd.data = 0;

iter = hd.next;

while (iter != hd) {

iter.data = 0;

iter = iter.next;

}

}

Part1: Cyclic List Traversal

hd

iter

Visited = ?

No way to express Visited using R alone

• R for every cell in the cycle contains all the cells in the cycle

//@ requires hd points to a cyclic list

//@ ensures v  R(hd): v.data = 0

void cyclic_simple(Cell hd) {

hd.data = 0;

iter = hd.next;

while (iter != hd) {

iter.data = 0;

iter = iter.next;

}

}

Cyclic List Traversal

hd

iter

Visited = ?

Proving even null-dereference is non-trivial

• Usually, every cycle of “next” has at least one distinguished cell

• Usually, the “head” of the list

• This cell breaks the symmetry in the list

• For each linking field “f”, a subset of fields in the heap are heads

• Denoted by Hf

• Cells denoted by

• Always includes null

y

Rf(x)

x

Rf(z)

z

y

New Predicates Rf and Bf

• Hf = Set of head cells for field f

• Rf(u)

• Set of cells u, u.f, u.f.f,…, until the first cell in H

• Bf(u)

• The first cell from the sequence u.f, u.f.f, …, that belongs to H

• The “block” for u

• Bf(x) = null

Bf(x) = y

Bf(y) = x

Bf(z) = x

• Given Hf, a set of “head” cells for a field f

• Well-founded field f

• For any cell u, the sequence u.f, u.f.f, …, intersects with a cell in Hf

• Well-founded heap

• Every linking field f is well-founded wrt to Hf

• i.e., every f cycle has at least one Hf cell

• Programmer must supply Hf

• Every mutation of the linking field f is required to preserve well-foundedness

• Restricted to programs maninpulating well founded heaps only

• Almost all list programs obey this restriction

//@ requires hd points to a cyclic list

//@ ensures v  R(hd): v.data = 0

void cyclic_simple(Cell hd) {

hd.data = 0;

iter = hd.next;

while (iter != hd) {

iter.data = 0;

iter = iter.next;

}

}

Cyclic List Traversal

hd

iter

Visited = ?

//@ requires hd  H  B(hd) = hd

//@ ensures v  R(hd): v.data = 0

void cyclic_simple(Cell hd) {

hd.data = 0;

iter = hd.next;

while (iter != hd) {

iter.data = 0;

iter = iter.next;

}

}

Cyclic List Traversal

hd

R(iter)

iter

Visited = (iter = hd)

? R(iter)

: R(hd) \ R(iter)

Loop invariant:

u Visited: u.data = 0

B(iter) = hd

• Axiom for R

• v  R (u)  (v = u  (u.next  H v  R(u.next))

• v  R(u)  (v = u  (u.next  null  v  R(u.next))

• Axiom for B

• B(u) = u.next  H ? u.next :B(u.next)

• Able to prove the example (similar to acyclic case) with these axioms

• x.f := y

• Issues

• R, B need to be updated

• Since f is updated

• Destructive updates may make the heap ill-founded

• Flag such programs as bad

• Updates to R, B (some cases)

• x.f := y

u

u

x

x

y

y

R(u) = R(u) \ R(x) {x}  R(y)

R(u) = R(u) \ R(x) {x}

B(u) = y

B(u) = B(y)

• x.f := y

Orphan cycle: Cycle with no H cells

x

y

Add assert ( x  R(y)  y  H ) before each x.f := y

• Hfis a program variable now

• Hf.Add(x)

• Adds the cell pointed to by x to Hf

• Useful when creating a cycle for the first time

• Hf.Remove(x)

• Removes the cell pointed to by x to Hf

• Remove one head when two cycles with a head each are fused

• Updates to Rf, Bf still remain quantifier-free

• Quantifier-free updates to auxiliary variables R, B

• Similar to acyclic case [Dong & Su ‘95]

• Very difficult to update R for cyclic lists in general

• Instrumentation captures “well-foundedness” precisely

• The instrumented program goes wrong (violates an assertion)iff the source program

• goes wrong (violates some assertions), or

• heap of the source program becomes not well-founded

• Base axiom for R

• v  R(u)  (v = u  (u.next  H v  R(u.next))

• Base axiom for B

B(u) = u.next  H ? u.next :B(u.next)

• Fundamental axioms

• The axioms capture the intended meaning of R and Bin any finite and well-founded state of the program

• Not possible to express finiteness and well-foundedness in first-order logic

• Derive new axioms from the base axioms

• Using induction

• For well-founded heaps

• We provide an induction principle to generate derived axioms from base axioms

• Proposed axiom: u. P(u)

• To prove P(u) for any cell u in a finite well-founded heap

• Base Case

• u.f  H  P(u)

• Establish for all u at a distance 1 from H cells

• Induction Step

• u.f  H  (P(u.f)  P(u))

• u.f has a shorter distance to H than u (well-founded induction)

• Transitivity

• R(u,v)  R(v,w)  R(u,w)

• Antisymmetry

• R(u,v)  R(v,u)  u = v

• Block

• R(u,v)  v H  u = v

• Bounded distinctness

• All cells in the set {u, u.f,…,} until the first H cell are distinct from each other

• Instantiate this for bounded sizes (e.g. 1)

• u.f  H  u  u.f

• Set of axioms are fairly fundamental properties and fairly intuitive

• Can be easily proved from the base axioms using the induction principle

• Suffice for a large set of examples

• Otherwise derive new axioms using the base axioms and induction

• Set of required axioms almost similar to acyclic case

• Allows us to update Rf, Bf relations using simple quantifier-free formulas

• Provides an induction principle to establish derived axioms easily

Instrumentation

Add R, B

+ Updates

+ Assertions

VC Generator

(UCLID)

Annotated

Source

Program

Theorem Prover

(UCLID)

Proved/Failure

Axioms for R, B

• Verification system for systems modeled in first-order logic

• Bryant, Lahiri, Seshia, CAV’02

• Checking verification conditions

• Uses quantifier instantiation

• Uses various decision procedures for uninterpreted functions, arrays, arithmetic

• Inferring loop invariants with indexed predicate abstraction

• Simple_cyclic

• List traversal

• Reverse

• In place reversal of an acyclic list

• Sorted_insert

• Inserts a cell in a sorted list

• Requires arithmetic reasoning

• Set_union

• Merges two equivalence classes implemented as cyclic lists

• Dlist_remove

• Removes a cell from a cyclic doubly linked list

• Proving Verification Conditions (VCs)

• Most examples take < 1 s

• Loop Invariant synthesis using indexed predicate abstraction

• Flanagan & Qadeer POPL’02

• Lahiri & Bryant VMCAI ‘04

• Principle

• Provide a set of predicates P over state variables + “index variables” X

• Intuitively X contains heap cells, or indices into arrays

• e.g. P = {Rnext(u,v), Bnext (u) = v, a[i] < a[j] + 1, …}

X = {u,v,i,j,…}

• Theorem

• Indexed predicate abstraction constructs the strongest loop invariant of the form X: (P)

•  is a Boolean combination of predicates in P

• //@requires null  Hnext

//@requires Bnext (l) = null

//@ensures Bnext (res) = null

//@ensures Rnext(res) = R0next (l)

Cell reverse (Cell l){

Cell curr = l;

Cell res = null;

while (curr != null){

Cell tmp = curr.next;

curr.next = res;

res = curr;

curr = tmp;

}

return res;

}

Predicates

X = {u}

P = {

u = null, u = curr, u = res, u = l0,

curr = null, l = l0,

Rnext(curr,u),

Rnext(res,u),

Rnext(l,u),

Hnext(u),

R0next(l0,u),

Bnext(u) = null

}

Tool constructs loop invariant in 0.57 sec

Synthesizing invariants in UCLID

• Predicates provided manually

• Used Barcelogic tool for theorem proving

• Note: Results significantly improved from paper

• Deciding ground formulas over

• Rf(u,v), ~Rf(u,v), u = f(v), u ≠ v, u  Hf, u Hf,u = Bf (v)

• Reduce dependency on derived axioms

• A complete framework when VCs are quantifier-free

• Solving quantifier-free queries after instantiating quantifiers

• Result

• Checking satisfiability of a conjunction NP-complete

• First-order axiomatization of reachability

• Nelson ’83, Lev-Ami et al. ’05

• First-order reasoning without reachability

• Necula & McPeak ’05

• Shape analysis with 3-valued logic

• Sagiv et al. ’99, …

• TVLA

• Predicate abstraction for lists

• Dams et al. ’03, Balaban et al. ’05, Manevich et al. ’05, Bingham ’06

• Separation logic

• O’Hearn et al. ’01, Reynolds ’02,

• Two new predicates R, B for well-founded heaps

• Instrumentation of source program with auxiliary variables for the predicates

• First-order axiomatization

• New induction principle

• Simpler derived axioms

• Implementation

• Leverage first-order theorem provers

• Indexed predicate abstraction provides a uniform scheme for synthesizing invariants