slide1 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Hob: A Tool for Verifying Data Structure Consistency PowerPoint Presentation
Download Presentation
Hob: A Tool for Verifying Data Structure Consistency

Loading in 2 Seconds...

play fullscreen
1 / 83

Hob: A Tool for Verifying Data Structure Consistency - PowerPoint PPT Presentation


  • 96 Views
  • Uploaded on

Patrick Lam, Viktor Kuncak, Martin Rinard MIT CSAIL Massachusetts Institute of Technology Cambridge, MA 02139. Hob: A Tool for Verifying Data Structure Consistency. Goal. Analyze Program to Verify Data Structure Consistency Properties

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Hob: A Tool for Verifying Data Structure Consistency' - fola


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1
Patrick Lam, Viktor Kuncak, Martin Rinard

MIT CSAIL

Massachusetts Institute of Technology

Cambridge, MA 02139

Hob: A Tool for Verifying Data Structure Consistency
slide2

Goal

Analyze Program to Verify Data Structure Consistency Properties

  • Local Properties (within a single data structure) for example, e in List. e.next.prev = e
  • Global Properties (multiple data structures)

for example, no object in list & array

slide3

Challenges

Scalability

  • Detailed static analyses are expensive!

Diversity

  • No single analysis could possibly handle all properties of interest!
applying multiple modular analyses
Inside

Outside

Applying Multiple Modular Analyses

Two views of data structures:

applying multiple modular analyses1
Inside:

full-complexity annotations

analyzed using arbitrary analysis plugin

Outside:

only sets in public module interfaces

all analysis plugins communicate via set specifications

Applying Multiple Modular Analyses

Two views of data structures:

outside module interfaces
Outside Module Interfaces

Public module interfaces are set-based.

For instance, the add procedure’s interface is:

proc add(p:Item)

requires p not in List

modifies List

ensures List’ = List  {p}

add() ensures that p belongs to the list after execution.

benefits of a set specification language
Benefits of a Set Specification Language

Captures important data structure aspects

Captures interface requirements

Provides productive perspective on program

verifying e in list e next prev e
Verifying“e in List. e.next.prev == e”

impl module List {

format Item { next : Item; prev : Item; }

proc add(p:Item) {

if (root==null) {

root = p;

p.next = null; p.prev = null;

} else {

p.next = root; p.prev = null;

root.prev = p; root = p;

}

}

}

assumes “e in List.

next.prev == e”

guarantees “e in List.

next.prev == e”

another property of add
Another property of add()
    • Adds object p to the list.
  • impl module List {
  • format Item { next : Item; prev : Item; }
  • proc add(p:Item) {
  • if (root==null) {
  • root = p;
  • p.next = null; p.prev = null;
  • } else {
  • p.next = root; p.prev = null;
  • root.prev = p; root = p;
  • }
  • }
  • }
verifying add
Verifying add()

proc add(p)

implementation

concrete state

  • Preserves invariant
      • e in List. e.next.prev == e.
  • Adds p to the linked list.
understanding add s effects
Understanding add()’s effects

abstract set

interface

proc add(p)

requires p not in List

modifies List

ensures List’ = List {p}

e in List. e.next.prev == e

e in List. e.next.prev == e

understanding add s effects1
Understanding add()’s effects

abstract set

interface

proc add(p)

requires p not in List

modifies List

ensures List’ = List {p}

e in List. e.next.prev == e

e in List. e.next.prev == e

But what do abstract sets mean?

assigning meanings to sets
Assigning Meanings to Sets

abstract set

e in List. e.next.prev == e

abstraction

function

concrete state

verifying data structure consistency
Verifying Data Structure Consistency

abstract set

interface

proc add(p)

requires p not in List

modifies List

ensures List’ = List {p}

e in List. e.next.prev == e

e in List. e.next.prev == e

abstraction

function

proc add(p)

implementation

concrete state

what happens next
What Happens Next?

invariant

translated interface

proc add(p)

e in List. e.next.prev == e

requires p not in root<next*>

ensures root<next*>’ = root<next*> {p}  frame

analysis plugin

proc add(p)

implementation

client code loops
Client code: Loops

Removing all items from a linked list:

proc clear()

requires init & card(Content’) >= 1 &

(Iter in Content)

modifies Content, Iter

ensures card(Content’) = 0 &

card(Iter’) = 0;

loop implementation
Loop Implementation

Implementation:

proc clear() {

bool e = isEmpty();

while “e’  card(Content’) = 0” {

removeFirst();

e = isEmpty();

}

}

analysis plugins
Analysis Plugins

Hob currently contains three analysis plugins:

  • Flags plugin
    • verifies generalized typestate properties
  • PALE plugin
    • verifies linked list data structure properties
  • Theorem proving plugin
    • verifies arbitrary program properties
verifying more detailed properties
Verifying More Detailed Properties

proc add(p)

implementation

concrete state

  • Preserves invariant o.next.prev == o.
  • Adds p to the linked list.
defining the linked list
Defining the Linked List

abst module List { analysis PALE;

S = { p : Entry | root<next*>p};

  • PALE analysis works with data structures that have a backbone and routing pointers

next

next

defining the linked list1
abst module List { analysis PALE;

S = { p : Entry | root<next*>p};

invariant type L = {

data next : L;

data next : L says that the backbone consists of the next references of the objects

Defining the Linked List

next

next

defining the linked list2
abst module List { analysis PALE;

S = { p : Entry | root<next*>p};

invariant type L = {

data next : L;

pointer prev : L [this^L.next = {prev}];

prev is a routing pointer in the data structure

prev is the inverse of next

So p.next.prev = p.prev.next = p

Defining the Linked List

prev

prev

next

next

defining the linked list3
Defining the Linked List

abst module List { analysis PALE;

S = { p : Entry | root<next*>p};

invariant type L = {

data next : L;

pointer prev : L [this^L.next = {prev}];

};

invariant data root : L;

}

root is the root of a data structure of L’s

prev

prev

root

next

next

procedure specifications and implementations
impl module List {

proc add(p:Item) {

if (root==null) {

root = e;

e.next = null;

e.prev = null;

} else {

e.next = root;

e.prev = null;

root.prev = e;

root = e;

}

} }

spec module List {

proc add(p:Item)

requires p not in S

modifies S

ensures S’ = S  {p};

}

Procedure Specifications and Implementations
slide29

Flags plugin

For this plugin, sets are defined by field values:

“All Entry objects whose field s is set to 2”

is

S = {x : Entry | “x.s = 2” }

We use dataflow analysis, propagating boolean formulas and verifying them with MONA.

slide30

Flags plugin

abst module Simple_flag {

use plugin “flags”;

S = {x : Entry | “x.s = 2” }

}

slide31

Flags plugin

impl module Simple_flag {

format Entry { s:int; }

proc makeS(e:Entry) { e.s = 2; }

}

abst module Simple_flag {

use plugin “flags”;

S = {x : Entry | “x.s = 2” }

}

spec module Simple_flag {

format Entry;

specvar S : Entry set;

proc makeS(e:Entry) modifies S ensures S’ = S + e;

}

verifying even more detailed properties
Verifying Even More Detailed Properties

Sometimes shape analysis just isn’t enough.

Ought to be able to do something!

s

c

a theorem proving example
A Theorem Proving Example

S = { x : Node | “EX i. 0 <= i & i < s & x=d[i] & x ~= null”}

s

c

global data structure consistency properties1
Global Data Structure Consistency Properties

Consider a process scheduler with a set of Running processes and a set of Idle processes.

Running: all processes in running list

Idle: all processes in idle array

Necessary invariant P:

RunningIdle = 

Running

Idle

where must hob check that p holds
Where must Hob check that P holds?

requires P ensures P

Process

Scheduler

problem specification aggregation
Problem: Specification Aggregation!

P Q R

Q R

Q

P  Q

P

Q R

Q

Q

R

  • Specifications become unwieldy
  • Explicit callee preconditions break modularity
beating specification aggregation
Beating Specification Aggregation

These invariants crosscut the specifications for the entire program, and would need to be included in all potential callers.

Scopes enable developers to factor out these invariants and state the invariant once and for all.

our solution to specification aggregation
Our Solution to Specification Aggregation

The developer:

  • Specifies a consistency property
  • Identifies a region where property may be temporarily violated during updates

The Hob analysis system:

  • Verifies the property in the program’s initial state and upon on exit from the region; and
  • May soundly assume the property holds upon entry to the region and outside the region
scopes in example
Scopes in Example

Scheduler Module

suspend(p)

  • RunningIdle =  may be violated anywhere within Scheduler, Idle Process, or Running Process modules
  • Scheduler must coordinate operations on Idle Process and Running Process Modules
  • Otherwise property may become permanently violated

resume(p)

add(p)

ins(p)

empty()

rem(p)

del()

Idle Process Module

Running Process Module

example scope
Example Scope

scope C {

invariant RunningIdle = ;

modules scheduler, idle, running;

export scheduler;

}

  • Property holds except within modules in scope
  • Sets of property included in modules in scope
  • Outside scope, may only invoke procedures in exported modules
scopes and analysis
Scopes and Analysis

System conjoins property to preconditions and postconditions of exported modules

Analysis verifies that 1) property holds initially and 2) procedures preserve property

Scheduler Module

suspend(p)

RunningIdle = 

resume(p)

add(p)

ins(p)

empty()

rem(p)

del()

benefits of scopes
Benefits of Scopes

Specifications may omit many properties that are locally irrelevant (aiding concision).

In particular, can omit crosscutting callee preconditions at caller sites (aiding modularity).

benefits of scopes1
Benefits of Scopes

Specifications may omit many properties that are locally irrelevant (aiding concision).

In particular, can omit crosscutting callee preconditions at caller sites (aiding modularity).

hob framework benchmarks
Hob Framework & Benchmarks
  • Implemented Hob System components:
    • Interpreter
    • Compiler to Java
    • Analysis framework
    • Pluggable analyses
      • Set/flag analysis
      • PALE analysis interface
      • Array analysis (VCs discharged via Isabelle)
  • Modules and programs
    • Data structures
    • Minesweeper, Water
    • HTTP server (http://hob.csail.mit.edu)
benchmark 1 minesweeper1
Benchmark 1: Minesweeper

750 lines of code, 236 lines of specification

Full graphical interface (model/view/controller)

Data structure consistency properties

Board cell state correlations

Correlations between state and actions

benchmark 2 water
Benchmark 2: Water

Time step computation, simulates liquid water

Computation consists of sequence of steps

Predict, correct, boundary box enforcement

Inter and intra molecular force calculations

2000 lines of code, 500 lines of specification

Typestate properties

State correlations – simulation, molecules, atoms

benchmark 3 web server1
Benchmark 3: Web Server
  • 580 lines of code, 218 lines of specification
  • Serves static HTML pages & logs requests
  • Typestate properties (e.g. on sockets)
  • ‘HTTP Request’ data structure:
    • Check proper iteration on request field lists
  • Soon:
    • Disjointness of request field lists
    • Consistency properties of server-side caches
related work
Related Work

Shape analyses

  • Moeller, Schwartzbach PLDI 2001
  • Ghiya, Hendren POPL 1996

Typestate

  • Strom, Yellin IEEE TOSEM 1986
  • DeLine, Fahndrich ECOOP 2004, PLDI 2001

Theorem provers

  • Isabelle, Athena, HOL, PVS, ACL2

Program specification

  • Eiffel, JML, Spec#

Verifiers – Program Verifier, Stanford Pascal Verifier, Larch, ESC/Modula-3/Java, Boogie

primary contribution
Primary Contribution

Hob framework for modular program analysis:

  • Abstract set specification language
  • Scope invariants; defaults and guards

Enables multiple (very precise and unscalable) analyses to interoperate

Verifies data structure consistency properties

Our implemented system combines high-level properties from markedly different analyses

context
Context

Software Systems

Composed of modules, with:

  • Encapsulated data structures
  • Exported procedures
  • Code
issues
Issues
  • Scalability
    • Powerful analyses can verify complex data structure consistency properties
    • Very detailed model of data structures
    • Unthinkable to analyze complete program
  • Diversity
    • Different analyses for different data structures
    • New analyses for new data structures
  • Need for multiple targeted local analyses that interoperate to share information
verifying o next prev o
Verifying “o.next.prev == o”

module M

add(o)

module List

Modules M and List share object o.

impl module List {

format Item { next : Item; prev : Item; }

proc add(o:Item) { … }

}

where to verify consistency properties
Where to verify consistency properties?

module M

add(o)

module List

impl module List {

format Item { next : Item; prev : Item; }

proc add(o:Item) { … }

}

Do we have to analyze module M as well?

aspect oriented constructs
Aspect-Oriented Constructs

Formats allow developers to write implementations which share objects between modules without harming local reasoning.

Scopes and defaults allow developers to write better, more concise, module specifications.

hob s module interface language

proc add()

proc add()

requires p not in S

modifies S

ensures S’ = S {p}

S = { p : Item | root<next*>p};

Hob’s Module Interface Language

Each module has a:

  • implementation section
  • specification section
  • abstraction section

Specifications (module interfaces) are expressed in the boolean algebra of sets

interface
Interface

spec module M {

interface1
Interface

spec module M {

specvar I : Item set;

Modules export abstract sets of objects, which:

  • are simply a specification mechanism
  • do not exist when program runs
  • characterize how objects participate in module’s encapsulated data structures
  • used to define module’s interface
interface2
Interface

spec module M {

specvar I : Item set;

proc add(p : Item)

requires (p  I)  p  null modifies I

ensures I’ = I  {p};

Each exported procedure has requires, modifies, and ensures clauses

Use (quantified) boolean algebra of sets

interface3
Interface

spec module M {

specvar I : Item set;

proc add(p : Item)

requires (p  I)  p  null modifies I

ensures I’ = I  {p};

proc del() returns p : Item

requires |I|  1 modifies I

ensures I’ = I – {p}  p  I  p  null;

  • Can also have cardinality constraints on sets
interface4
Interface

spec module M {

specvar I : Item set;

proc add(p : Item)

requires (p  I)  p  null modifies I

ensures I’ = I  {p};

proc del() returns p : Item

requires |I|  1 modifies I

ensures I’ = I – {p}  p  I  p  null;

proc empty() returns b : bool

ensures b  |I| = 0;

}

benefits of a set spec language 1
Benefits of a Set Spec Language (1)

Capture important data structure aspects

Can capture interface requirements

Provide productive perspective on program

  • Sets characterize changing object roles
  • Set membership changes reflect role changes
benefits of a set spec language 2
Benefits of a Set Spec Language (2)

Promote verified connection between

design (object model) and implementation

abstraction sections connecting interfaces and implementations
Abstraction Sections: Connecting Interfaces and Implementations

abst module idle { analysis PALE;

Idle = { p : Process | root<next*>p};

invariant type L = {

data next : L;

pointer prev : L [this^L.next = {prev}];

};

invariant data root : L;

}

Hob’s analysis plugins use abstraction modules to assign meanings to sets in proc interfaces.

prev

prev

root

next

next

aspect oriented constructs1
Aspect-Oriented Constructs

Formats allow developers to write implementations which share objects between modules without harming local reasoning.

Scopes and defaults allow developers to write better, more concise, module specifications.

standard usage scenario
Standard Usage Scenario

Modules Coordinate Data Structure Operations

Leaf Modules Encapsulate Data Structures

  • Specification/analysis complexity in leaf modules
  • Most developers never even see this complexity
outcalls
Outcalls

So far, all calls enter and exit scopes from top

What about outcalls from scope?

Scope

invariant issue
Invariant Issue

Invariant may be violated inside scope

If callee uses invariant (transitively), must reestablish invariant before call

If callee does not use invariant (transitively), should be able to call with invariant violated

Our approximation: restore invariant before reentrant outcalls

Scope

potential policy variants
Potential policy variants

Could have outcalls without invariant restoration when appropriate

A procedure can declare invariants it uses

If so, can only call procedures that use at most these invariants

If an outcalled procedure does not use invariant, do not need to restore it

slide82

Simple Analysis Example: Boolean Variables

impl module S {

reference b:bool;

proc set_true() { b = true; }

}

spec module S {

specvar b:bool;

proc set_true() modifies b ensures b’;

}