Automatic abstraction in smt based unbounded software model checking
This presentation is the property of its rightful owner.
Sponsored Links
1 / 72

Automatic Abstraction in SMT-Based Unbounded Software Model Checking PowerPoint PPT Presentation


  • 77 Views
  • Uploaded on
  • Presentation posted in: General

Automatic Abstraction in SMT-Based Unbounded Software Model Checking. Anvesh Komuravelli Carnegie Mellon University. Joint work with Arie Gurfinkel , Sagar Chaki and Edmund Clarke. The Problem. Safe. + Proof. Automatic analysis for assertion failures. Program P + Assertions. Unsafe.

Download Presentation

Automatic Abstraction in SMT-Based Unbounded Software Model Checking

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Automatic abstraction in smt based unbounded software model checking

Automatic Abstraction in SMT-Based Unbounded Software Model Checking

AnveshKomuravelli

Carnegie Mellon University

Joint work with ArieGurfinkel, SagarChaki and Edmund Clarke


The problem

The Problem

Safe

+ Proof

Automatic analysis for

assertion failures

Program P

+ Assertions

Unsafe

+ Counterexample

Is it empty?

Unknown

+ Partial Proof

error(P)

reach(P)

Software Model Checking


Over approximation driven od

Over-approximation Driven (OD)

error(P)

reach(P)


Over approximation driven od1

Over-approximation driven (OD)

error(P)

reach(P)


Over approximation driven od2

Over-approximation driven (OD)


Under approximation d riven ud

Under-approximation Driven (UD)

reach(P)

error(P)


Under approximation driven ud

Under-approximation driven (UD)

error(P)

reach(P)


Under approximation driven ud1

Under-approximation driven (UD)


Key recent advancements

Key Recent Advancements


Our strategy

Our Strategy

error(P)

reach(P)

Under-approx.

Abstract

Under-approx.


Our strategy1

Our Strategy

error(P)

reach(P)

Under-approx.

Abstract

Under-approx.

Refine


Our strategy2

Our Strategy

error(P)

reach(P)

Under-approx.

Abstract

Under-approx.

Refine

Abstract


Our strategy3

Our Strategy

error(P)

reach(P)

And so on …


Our strategy4

Our Strategy

reach(P) is covered

error(P)

reach(P)

Abstractions guide the SMT solver to look for general proofs


It s based on ud

It’s based on UD

Under-approximations

A

b

s

t

r

a

c

t


It s based on ud1

It’s based on UD

Under-approximations

need not be monotonic

A

b

s

t

r

a

c

t


Spacer is based on ud

Spacer is based on UD

Under-approximations

non-trivial abstraction

A

b

s

t

r

a

c

t


Spacer

Spacer

Program

Abstract

Under-Approximate

Refine

Proof-Based Abstraction

CEGAR

No

No

Check Safety

Feasible?

Feasible?

Safety Proof

Counterexample

Yes

Yes


Why abstraction

Why Abstraction?

x = y = z = w = 0;

while (*) {

x = *; y = *;

assume (0 ≤ y ≤ 100x);

if (y > 10w && z ≥ 100x) {

y = −y;

}

t = 1;

w += t; z += 10t;

}

assert (0 ≤ y)

only way to fail the assertion


Ud reasoning

UD Reasoning

y ≤ 100x

x = y = z = w = 0;

while (*) {

x = *; y = *;

assume (0 ≤ y ≤ 100x);

if (y > 10w && z ≥ 100x) {

y = −y;

}

t = 1;

w += t; z += 10t;

}

assert (0 ≤ y)

1st Iteration:

w = 0, z = 0


Ud reasoning1

UD Reasoning

y ≤ 100x

x = y = z = w = 0;

while (*) {

x = *; y = *;

assume (0 ≤ y ≤ 100x);

if (y > 10w && z ≥ 100x) {

y = −y;

}

t = 1;

w += t; z += 10t;

}

assert (0 ≤ y)

2nd Iteration:

w = 1, z =10


Ud reasoning2

UD Reasoning

y ≤ 100x

x = y = z = w = 0;

while (*) {

x = *; y = *;

assume (0 ≤ y ≤ 100x);

if (y > 10w && z ≥ 100x) {

y = −y;

}

t = 1;

w += t; z += 10t;

}

assert (0 ≤ y)

3rd Iteration:

w = 2, z = 20

And so on…


Automatic abstraction in smt based unbounded software model checking

But …

x = y = z = w = 0;

while (*) {

x = *; y = *;

assume (0 ≤ y ≤ 100x);

if (y > 10w && z ≥ 100x) {

y = −y;

}

t = 1;

w += t; z += 10t;

}

assert (0 ≤ y)

The value ‘1’ doesn’t matter!


Automatic abstraction in smt based unbounded software model checking

But …

x = y = z = w = 0;

while (*) {

x = *; y = *;

assume (0 ≤ y ≤ 100x);

if (y > 10w && z ≥ 100x) {

y = −y;

}

t = *;

w += t; z += 10t;

}

assert (0 ≤ y)


Ud reasoning on the abstraction

UD Reasoning on the Abstraction

y ≤ 100x

All Iterations

x = y = z = w = 0;

while (*) {

x = *; y = *;

assume (0 ≤ y ≤ 100x);

if (y > 10w && z ≥ 100x) {

y = −y;

}

t = *;

w += t; z += 10t;

}

assert (0 ≤ y)

2nd Iteration

Redundant

w = t, z = 10t

Resolve t away

z = 10w


Original example

Original Example

μZ (SMT-Based Model Checker,

part of Z3)

Cannot solve in an hour

x = y = z = w = 0;

while (*) {

if (*) {x++; y += 100;}

else if (*)

if (x ≥ 4) {x++; y++;}

else if (y > 10w && z ≥ 100x) {

y = −y;

}

t = 1;

w += t; z += 10t;

}

assert (!(x ≥ 4 && y ≤ 2))

Solves an abstraction in < 1 sec.

t = *;

Spacer (our tool)

Finds a proof in a min.

Source: Automatically Refining Abstract Interpretations, Gulavani, Chakraborty, Nori and Rajamani, TACAS ‘08.


What s the magic

What’s the magic?

Focused Proofs

  • Abstractions guide the SMT solver to look for certain kind of proofs

  • Avoid proofs specific to an under-approximation

How to obtain abstractions?

  • From proofs of under-approximations! (Proof-Based Abstraction)

  • Hope: What’s sufficient for the under-approximation is sufficient in general

  • Downside: If abstraction is too coarse, need to refine (CEGAR)


Spacer1

Spacer

Program

Abstract

Under-Approximate

Refine

Proof-Based Abstraction

CEGAR

No

No

Check Safety

Feasible?

Feasible?

Safety Proof

Counterexample

Yes

Yes


Schematic example

Schematic Example

init_stmt;

c = 0;

while (*) {

// invar_1, invar_2

// invar_3, invar_4

assume (c < k1);

if (*) {

v1 = e1; v2 = e2;

} else {

v3 = e3; v4 = e4;

}

v5 = e5; v6 = e6;

c += 1;

}

assert (safe);

Loop Invariants

Add Counters

Under-approximate

Solve


Schematic example1

Schematic Example

Treat as guessed

unbounded invariants.

Essentially like Houdini [FL’01].

init_stmt;

c = 0;

assume (invar_1, invar_2);

while (*) {

// invar_1, invar_2

// invar_3, invar_4

assume (c < k1);

if (*) {

v1 = e1; v2 = e2;

} else {

v3 = e3; v4 = e4;

}

v5 = e5; v6 = e6;

c += 1;

assume (invar_1, invar_2);

}

assert (safe);

Specific to under-approx.

[FL’01] Houdini, an annotation

assistant for ESC/Java,

C. Flanagan and K.R.M. Leino, 2001

Extract Unbounded

Invariants

Unbounded!

Strengthen

with

Invariants

Under-approximate

Feasible?

Solve


Schematic example2

Schematic Example

init_stmt;

c = 0;

assume (invar_1, invar_2);

while (*) {

// invar_1, invar_2

if (*) {

v1 = e1; v2 = e2;

} else {

v3 = e3; v4 = e4;

}

v5 = e5; v6 = e6;

c += 1;

assume (invar_1, invar_2);

}

assert (safe);

Does not prove

the assertion

Under-approximate

Feasible?

NO

Solve


Schematic example3

Schematic Example

init_stmt;

c = 0;

assume (invar_1, invar_2);

while (*) {

// invar_1, invar_2

// invar_3, invar_4

assume (c < k1);

if (*) {

v1 = e1; v2 = e2;

} else {

v3 = e3; v4 = e4;

}

v5 = e5; v6 = e6;

c += 1;

assume (invar_1, invar_2);

}

assert (safe);

Redundant

for the proof

Under-approximate

Feasible?

NO

Abstract

Solve


Schematic example4

Schematic Example

init_stmt;

c = 0;

assume (invar_1, invar_2);

while (*) {

// invar_1, invar_2

// invar_3, invar_4

assume (c < k1);

if (*) {

v1 = e1; v2 = *;

} else {

v3 = e3; v4 = *;

}

v5 = e5; v6 = *;

c += 1;

assume (invar_1, invar_2);

}

assert (safe);

Proof-Based

Abstraction

Under-approximate

Feasible?

NO

Abstract

Solve


Schematic example5

Schematic Example

init_stmt;

c = 0;

assume (invar_1, invar_2);

while (*) {

assume (c < k2);

if (*) {

v1 = e1; v2 = *;

} else {

v3 = e3; v4 = *;

}

v5 = e5; v6 = *;

c += 1;

assume (invar_1, invar_2);

}

assert (safe);

k2 > k1

Abstract

Counterexample!

Concretize

Concrete control

path is infeasible

Under-approximate

Feasible?

NO

Refine

Solve


Schematic example6

Schematic Example

init_stmt;

c = 0;

assume (invar_1, invar_2);

while (*) {

assume (c < k2);

if (*) {

v1 = e1; v2 = e2;

} else {

v3 = e3; v4 = e4;

}

v5 = e5; v6 = *;

c += 1;

assume (invar_1, invar_2);

}

assert (safe);

CEGAR

Under-approximate

Feasible?

NO

Refine

Solve


Schematic example7

Schematic Example

init_stmt;

c = 0;

assume (invar_1, invar_2);

while (*) {

// invar_5

// invar_6

assume (c < k2);

if (*) {

v1 = e1; v2 = e2;

} else {

v3 = e3; v4 = e4;

}

v5 = e5; v6 = *;

c += 1;

assume (invar_1, invar_2);

}

assert (safe);

Unbounded

Invariants

Under-approximate

Feasible?

YES

Solve


Spacer2

Spacer

Program

Abstract

Under-Approximate

Refine

Proof-Based Abstraction

CEGAR

No

No

Check Safety

Feasible?

Feasible?

Safety Proof

Counterexample

Yes

Yes


Detailed example

DetailedExample

x = y = z = w = 0;

while (*) {

if

::x++; y += 100;

::(x ≥ 4) -> x++; y++;

::(y > 10w && z ≥ 100x) -> y = −y;

fi

w++; z += 10;

}

assert (!(x ≥ 4 && y ≤ 2));

if (nd ()) {x++; y += 100;}

else if (nd () && x ≥ 4) {x++; y++;}

else if (y > 10w && z ≥ 100x) {y = −y;}

else assume (0);

C-like

non-deterministic choice

(e.g. as in Promela)


Detailed example1

DetailedExample

x = y = z = w = 0;

c = 0;

while (*) {

// (y > 10w) => (z < 100x), z ≤ 100x,

// x ≤ 2, c ≤ 0 => x ≤ 0, c ≤ 1 => x ≤ 1

assume (c < 2);

if

::x++; y += 100;

::(x ≥ 4) -> x++; y++;

::(y > 10w && z ≥ 100x) -> y = −y;

fi

w++; z += 10;

c += 1;

}

assert (!(x ≥ 4 && y ≤ 2));

Loop Invariants

Add Counters

Under-approximate

Solve


Detailed example2

DetailedExample

x = y = z = w = 0;

c = 0;

while (*) {

// (y > 10w) => (z < 100x), z ≤ 100x,

// x ≤ 2, c ≤ 0 => x ≤ 0, c ≤ 1 => x ≤ 1

assume (c < 2);

if

::x++; y += 100;

::(x ≥ 4) -> x++; y++;

::(y > 10w && z ≥ 100x) -> y = −y;

fi

w++; z += 10;

c += 1;

}

assert (!(x ≥ 4 && y ≤ 2));

Inductive Invariant

Safe

Under-approximate

Solve


Detailed example3

DetailedExample

x = y = z = w = 0;

c = 0;

assume (y > 10w => z < 100x, z ≤ 100x);

while (*) {

//(y > 10w) => (z < 100x), z ≤ 100x,

// x ≤ 2, c ≤ 0 => x ≤ 0, c ≤ 1 => x ≤ 1

assume (c < 2);

if

::x++; y += 100;

::(x ≥ 4) -> x++; y++;

::(y > 10w && z ≥ 100x) -> y = −y;

fi

w++; z += 10;

c += 1;

assume (y > 10w => z < 100x, z ≤ 100x);

}

assert (!(x ≥ 4 && y ≤ 2));

Preserved!

Specific to under-approx.

Extract Unbounded

Invariants

Depend on counter

Strengthen

with

Invariants

Under-approximate

Feasible?

Solve


Detailed example4

DetailedExample

x = y = z = w = 0;

c = 0;

assume (y > 10w => z < 100x, z ≤ 100x);

while (*) {

//(y > 10w) => (z < 100x), z ≤ 100x,

if

::x++; y += 100;

::(x ≥ 4) -> x++; y++;

::(y > 10w && z ≥ 100x) -> y = −y;

fi

w++; z += 10;

c += 1;

assume (y > 10w => z < 100x, z ≤ 100x);

}

assert (!(x ≥ 4 && y ≤ 2));

Does not prove

the assertion

Under-approximate

Feasible?

NO

Solve


Detailed example5

DetailedExample

x = y = z = w = 0;

c = 0;

assume (y > 10w => z < 100x, z ≤ 100x);

while (*) {

// (y > 10w) => (z < 100x), z ≤ 100x,

// x ≤ 2, c ≤ 0 => x ≤ 0, c ≤ 1 => x ≤ 1

assume (c < 2);

if

::x++; y += 100;

::(x ≥ 4) -> x++; y++;

::(y > 10w && z ≥ 100x) -> y = −y;

fi

w++; z += 10;

c += 1;

assume (y > 10w => z < 100x, z ≤ 100x);

}

assert (!(x ≥ 4 && y ≤ 2));

Redundant

Under-approximate

Feasible?

NO

Abstract

Solve


Detailed example6

DetailedExample

x = y = z = w = 0;

c = 0;

assume (y > 10w => z < 100x, z ≤ 100x);

while (*) {

// (y > 10w) => (z < 100x), z ≤ 100x,

// x ≤ 2, c ≤ 0 => x ≤ 0, c ≤ 1 => x ≤ 1

assume (c < 2);

if

::x++; y = *;

::(x ≥ 4) -> x++; y = *;

::(y > 10w && z ≥ 100x) -> y = *;

fi

w = *; z = *;

c += 1;

assume (y > 10w => z < 100x, z ≤ 100x);

}

assert (!(x ≥ 4 && y ≤ 2));

Fails

Enlarge error

Under-approximate

Feasible?

NO

Abstract

Solve


Detailed example7

DetailedExample

x = y = z = w = 0;

c = 0;

assume (y > 10w => z < 100x, z ≤ 100x);

while (*) {

// (y > 10w) => (z < 100x), z ≤ 100x,

// x ≤ 2, c ≤ 0 => x ≤ 0, c ≤ 1 => x ≤ 1

assume (c < 2);

if

::x++; y = *;

::(x ≥ 4) -> x++; y = *;

::(y > 10w && z ≥ 100x) -> y = *;

fi

w = *; z = *;

c += 1;

assume (y > 10w => z < 100x, z ≤ 100x);

}

assert (!(x ≥ 4));

Under-approximate

Feasible?

NO

Abstract

Solve


Detailed example8

DetailedExample

x = y = z = w = 0;

c = 0;

assume (y > 10w => z < 100x, z ≤ 100x);

while (*) {

assume (c < 4);

if

::x++; y = *;

::(x ≥ 4) -> x++; y = *;

::(y > 10w && z ≥ 100x) -> y = *;

fi

w = *; z = *;

c += 1;

assume (y > 10w => z < 100x, z ≤ 100x);

}

assert (!(x ≥ 4));

Counterexample!

Increment x to 4

Choose y arbitrarily

Concretize

Concrete control

path is infeasible

Under-approximate

Feasible?

NO

Refine

Solve


Detailed example9

DetailedExample

x = y = z = w = 0;

c = 0;

assume (y > 10w => z < 100x, z ≤ 100x);

while (*) {

assume (c < 4);

if

::x++; y += 100;

::(x ≥ 4) -> x++; y++;

::(y > 10w && z ≥ 100x) -> y = −y;

fi

w = *; z = *;

c += 1;

assume (y > 10w => z < 100x, z ≤ 100x);

}

assert (!(x ≥ 4 && y ≤ 2));

Under-approximate

Feasible?

NO

Refine

Solve


Detailed example10

DetailedExample

x = y = z = w = 0;

c = 0;

assume (y > 10w => z < 100x, z ≤ 100x);

while (*) {

//(y > 10w) => (z < 100x), z ≤ 100x

// y > 0, (x > 0) => (y ≥ 100)

assume (c < 4);

if

::x++; y += 100;

::(x ≥ 4) -> x++; y++;

::(y > 10w && z ≥ 100x) -> y = −y;

fi

w = *; z = *;

c += 1;

assume (y > 10w => z < 100x, z ≤ 100x);

}

assert (!(x ≥ 4 && y ≤ 2));

Unbounded

Inductive Invariant

Safe

Under-approximate

Feasible?

YES

Solve


Implementation details unbounded invariants

Implementation Details – Unbounded Invariants

Pre-Lemmas

Concrete

Counters

Post-Lemmas

Goal

Find maximal

such that


Implementation details unbounded invariants1

Implementation Details – Unbounded Invariants

SAT?

UNSAT

SAT

with

true

Repeat until fixed point


Implementation details unbounded invariants2

Implementation Details – Unbounded Invariants

Introduce Assumption variables

Maximal subset of true post-lemmas

Minimal number of bi’s to be set to false

Fixed point Iteration:


Implementation details unbounded invariants3

Implementation Details – Unbounded Invariants

disabled

disabled

Iteration 1

Iteration 2


Implementation details abstraction

Implementation Details – Abstraction

Introduce Assumption variables


Implementation details abstraction1

Implementation Details – Abstraction

Are all lemmas necessary?


Implementation details abstraction2

Implementation Details – Abstraction

Introduce Assumption variables for lemmas


Spacer tool

Spacer Tool

Program

Abstract

Under-Approximate

Refine

Proof-Based Abstraction

CEGAR

No

No

Check Safety

Feasible?

Feasible?

Safety Proof

Counterexample

Yes

Yes


Spacer tool1

Spacer Tool

Program

Abstract

Under-Approximate

Refine

Proof-Based Abstraction

CEGAR

No

No

Check Safety

Feasible?

Feasible?

Safety Proof

Counterexample

Yes

Yes

μZ Horn-Clause Solver

(part of Z3)


Spacer tool2

Spacer Tool

Program

Horn-Clause Encoding

Abstract

Under-Approximate

Refine

Proof-Based Abstraction

CEGAR

No

No

Check Safety

Feasible?

Feasible?

Safety Proof

Counterexample

Yes

Yes

μZ Horn-Clause Solver

(part of Z3)


Spacer tool3

Spacer Tool

C Program

Preprocessing

UFO Frontend (based on LLVM)

Simplification, Large Block Encoding, etc.

Horn Clause Encoding

Implemented using UFO Frontend


Results on sv comp 13 benchmarks

Results on SV-COMP’13 Benchmarks

Abstraction did not help

for UNSAFE

ALSO,

not a challenging pool

of benchmarks


Results on sv comp 13 benchmarks1

Results on SV-COMP’13 Benchmarks


Results on sv comp 13 benchmarks2

Results on SV-COMP’13 Benchmarks

~1 min.

Not very meaningful to compare


Results on sv comp 13 benchmarks3

Results on SV-COMP’13 Benchmarks

< 5 min.

Mixed Results


Results on sv comp 13 benchmarks4

Results on SV-COMP’13 Benchmarks

Advantage!


Results on sv comp 13 benchmarks5

Results on SV-COMP’13 Benchmarks

Advantage!

Time-out

Mem-out


Conclusion

Conclusion

Focused Proofs

  • Abstractions guide the SMT solver to look for certain kind of proofs

  • Avoid proofs specific to an under-approximation

How to obtain abstractions?

  • From proofs of under-approximations! (Proof-Based Abstraction)

  • Hope: What’s sufficient for the under-approximation is sufficient in general

  • Downside: If abstraction is too coarse, need to refine (CEGAR)

Contributions

  • A framework for automated abstraction in SMT-based Software Model Checking

  • Implementation using an existing SMT-based model checker with practical advantage


Conclusion contd

Conclusion (contd…)

Why does PBA work?

  • Post-pruning of Proofs during Abstraction (Local vs. Global Proofs)

  • Non-monotonic abstractions

  • Major role of invariants (exploit the generality of proofs of under-approximations

Visit spacer.bitbucket.org to

download tool and detailed slides!


On going and future work

On-going and Future Work

Observation: Fixed granularity of abstraction – at the program level

Observation: Restricted space of abstractions

Questions: When/How to abstract/refine?

Observation: Proofs too dependent on counter constraints (i.e. underapprox.)

Question: How to use counters only when needed? In general, how to minimize the use of a given set of assumptions?

Observation: Abstraction is done offline, after obtaining a proof of an under-approximation.

Question: How does an on-the-fly abstraction work? When each transition is treated as a recursion-free procedure, it is similar to summarizing procedures on-the-fly. Also, how to handle recursion?


Read our cav 13 paper for details

Read our CAV’13 paper for details…

Questions?


Extra slides

Extra Slides


Smt based model checking

SMT-Based Model Checking

Discharge Verification Condition on SMT solver

init

Possibility 1 : UNSAFE

Possibility 2 : SAFE

Path Interpolants (McMillan ‘06)

error

CFG

Loop-Free Unrolling


Smt based model checking1

SMT-Based Model Checking

Discharge Verification Condition on SMT solver

init

Possibility 1 : UNSAFE

Possibility 2 : SAFE

DAG Interpolants [AGC’12]

error

Continue Until Convergence

CFG

[AGC’12] : From Under-approximations to Over-approximations and Back,

Albarghouthi, Gurfinkel and Chechik, TACAS ‘12

Further Unrolling


  • Login