Software Model Checking with SLAM

1. Software Model Checking withSLAM Thomas Ball Testing, Verification and Measurement Sriram K. Rajamani Software Productivity Tools Microsoft Research http://research.microsoft.com/slam/

2. People behind SLAM Summer interns • Sagar Chaki, Todd Millstein, Rupak Majumdar (2000) • Satyaki Das, Wes Weimer, Robby (2001) • Jakob Lichtenberg, Mayur Naik (2002) Visitors • Giorgio Delzanno, Andreas Podelski, Stefan Schwoon Windows Partners • Byron Cook, Vladimir Levin, Abdullah Ustuner

3. Outline • Part I: Overview (30 min) • overview of SLAM process • demonstration (Static Driver Verifier) • lessons learned • Part II: Basic SLAM (1 hour) • foundations • basic algorithms (no pointers) • Part III: Advanced Topics (30 min) • pointers + procedures • imprecision in aliasing and mod analysis

4. Part I: Overview of SLAM

5. What is SLAM? SLAM is a software model checking project at Microsoft Research • Goal: Check C programs (system software) against safety properties using model checking • Application domain: device drivers Starting to be used internally inside Windows • Working on making this into a product

6. Read for understanding Drive testing tools Precise API Usage Rules (SLIC) New API rules Defects Software Model Checking 100% path coverage Rules Static Driver Verifier Development Testing Source Code

7. SLAM – Software Model Checking • SLAM innovations • boolean programs: a new model for software • model creation (c2bp) • model checking (bebop) • model refinement (newton) • SLAM toolkit • built on MSR program analysis infrastructure

8. SLIC • Finite state language for stating rules • monitors behavior of C code • temporal safety properties (security automata) • familiar C syntax • Suitable for expressing control-dominated properties • e.g. proper sequence of events • can encode data values inside state

9. Locking Rule in SLIC state { enum {Locked,Unlocked} s = Unlocked; } KeAcquireSpinLock.entry { if (s==Locked) abort; else s = Locked; } KeReleaseSpinLock.entry { if (s==Unlocked) abort; else s = Unlocked; } State Machine for Locking Rel Acq Unlocked Locked Rel Acq Error

10. The SLAM Process boolean program c2bp prog. P prog. P’ slic bebop SLIC rule predicates path newton

11. Example Does this code obey the locking rule? do { KeAcquireSpinLock(); nPacketsOld = nPackets; if(request){ request = request->Next; KeReleaseSpinLock(); nPackets++; } } while (nPackets != nPacketsOld); KeReleaseSpinLock();

12. Example Model checking boolean program (bebop) do { KeAcquireSpinLock(); if(*){ KeReleaseSpinLock(); } } while (*); KeReleaseSpinLock(); U L L L U L U L U U E

13. Example Is error path feasible in C program? (newton) do { KeAcquireSpinLock(); nPacketsOld = nPackets; if(request){ request = request->Next; KeReleaseSpinLock(); nPackets++; } } while (nPackets != nPacketsOld); KeReleaseSpinLock(); U L L L U L U L U U E

14. Example Add new predicate to boolean program (c2bp) b : (nPacketsOld == nPackets) do { KeAcquireSpinLock(); nPacketsOld = nPackets;b = true; if(request){ request = request->Next; KeReleaseSpinLock(); nPackets++;b = b ? false : *; } } while (nPackets != nPacketsOld); !b KeReleaseSpinLock(); U L L L U L U L U U E

15. Example Model checking refined boolean program (bebop) b : (nPacketsOld == nPackets) do { KeAcquireSpinLock(); b = true; if(*){ KeReleaseSpinLock(); b = b ? false : *; } } while ( !b); KeReleaseSpinLock(); U L b L b L b U b !b L U b L U b U E

16. Example Model checking refined boolean program (bebop) b : (nPacketsOld == nPackets) do { KeAcquireSpinLock(); b = true; if(*){ KeReleaseSpinLock(); b = b ? false : *; } } while ( !b); KeReleaseSpinLock(); U L b L b L b U b !b L U b L b U

17. Observations about SLAM • Automatic discovery of invariants • driven by property and a finite set of (false) execution paths • predicates are not invariants, but observations • abstraction + model checking computes inductive invariants (boolean combinations of observations) • A hybrid dynamic/static analysis • newton executes path through C code symbolically • c2bp+bebop explore all paths through abstraction • A new form of program slicing • program code and data not relevant to property are dropped • non-determinism allows slices to have more behaviors

18. Part I: Demo Static Driver Verifier results

19. Part I: Lessons Learned

20. SLAM • Boolean program model has proved itself • Successful for domain of device drivers • control-dominated safety properties • few boolean variables needed to do proof or find real counterexamples • Counterexample-driven refinement • terminates in practice • incompleteness of theorem prover not an issue

21. What is hard? • Abstracting • from a language with pointers (C) • to one without pointers (boolean programs) • All side effects need to be modeled by copying (as in dataflow) • Open environment problem

22. What stayed fixed? • Boolean program model • Basic tool flow • Repercussions: • newton has to copy between scopes • c2bp has to model side-effects by value-result • finite depth precision on the heap is all boolean programs can do

23. What changed? • Interface between newton and c2bp • We now use predicates for doing more things • refine alias precision via aliasing predicates • newton helps resolve pointer aliasing imprecision in c2bp

24. Scaling SLAM • Largest driver we have processed has ~60K lines of code • Largest abstractions we have analyzed have several hundred boolean variables • Routinely get results after 20-30 iterations • Out of 672 runs we do daily, 607 terminate within 20 minutes

25. Scale and SLAM components • Out of 67 runs that time out, tools that take longest time: • bebop: 50, c2bp: 10, newton: 5, constrain: 2 • C2bp: • fast predicate abstraction (fastF) and incremental predicate abstraction (constrain) • re-use across iterations • Newton: • biggest problems are due to scope-copying • Bebop: • biggest issue is no re-use across iterations • solution in the works

26. SLAM Status • 2000-2001 • foundations, algorithms, prototyping • papers in CAV, PLDI, POPL, SPIN, TACAS • March 2002 • Bill Gates review • May 2002 • Windows committed to hire two people with model checking background to support Static Driver Verifier (SLAM+driver rules) • July 2002 • running SLAM on 100+ drivers, 20+ properties • September 3, 2002 • made initial release of SDV to Windows (friends and family) • April 1, 2003 • made wide release of SDV to Windows (any internal driver developer)

27. Part II: Basic SLAM

28. C- Types  ::= void | bool | int | ref  Expressions e ::= c | x | e1 op e2 | &x | *x LExpression l ::= x | *x Declaration d ::=  x1,x2 ,…,xn Statements s ::= skip | goto L1,L2 …Ln | L: s | assume(e) | l = e | l = f (e1 ,e2 ,…,en) | return x | s1; s2 ;… ; sn Procedures p ::=f (x1: 1,x2 : 2,…,xn : n ) Program g ::= d1 d2 … dn p1 p2 … pn

29. C-- Types  ::= void | bool | int Expressions e ::= c | x | e1 op e2 LExpression l ::= x Declaration d ::=  x1,x2 ,…,xn Statements s ::= skip | goto L1,L2 …Ln | L: s | assume(e) | l = e | f (e1 ,e2 ,…,en) | return | s1; s2 ;… ; sn Procedures p ::=f (x1: 1,x2 : 2,…,xn : n ) Program g ::= d1 d2 … dn p1 p2 … pn

30. BP Types  ::= void | bool Expressions e ::= c | x | e1 op e2 LExpression l ::= x Declaration d ::=  x1,x2 ,…,xn Statements s ::= skip | goto L1,L2 …Ln | L: s | assume(e) | l = e | f (e1 ,e2 ,…,en) | return | s1; s2 ;… ; sn Procedures p ::=f (x1: 1,x2 : 2,…,xn : n ) Program g ::= d1 d2 … dn p1 p2 … pn

31. Syntactic sugar goto L1, L2; L1: assume(e); S1; goto L3; L2: assume(!e); S2; goto L3; L3: S3; if (e) { S1; } else { S2; } S3;

32. Example, in C void cmp (int a , int b) { if (a == b) g = 0; else g = 1; } int g; main(int x, int y){ cmp(x, y); if (!g) { if (x != y) assert(0); } }

33. Example, in C-- int g; main(int x, int y){ cmp(x, y); assume(!g); assume(x != y) assert(0); } void cmp(int a , int b) { goto L1, L2; L1: assume(a==b); g = 0; return; L2: assume(a!=b); g = 1; return; }

34. The SLAM Process boolean program c2bp prog. P prog. P’ slic bebop SLIC rule predicates path newton

35. c2bp: Predicate Abstraction for C Programs Given • P : a C program • F = {e1,...,en} • each ei a pure boolean expression • each ei represents set of states for which ei is true Produce a boolean programB(P,F) • same control-flow structure as P • boolean vars {b1,...,bn} to match {e1,...,en} • properties true of B(P,F) are true of P

36. Assumptions Given • P : a C program • F = {e1,...,en} • each eia pure boolean expression • each eirepresents set of states for which ei is true • Assume: each ei uses either: • only globals (global predicate) • local variables from some procedure (local predicate for that procedure) • Mixed predicates: • predicates using both local variables and global variables • complicate “return” processing • covered in advanced topics

37. C2bp Algorithm • Performs modular abstraction • abstracts each procedure in isolation • Within each procedure, abstracts each statement in isolation • no control-flow analysis • no need for loop invariants

38. int g; main(int x, int y){ cmp(x, y); assume(!g); assume(x != y) assert(0); } void cmp (int a , int b) { goto L1, L2 L1: assume(a==b); g = 0; return; L2: assume(a!=b); g = 1; return; } Preds: {x==y} {g==0} {a==b}

39. int g; main(int x, int y){ cmp(x, y); assume(!g); assume(x != y) assert(0); } void cmp (int a , int b) { goto L1, L2 L1: assume(a==b); g = 0; return; L2: assume(a!=b); g = 1; return; } void cmp ( {a==b} ) { } decl {g==0} ; main( {x==y} ) { } Preds: {x==y} {g==0} {a==b}

40. int g; main(int x, int y){ cmp(x, y); assume(!g); assume(x != y) assert(0); } void equal (int a , int b) { goto L1, L2 L1: assume(a==b); g = 0; return; L2: assume(a!=b); g = 1; return; } void cmp ( {a==b} ) { goto L1, L2; L1: assume( {a==b} ); {g==0} = T; return; L2: assume( !{a==b} ); {g==0} = F; return; } decl {g==0} ; main( {x==y} ) { cmp( {x==y} ); assume( {g==0} ); assume( !{x==y} ); assert(0); } Preds: {x==y} {g==0} {a==b}

41. C-- Types  ::= void | bool | int Expressions e ::= c | x | e1 op e2 LExpression l ::= x Declaration d ::=  x1,x2 ,…,xn Statements s ::= skip | goto L1,L2 …Ln | L: s | assume(e) | l= e | f (e1 ,e2 ,…,en) | return | s1; s2 ;… ; sn Procedures p ::=f (x1: 1,x2 : 2,…,xn : n ) Program g ::= d1 d2 … dn p1 p2 … pn

42. Abstracting Assigns via WP • Statement y=y+1 and F={ y<4, y<5 } • {y<4},{y<5} = ((!{y<5} || !{y<4}) ? F : *), {y<4} ; • WP(x=e,Q) = Q[x -> e] • WP(y=y+1, y<5) = (y<5) [y -> y+1] = (y+1<5) = (y<4)

43. WP Problem • WP(s, ei) not always expressible via { e1,...,en} • Example • F = { x==0, x==1, x<5} • WP( x=x+1,x<5 ) = x<4 • Best possible: x==0 || x==1

44. Abstracting Expressions via F • F = { e1,...,en} • ImpliesF(e) • best boolean function over F that impliese • ImpliedByF(e) • best boolean function over F implied bye • ImpliedByF(e) = !ImpliesF(!e)

45. e ImpliedByF(e) ImpliesF(e) ImpliesF(e) and ImpliedByF(e)

46. Computing ImpliesF(e) • minterm m = d1 && ... && dn • where di = ei or di = !ei • ImpliesF(e) • disjunction of all minterms that imply e • Naïve approach • generate all 2n possible minterms • for each minterm m, use decision procedure to check validity of each implication me • Many optimizations possible

47. Abstracting Assignments • if ImpliesF(WP(s, ei)) is true before s then • ei is true after s • if ImpliesF(WP(s, !ei)) is true before s then • ei is false after s {ei} = ImpliesF(WP(s, ei)) ? true : ImpliesF(WP(s, !ei)) ? false : *;

48. Assignment Example Statement in P: Predicates in E: y = y+1; {x==y} Weakest Precondition: WP(y=y+1, x==y) = x==y+1 ImpliesF( x==y+1 ) = false ImpliesF( x!=y+1 ) = x==y Abstraction of assignment in B: {x==y} = {x==y} ? false : *;

49. Absracting Assumes • WP( assume(e) , Q) = eQ • assume(e) is abstracted to: assume( ImpliedByF(e) ) • Example: F = {x==2, x<5} assume(x < 2) is abstracted to: assume( {x<5} && !{x==2} )

50. Abstracting Procedures • Each predicate in F is annotated as being either global or local to a particular procedure • Procedures abstracted in two passes: • a signature is produced for each procedure in isolation • procedure calls are abstracted given the callees’ signatures