Verifying Concurrent Programs with Chalice

1. Verifying Concurrent Programs with Chalice K. Rustan M. Leino RiSE, Joint work with: Peter Müller (ETH Zurich) Jan Smans (KU Leuven) Special thanks toMike Barnett VMCAI, Madrid, Spain, 18 January 2010

2. Concurrent programs • Interleaving of thread executions • Unbounded number of: threads, locks, … • We need some basis for doing the reasoning • A way of thinking!

3. Chalice • Experimental language with focus on: • Shared-memory concurrency • Static verification • Key features • Memory access governed by a model of permissions • Sharing via locks with monitor invariants • Copy-free non-blocking channels • Deadlock checking, dynamic lock re-ordering • Other features • Classes; Mutual exclusion and readers/writers locks; Fractional permissions; Two-state monitor invariants; Asynchronous method calls; Memory leak checking; Logic predicates and functions; Ghost and prophecy variables

4. Dealing with memory (the heap) • Access to a memory location requires permission • Permissions are held by activation records • Syntax for talking about permission to y: acc(y)

5. demo Inc

6. Transfer of permissions acc(c.y) method Main(){var c := new Counter; callc.Inc();} methodInc() requiresacc(y); ensuresacc(y);{ y := y + 1; }

7. The two halves of a call • call == fork + join is semantically like … but is compiled to more efficient code callx,y := o.M(E, F); forktk := o.M(E, F); joinx,y := tk;

8. demo Parallel Inc

9. Passing permissions to threads classXYZ { var x: int; vary: int; var z: int; method Main() {var c := new XYZ; forkc.A();forkc.B(); } … } method A() requiresacc(x); { x := x + 1; } method B() requiresacc(y) && acc(z); { y := y + z; }

10. Read permissions • acc(y) write permission to y • rd(y) read permission to y • At any one time, at most one thread can have write permission to a location

11. Passing permissions to threads classFib { var x: int; vary: int; var z: int; method Main() {var c := new Fib; forkc.A();forkc.B(); } … } method A() requiresrd(x) && acc(y) { y := x + 21; } method B() requiresrd(x) && acc(z) { z := x + 34; }

12. Fractional permissions • acc(y) 100% permission to y • acc(y, p) p% permission to y • rd(y) read permission to y • Write access requires 100% • Read access requires >0% • = + •  acc(y) acc(y,69) acc(y,31) rd(y) acc(y,)

13. Shared state • What if two threads want write access to the same location? method A() … { y := y + 21; } classFib { vary: int; method Main() {var c := new Fib; forkc.A();forkc.B(); } … } ? acc(c.y) method B() … { y := y + 34; }

14. Monitors method A() … { acquirethis; y := y + 21; releasethis; } classFib { vary: int;invariantacc(y); method Main() {var c := new Fib; share c; forkc.A();forkc.B(); } … } acc(c.y) method B() … { acquirethis; y := y + 34; releasethis; } acc(y)

15. Locks and permissions • The concepts • holding a lock, and • having permissions are orthogonal to one another • In particular: • Holding a lock does not imply any right to read or modify shared variables • Their connection is: • Acquiring a lock obtains some permissions • Releasing a lock gives up some permissions

16. Monitor invariants • Like other specifications, monitors can hold both permissions and conditions • Example: invariantacc(y) && 0 ≤ y acc(y)

17. demo OwickiGries counter [Chalice encoding by Bart Jacobs]

18. Abstraction classMyClass { varx,y: int; predicate Valid { acc(c.x) && acc(c.y) && x ≤ y } … }

19. Abstraction classMyClass { varx,y: int; predicate Valid { acc(c.x) && acc(c.y) && x ≤ y } method New() returns (c: MyClass)ensuresc.Valid; { … } method Mutate()requiresc.Valid;ensuresc.Valid; { … } }

20. Abstraction classMyClass { varx,y: int; predicate Valid { acc(c.x) && acc(c.y) && x ≤ y } method New() returns (c: MyClass)ensuresc.Valid; {var c := newMyClass{ x := 3, y := 5 }; foldc.Valid; } method Mutate()requiresc.Valid;ensuresc.Valid; {unfoldc.Valid;c.y := c.y + 3;foldc.Valid; } } acc(c.x) • acc(c.y)x ≤ y c.Valid

21. Abstraction classMyClass { varx,y: int; predicate Valid { acc(c.x) && acc(c.y) && x ≤ y } method New() returns (c: MyClass)ensuresc.Valid; {var c := newMyClass{ x := 3, y := 5 }; foldc.Valid; } method Mutate()requiresc.Valid;ensuresc.Valid; {unfoldc.Valid;c.y := c.y + 3;foldc.Valid; } } acc(c.x) • acc(c.y)x ≤ y c.Valid c.Valid

22. Channels channelCh(c: Cell, z: int) whereacc(c.y) && c.y ≤ z;

23. Channels channelCh(c: Cell, z: int) whereacc(c.y) && c.y ≤ z; class Cell { varx,y: int; method Producer(ch: Ch) {var c := new C { x := 0, y := 0 }; sendch(c, 5); } method Consumer(ch: Ch) {receivec,z := ch; … } } acc(c.y) acc(c.x)

24. Channels channelCh(c: Cell, z: int) whereacc(c.y) && c.y ≤ z; class Cell { varx,y: int; method Producer(ch: Ch) {var c := new C { x := 0, y := 0 }; sendch(c, 5); } method Consumer(ch: Ch) {receivec,z := ch; … } } acc(c.y)c.y ≤ z

25. Preventing deadlocks A deadlock is the situation where a nonempty set (cycle) of threads each waits for a resource (e.g., lock) that is held by another thread in the set • Deadlocks are prevented by making sure no such cycle can ever occur • The program partially order locks • The program is checked to acquire locks in strict ascending order

26. Wait order • Wait order is a dense partial order(Mu, <<) with a bottom element  • << is the strict version of << • The wait level of an object o is stored in a mutable ghost field o.mu • Accessing o.mu requires appropriate permissions, as for other fields

27. Example: Avoiding deadlocks method M() { acquire a; acquire b; … } method N() { acquire b; acquire a; … } • With these preconditions, both methods verify • The conjunction of the preconditions is false, so the methods can never be invoked at the same time requiresrd(a.mu); requiresrd(b.mu); requiresrd(a.mu) requiresrd(b.mu) requireswaitlevel<< a.mu; requiresa.mu << b.mu; requireswaitlevel<< b.mu; requiresb.mu << a.mu;

28. Setting the wait order • Recall, the wait level of an object o is stored in the ghost field o.mu • Initially, the .mu field is  • The .mu field is set by the share statement: picks some wait level strictly betweenL and H, and sets o.mu to that level • Provided L << H and neither denotes an extreme element, such a wait level exists, since the order is dense • Changing o.mu requires acc(o.mu), as usual share o between L and H;

29. Formal semantics of Chalice • Given as translation to Boogie Chalice • Boogie is an intermediate verification language Boogie Z3

30. Encoding the heap • Chalice: o.f • Boogie: Heap[ o, f ] where Heap is declared to be a map fromobjects and field names to values • To encode permissions, use another map: Mask

31. Encoding a call method M() requires P; ensures Q; call M() = Exhale[[ P ]]; Inhale[[ Q ]]

32. Exhale and Inhale • Defined by structural induction • For expression P without permission predicates • Exhale P ≡ assert P • Inhale P ≡ assume P • Exhaleacc(o.f, p) ≡ assertp ≤ Mask[o,f]; Mask[o,f] := Mask[o,f] – p; • Inhaleacc(o.f, p) ≡if (Mask[o,f] == 0) { havoc Heap[o,f]; } Mask[o,f] := Mask[o,f] + p;

33. Example classCell { var y: int; method Square() requiresacc(y); ensuresacc(y) && y = old(y*y); call M() = assert 100 ≤ Mask[this,y];Mask[this,y] := Mask[this,y] – 100;oldH := Heap;if (Mask[this,y] = 0) { havoc Heap[this,y]; }Mask[this,y] := Mask[this,y] + 100;assume Heap[this,y] = oldH[this,y] * oldH [this,y];

34. demo Boogie translation demo

35. An alternative to old method Square(c: Cell) requiresacc(c.y); ensuresacc(c.y) && c.y == old(c.y*c.y); • Logical constant: Let the verifier figure out K! method Square(c: Cell, ghost K: int) requiresacc(c.y) && K == c.y; ensuresacc(c.y) && c.y == K*K; call Square(c, c.y); call Square(c, *);

36. Square only readsc.y method Square(c: Cell) returns (r: int) requiresacc(c.y); ensuresacc(c.y) && r == c.y*c.y; ε loses procedural abstraction method Square(c: Cell) returns (r: int) requiresacc(c.y, ε); ensuresacc(c.y,ε) &&r == c.y*c.y; method Square(c: Cell, ghost K: perm)returns(r: int) requiresacc(c.y, K); ensuresacc(c.y, K) && r == c.y*c.y;

37. Language questions method Square(c: Cell, ghost K: perm)returns (r: int) requiresacc(c.y, K); ensuresacc(c.y, K) && r == c.y*c.y; • A better notation? rd(c.y)? Does that pick one K for each method activation? • Can these 3 kinds of “parameters” (real, ghost, permission) be treated more uniformly? • Which kinds should be indicated explicitly by the programmer and which should be figured out by the compiler?

38. Chalice summary • Permissions guide what memory locations are allowed to be accessed • Activation records can hold permissions • Permissions can also be stored in various “boxes” (monitors, predicates, channels) • Permissions can be transferred between activation records and boxes • Locks grant mutually exclusive access to monitors

39. Try it for yourself • Chalice (and Boogie) available as open source:http://boogie.codeplex.com • Tutorial and other papers available from:http://research.microsoft.com/~leino