1 / 64

A Compositional Method for Verifying Software Transactional Memory Implementations

A Compositional Method for Verifying Software Transactional Memory Implementations. Serdar Tasiran Koç University I stanbul, Turkey. Thanks: Rustan Leino, Tim Harris, Shaz Qadeer, Mike Barnett, Dave Detlefs, Mike Magruder, Yossi Levanoni,. The STM Verification Problem.

carson
Download Presentation

A Compositional Method for Verifying Software Transactional Memory Implementations

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Compositional Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey Thanks: Rustan Leino, Tim Harris, Shaz Qadeer, Mike Barnett, Dave Detlefs, Mike Magruder, Yossi Levanoni, ...

  2. The STM Verification Problem Software transactional memory (STM): Code blocks marked “atomic” STM implementation guarantees atomic, serialized, non-blocking execution, composability Complexity shifted to STM implementer Complicated, tricky code: Conflict detection, rollback, ordering, ... TM will be used widely Correctness as critical as the rest of the computing platform Runtime, compiler, processor, ... Real STM implementations are ~10K lines Interact with the runtime, OS, garbage collector,... Goal: Devise modular, re-usable method for proving correctness Mechanically check most error-prone parts

  3. Approach At the algorithm-level, STM’s are well understood Key ideas in correctness proof known Algorithm level: Large-grain atomic actions Field write, read Transaction commit, roll back Example: Bartok STM, a write-in-place, lazy-invalidate STM Idea/earlier work: Do algorithm-level proof once Boil down to properties STM must satisfy: EW, VR, CU Sufficient condition for correctness Check if STM implementation satisfies EW, VR, CU Formulate each as sequential assertion check, verify with Boogie

  4. Not so simple! Implementation-level executions Algorithm-level executions Problem 1: More variables STM implementation variables: Logs, per-object metadata, ... Problem 2: Finer-grain, smaller actions  more interleavings Atomic action: One instruction in an STM function implementation Reasoning at implementation level more difficult Serializability proofs messy to write, check Approach: Define, prove correspondence between algorithm- and implementation-level executions Algorithm-level proof carries over to implementation-level executions.

  5. Proof Approach Algorithm-level execution satisfying EW +VR + CU Implementation-level execution

  6. Proof Approach Algorithm-level execution satisfying EW +VR + CU Insert marker actions: “commit”, “undoLastLogEntry” Merge chains of STM internal state transitions “Coarse-atomic” execution with serial undo’s Verify NOWS, VRS properties “Coarse-atomic” execution Abstract Read operations Implementation-level execution

  7. Intuition for proof steps • Abstract some actions • Prove non-interference • Commute • Prove larger blocks atomic • Abstract some actions • Prove non-interference • Commute • Prove larger blocks atomic

  8. Outline OTFJ: “Our” Transactional Featherweight Java Algorithm-level and implementation-level semantics Correctness Pure serializability Algorithm-level proof Distill to three required properties Relating implementation and algorithm levels What to abstract, verify at implementation level? Discussion

  9. “Our” Transactional Featherweight Java (OTFJ) : Syntax P ::= 0 | P|P | t[e] L :: = Class C{f1,f2,...,fn; M1, M2,...,Mk} M :: = m(x1,x2, ..., xp){ e; } s :: = v | s.f | s.m(s1,...,sn) | s.f := s | new C() | lbl: onacid; s; commit | null e :: = s | s; s | spawn s v :: = r | v.fi | v.m(v1,...vn) | v.fi = v

  10. OTFJ: Algorithm-level semantics

  11. OTFJ: Algorithm-level semantics

  12. OTFJ: Algorithm-level semantics

  13. Algorithm-Level OTFJ Semantics • Begin Transaction • Field write • Field read • OK2Commit? • RollBackTransaction OTFJ Program AbstractSTM • OK2Commit  • OK2Commit • Undo last log entry

  14. OTFJ Semantics: (Transactional) Field Read Program State STM State p1 s1 Open4Read(r) p1 s2 read r.fi p2 s2 ”read r.fi” added to transaction read log

  15. Abstract STM:This transitionleft unspecified OTFJ Semantics: Field Read Program State STM State p1 s1 Open4Read(r) p1 s2 read r.fi p2 s2 ”read r.fi” added to transaction read log

  16. OTFJ Semantics: Field Write Program State STM State p1 s1 Open4Write(v) p1 s2 OK2Write(v)  v.fi:= rnew p2 s2 ”write (v.fi, rnew, rold)”added to transaction write/undo log

  17. OTFJ Semantics: Field Write Program State STM State p1 s1 Abstract STM:This transitionleft unspecified Open4Write(v) p1 s2 OK2Write(v)  v.fi:= rnew p2 s2 ”write (v.fi, rnew, rold)”added to transaction write/undo log

  18. OTFJ Semantics: Transaction Commit Program State STM State OK2Commit(s1) p1 s1 Commit p1 s2 p2 s2 • Logs • appended to parent transaction’s logs • discarded if top-level transaction

  19. OTFJ Semantics: Transaction Rollback STM State Program State !OK2Commit(s1) p1 s1 p1 s2

  20. OTFJ Semantics: Transaction Rollback STM State Program State !OK2Commit(s1) p1 s1 s2 p1  Xactionrolled backusing logentries  STM stateupdated to reflect each rolled-back log entry . . . 

  21. Algorithm-level semantics: What is atomic? • Execution fragments considered atomic • STM: Open4Read  Prog: Field Read • STM: Open4Write  Prog: Field Write • STM: CommitTransaction  Prog: Commit • Rolling back entire transaction • UpdateSTMState  UndoLastLogEntry UpdateSTMState  UndoLastLogEntry ... UpdateSTMState  UndoLastLogEntry Rewind program to beginning of transaction 21

  22. Outline • OTFJ: “Our” Transactional Featherweight Java • Algorithm-level and implementation-level semantics • Correctness • Pure serializability • Algorithm-level proof • Distill to three required properties • Relating implementation and algorithm levels • What to abstract, verify at implementation level? • Discussion

  23. Correctness: Equivalence of Executions • Equivalence: Executions equivalent if • Same set of threads • Same program end state • For each thread: Same sequence of actions • Alternative view: • Swapping independent actions yields equivalent execution • Dependent: Access same variable, at least one is a write • Equivalence modulo undo’s • Remove all actions by rolled-back transactions • Is what’s left equivalent?

  24. Correctness: Serializability • Serial execution: • Conflict-free: No transaction is rolled-back • Purely serial: Serial and conflict-free • Serializability • Is each execution equivalent to a serial execution? • Pure serializability • Is each execution equivalent modulo undo’s to a purely serial execution? . . . . . . . . . . . . Action by transaction Tx Action by transaction Tx Must belong to Tx, ora child of Tx

  25. Algorithm-Level Serializability: Sufficient Conditions • Exclusive Writes (EW) No other Tx’ writes to obj betweenOpen4Write(Tx, obj) ....... Commit(Tx)/Undo(Tx) • Valid Reads (VR)obj not modified by another Tx’ during Open4Read(Tx,obj) ....... Commit(Tx) • Correct Undo’s (CU)Open4Update(Tx, obj) ....... Undo(Tx) Theorem: EW + VR + CU ==> Pure serializability State of obj same at these two points

  26. Outline • OTFJ: “Our” Transactional Featherweight Java • Algorithm-level and implementation-level semantics • Correctness • Pure serializability • Algorithm-level proof • Distill to three required properties • Relating implementation and algorithm levels • What to abstract, verify at implementation level? • Discussion

  27. Proof Approach Algorithm-level execution satisfying EW +VR + CU Insert marker actions: “commit”, “undoLastLogEntry” Merge chains of STM internal state transitions “Coarse-atomic” execution with serial undo’s Verify NOWS, VRS properties “Coarse-atomic” execution Abstract Read operations Implementation-level execution

  28. Semantics: Implementation-level Executions High Level Implementation Level 1 2 (th1,1 ) 1 … … m 1 … … 2 2 … 1 n (th2,2 ) 2 m … n

  29. Implementation-level Atomic Actions 1 2 3 4 5 6 7 8 9 10 11

  30. Implementation-level Atomic Actions 1 2 3

  31. Atomic read The rest:Localvariableaccesses

  32. Desired: STM Methods Acting on Single Objects are Atomic Atomic read Atomic compare and swap The rest:Localvariableaccesses

  33. The need for abstraction: Actions do not commute Read CAS CAS • If CAS fails, can’t commute Read or CAS across CAS.

  34. The need for abstraction: Actions do not commute NDRead CAS CAS • NDRead: Non-deterministically • acts like regular Read, or • reads arbitrary value, causes failed CAS • NDRead leading to failed CAS commutes with all actions

  35. Every execution is equivalent to a coarse atomic one ValidateRead(Tx1,obj1) ValidateRead(Tx3,obj3) ValidateRead(Tx1,obj4) Open4Write(Tx2,obj2) ValidateRead(Tx3,obj6) Open4Write(Tx2,obj5) • Abstract read actions • Prove non-interference • Commute • Prove larger blocks atomic

  36. Proof Approach Algorithm-level execution satisfying EW +VR + CU Insert marker actions: “commit”, “undoLastLogEntry” Merge chains of STM internal state transitions “Coarse-atomic” execution with serial undo’s Verify NOWS, VRS properties “Coarse-atomic” execution Abstract Read operations Implementation-level execution

  37. Proof Approach Algorithm-level execution satisfying EW +VR + CU Insert marker actions: “commit”, “undoLastLogEntry” Merge chains of STM internal state transitions “Coarse-atomic” execution with serial undo’s Boogie! Verify NOWS, VRS properties “Coarse-atomic” execution Abstract Read Operations Implementation-level execution

  38. Non-Overlapping Write Spans (NOWS) ==> EW [OpenForUpdate(Tx1, obj), Close*Obj(Tx1, obj) ] does not overlap with [OpenForUpdate(Tx2, obj), Close*Obj(Tx2, obj) ] Approach: Assume Tx executed OpenForUpdate(Tx,obj) and not Close*Obj(Tx,obj) ExclusiveOwner(Tx,obj):A formula that says obj metadata indicates that Tx is the exclusive write owner and other good things. Prove: [Remember all STM methods are atomic]Any possible method execution by another thread Tx_bad leaves ExclusiveOwner(Tx,obj) unchanged. 38

  39. Checking NOWS with Boogie Havoc(obj’s state and metadata) OpenForUpdate(Tx_good, obj) assume( ExclusiveOwner(Tx_good, obj) ); 39

  40. Checking NOWS with Boogie Havoc(obj’s state and metadata) OpenForUpdate(Tx_good, obj) assume( ExclusiveOwner(Tx_good, obj) ); assume( Tx_bad != Tx_good); OpenForUpdate(Tx_bad, obj); assert(ExclusiveOwner(Tx_good, obj)); 40

  41. Valid Read Spans (VRS) ==> VR [OpenForRead(Tx1,obj), (Successful)ValidateRead(Tx1,obj) ] does not overlap with [OpenForUpdate(Tx2, obj), Close*Obj(Tx2, obj) ] 41

  42. Checking VRS with Boogie InterfereWith(Tx, obj); OpenForRead(Tx,obj); InterfereWith(Tx, obj); if (*) OpenForUpdate(Tx,obj); InterfereWith(Tx, obj); ValidateRead(Tx,obj); assert(interferedWith ==> Tx.invalid); 42

  43. Checking VRS with Boogie InterfereWith(Tx, obj); OpenForRead(Tx,obj); InterfereWith(Tx, obj); if (*) OpenForUpdate(Tx,obj); InterfereWith(Tx, obj); ValidateRead(Tx,obj); assert(interferedWith<==> Tx.invalid); 43

  44. Checking VRS with Boogie InterfereWith: Represents effects of ( + Close) (Open + Close)* ( Open + ) InterfereWith (Transaction Tx, Object obj) { while (*) { Tx_bad = non-deterministically chosen transaction assume(Tx_bad != Tx); if (*) OpenForUpdate(Tx_bad, obj); if (*) Close*Obj(Tx_bad, obj); } } 44

  45. Checking VRS with Boogie InterfereWith(Tx, obj); OpenForRead(Tx,obj); InterfereWith(Tx, obj); if (*) OpenForUpdate(Tx,obj); InterfereWith(Tx, obj); ValidateRead(Tx,obj); assert(interferedWith<==> Tx.invalid); 45

  46. Checking VRS with Boogie Challenge: Finding pre- and post-conditions for InterfereWith() Boogie has the loop invariant inference (using abstract interpretation) capability to automate this But had a few bugs InterfereWith (Transaction Tx, Object obj) { while (*) { Tx_bad = non-deterministically chosen transaction assume(Tx_bad != Tx); if (*) OpenForUpdate(Tx_bad, obj); if (*) Close*Obj(Tx_bad, obj); } } 46

  47. Checking VRS with Boogie Wrote pre/post-condition pair, verified by induction, again using Boogie but on straightline code. Simple post-condition: If object opened and closed by Tx_bad, version number is bigger, obj (metadata) is quiescent. Otherwise, obj metadata has info related to Tx_bad in it. InterfereWith (Transaction Tx, Object obj) ensures (interferedWith ==> PostCondition(Tx, obj)); ensures (!interferedWith ==>Unchanged(Tx,obj)) { InterfereWith(Tx, obj); if (*) OpenForUpdate(Tx_bad, obj); if (*) Close*Obj(Tx_bad, obj); } 47

  48. Checking VRS with Boogie: Bugs detected Bugs in STM pseudocode showed up when checking this proof obligation with Boogie Some interleavings had been overlooked in pseudocode Including the one in the PLDI ’06 paper Tx_bad: opens for update Tx_good: opens for read Tx_bad: modifies object Tx_bad: closes updated object Tx_good: opens for update Tx_good: validates read (shouldn’t have) Transaction should be invalidated, it isn’t. After putting in fixes, check passed Possible to make errors in STM design at this level Checks take ~5 minutes on my desktop 48

  49. Proof Approach Algorithm-level execution satisfying EW +VR + CU Insert marker actions: “commit”, “undoLastLogEntry” Merge chains of STM internal state transitions “Coarse-atomic” execution with serial undo’s Verify NOWS, VRS properties “Coarse-atomic” execution Abstract Read Operations Implementation-level execution

  50. ImplementationAlgorithm-level: Committed Transactions ValidateRead(Tx,obj2) ValidateRead(Tx,objn) ValidateRead(Tx,obj3) ValidateRead(Tx,obj1) . . . . . . Commit(Tx) Commute all of Tx’s actions here

More Related