An Inheritance-Based Technique for Building Simulation Proofs Incrementally

1. An Inheritance-Based Technique for Building Simulation Proofs Incrementally Idit Keidar, Roger Khazan, Nancy Lynch, Alex Shvartsman MIT Lab for Computer Science Theory of Distributed Systems Group

2. State of the Art Software Engineering • Managing complexity of software systems • Modularity: interacting system components • Incremental techniques: OO, inheritance • Formal modeling and verification • Modularity: compositional theorems • Incremental techniques: lag behind • Limited scalability • Not sufficiently cost-effective

3. Our Work:Scalable Formal Methods • Provide incremental techniques for • Specifying systems • Modeling systems • Reusing formal proofs about systems • Evolved as part of our experience modeling complex group communication service • Middleware with intricate semantics • Implemented in C++ • [Keidar, Khazan, ICDCS 2000]

4. Talk Outline • Motivation: need formal framework for incremental proofs • The approach, in a nutshell • Background: specifications, simulation proofs • The challenge: reuse simulation proofs • The solution • Modification constructs • Proof reuse theorem • Experience using the technique • Summary

5. Our Approach, in a Nutshell

6. OO SWE Techniques Formal Modeling and Verification Techniques Incremental Specification, Modeling, and Proof Reuse Techniques The Idea

7. Inheritance • Incremental modification of components • Many different kinds. We consider: • Specialization (sub-typing, substitution) • Child constraints parent behavior • E.g., Parent is unordered messaging protocol; Child specializes Parent to ordered messaging • Interface extension • Together with specialization allows new behavior, but does not override parent behavior (“sub-classing for extension”)

8. parent parent Specification S’ System A’ Inheritance in Spec and Proofs Specification S System A Implements ?! Prove that A’ implements S’ by relying on proof that A implements S, but without repeating reasoning of that proof.

9. Background: Specifications and Proofs

10. Specifying and Modeling System • Abstract state machines [Lampson; Schneider ’93] • States (named variables) • Actions • Specify in which states each action is enabled and how it modifies state • Some actions are externally observable • State is not externally observable • Execution: “state, action, state, action, …” • Defined behavior: all possible sequences of externally observable actions Traces

11. Example -- Monotonic Sequence SpecificationUpSeq State:Integer last, init any Actions: print(x) pre:x  last eff: last:= x • Sample Traces: • 1, 3, 4, … • -10, -5, 1, 2, 3, 5, 8, 13, …

12. What “Implements” Means? • System A implements specification S if every trace of A is a trace of S “Implements”  “trace inclusion” • A is indistinguishable from S by looking only at A’s traces

13. Example -- Fib Sequence SystemFibSeq State:Integer n=0,m=1 Actions: print(x) pre:x == n + m eff:n := m m := x • Trace: 1, 2, 3, 5, 8, 13, … • is also trace of monotonic sequence UpSeq

14. Proving that A Implements S • Simulation mapping / abstraction function F : {the states of A}  {the states of S} • F maps initial states of A to initial states of S • For every action of A and for every state t, if (t, , t’) is a step of A then there is a sequence of actions of S that starts in F(t), ends in F(t’), and has the same trace as . • Simulation Mapping  Trace Inclusion

15. Spec S: F(t) F(t’) Action  F F System A: t’ t Action   of A Simulates  of S: • pre(S.) holds whenever pre(A.) holds • State of A after eff(A.) is executed maps into state of S after eff(S.), assuming pre-states map

16. F last:=m • For simulation proof, need to show: • pre(FibSeq.print(x))pre(UpSeq.print(x)) (x=n+m)(xlast=m); relies on n0 • last still equals m in post-state FibSeq Implements UpSeq UpSeq.print(x) pre:xlast eff:last:=x FibSeq.print(x) pre:x==n+m eff:n:=m m:=x

17. Simulation Proofs -- Benefits • Complete [e.g., Abadi and Lamport ‘93] • Any finite trace inclusion can be shown • Tractable • Inductive reasoning • Reason about single steps, not about executions • Verifiable by humans and/or machines

18. The Challenge: Reuse of Simulation Proofs

19. Specification S System A Simulates Parent Parent Specification S’ System A’ ?! Our Proof Reuse Goal A formal framework • forproving that A’ simulates S’ • without repeating parent’s proof.

20. Why Not Immediate Traces of S Traces of A Traces of S’ Traces of S’ Traces of S’ Traces of A’

21. The Solution: Formal Framework for Incremental Specifications and Simulation Proofs

22. What We Did • Specialization and interface extensionfor specifying and modeling systems • “Proof Reuse” Theorem • Defines simulation between children • Reuses and extends simulation between parents • Requires proving conditions about extension • Involves reasoning only about modifications

23. Specialization Construct • Child may only restrict parent behavior • In precondition-effect notation, child can • Introduce new state variables • Restrict parent actions with new preconditions • Add new effects that modify new variables only A’ = specialize(A)(NewVars, ActionRestriction)

24. Recall: Monotonic Sequence SpecificationUpSeq State:Integer last, init any Actions: print(x) pre:x  last eff: last:= x • Sample Traces: • 1, 3, 4, … • -10, -5, 1, 2, 3, 5, 8, 13, …

25. Specialization of UpSeq: Accelerating Sequence, “Guiness” SpecificationGnSeq specializesUpSeq New State:Integer diff, init any Action Restriction: print(x) new pre:x-last  diffnew eff:diff:= x-last • Sample Trace: 1, 2, 3, 5, 8, 13, … • also trace of UpSeq

26. Using “Proof Reuse” Theorem • In order to show simulation from A’ to S’: • Extend simulation mapping F from A to S with mapping F’from states of A’ to new variables of S’ • Reason only about how S’ restricts S: • new preconditions are enabled; and • mapping F’ is preserved after new effects occur • Allows to reuseF and simulation proof of F

27. F’ FibSeq ?! Example F UpSeq FibSeq Simulates Specializes GnSeq

28. GnSeq.print(x) pre:x-lastdiff eff: diff:=x-last • For simulation proof, need to show : • pre(FibSeq.print(x))  newpre(GnSeq.print(x)) (x=n+m)(x-last  diff) • diff still equals m-n in post-state FibSeq Implements GnSeq last:=m UpSeq.print(x) pre:xlast eff: last:=x FibSeq.print(x) pre: x==n+m eff: n:=m m:=x diff:=m-n

29. Experience: Using the Technique

30. Group Communication • Powerful building blocks for fault tolerant distributed systems • Reliable multicast to groups • Group membership “who is in the group” • Virtual Synchrony semantics synchronize messages and membership changes • Processes see events in same order • Formal specification, modeling, and verification: a challenge

31. Keidar and Khazan, ICDCS 2000 • Modeling a full-fledged group communication service • Specification • Algorithm description, matching C++ implementation (9,000 lines) • Environment specification matching services we use (developed by other teams): • membership server (20,000 lines) • reliable communication service (4,000 lines) • Simulation proof from algorithm to spec

32. Incremental Specification, Algorithm and Proof • FIFO communication in views • Algorithm: half page • Proof 1: 5 pp., 7 major invariants, history vars • Virtual Synchrony + Transitional Set • Algorithm modification: full page • Proof 2: 2.5 pp., 1 invariant • Proof 3: 2.5 pp., 3 invariants, prophecy vars • Self Delivery • Proof 4: 2.5 pp., 3 invariants

33. A Modeling Methodology • Child cannot write to parent data structures • Sometimes need hooks at parent for potential children to specify policy • Non-determinism at parent • Parent specifies coherence, safety • Like “abstract” or “virtual” methods • Example: forwarding of messages to others • At parent arbitrary forwarding, but forwarded messages go into “right” place • Child specifies forwarding policy

34. Benefits of Reuse • Present complex algorithm step by step • Easy to see which part of algorithm corresponds to which part of spec • Efficient: re-use of data structures (as opposed to compositional layered approach) • Manageable proof • Focus attention on specific property • No need to re-prove that previous proof preserves (contrast with layers)

35. Summary of Contributions • Formal framework for incremental development of specs & simulation proofs • Child simulation proof may reuseparent proof • involves reasoning only about modified parts • Successfully used to model and validate complex communication system [ICDCS 00] • Formalism extends IOA programming and modeling language [Garland&Lynch] • Suitable for other state-machine models

36. The End