Verification As a Matter Of Course

85 Views

Download Presentation
## Verification As a Matter Of Course

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**VerificationAs a Matter Of Course**Bertrand Meyer ETH Zurich, Eiffel Software & ITMO With Cristiano Calcagno, Carlo Furia, Alexander Kogtenkov*, Martin Nordio, Manuel Oriol+, Yu Pei, Nadia Polikarpova, Emmanuel Stapf*, Julian Tschannen, Stephan van Staden, Yi Wei, Andreas Zeller#& others *Eiffel Software +ETH, then U. of York, now ABB #U. Saarbrücken Draft notes, LASER Summer School, Procchio (Elba), September 2011**Contents**• The goal: trusted components • The EVE environment • Technical developments: • 3.1 Towards full contracts • 3.2 Negative variables & the essence of object-oriented programming • 3.3 Language convention for frames • 3.4 The alias calculus • 3.5 A Calculus of Object Programs • Conclusion and assessment • Not included: concurrency**- 1 -**Scope & Goals**Three forms of software development**Simple Web sites, spreadsheets, … Enterprise computing, simulation, “soft” real-time, … Enterprise computing, simulation, “soft” real-time, … Transportation, critical infrastructure, “hard” real-time, … • 1. Casual • 2. Professional • 3. Critical**Language techniques for professional development**Dynamic allocation Classes, objects Inheritance Polymorphism,dynamic binding Exception handling Closures (agents) Exceptions Genericity ...**Research context (1): method and language**• Our work is based on Eiffel: • Method and language for quality-focused development • For professional systems, including very large ones • Continuous, seamless development • ISO standard, significant user base and code base • Libraries • Excellent development environment: EiffelStudio • Some advantages: • Built-in, deeply ingrained use of contracts: preconditions, postconditions, class invariants • Open-source, no patent or other proprietary issues • We can shape the future method, language and IDE • Java is not a Law of Nature**- 2 -**The EVE environment**Research context (2): EVE**• Eiffel Verification Environment • Open source • Developed at ETH, others’ contributions welcome • Continuous integration (weekly) with EiffelStudio • The platform for trying new ideas • All ETH and ITMO contributions included**Verification As a Matter Of Course**Test results AutoProof Suggestions Inter.prover Test execution EVE AutoTest Test case generation Aliasanalysis Suggestions Invariantinference Sep. logicprover Invariantinference Arbiter AutoFix**Scores**Carlo Furia, Julian Tschannen, Martin Nordio • Failed test: score = -1 • Successful test run: 0 < score <= 0.9 • Correctness proof: score = 1 May be scaled down if some aspects not taken into account, e.g. arithmetic overflow • In practice (AutoProof): • Successful proof: score = 1 • Out-of-memory, timeout: score = 0 • Failed proof with abstract trace: score = -1**Combining static and dynamic techniques**• Static verification: + More modular + Scales up better - Not always feasible - Often tests idealized version of the software - Captures only specified properties • Testing: + Generally possible + Can be automated + Tests software as it is + Can integrate non-software parts - Partial (very!) - Captures only specified properties - Not always feasible**Testing**“Ich bin der Geist, der stets verneint”Goethe, Faust, Act I FiodorChaliapineas Mephistopheles To test a software system is to try to make it fail**AutoTest**Ilinca CiupaAndreas LeitnerYi WeiEmmanuel StapfArno FivaManuel Oriol… • (e.g. IEEE Computer, Sep. 2009) • Integral part of EiffelStudio • Has already uncovered hundreds of bugs in libraries and other software • Three components: • Test generation • Test extraction (from failures) • Manual tests**AutoTest: Test generation**• Input: set of classes • Generates instances, calls routines with automaticallyselected arguments • Oracles are contracts: • Direct precondition violation: skip(precondition satisfaction techniques minimize this) • Postcondition/invariant violation: bug found • Any test (manual or automated) that fails becomes part of the test suite**AutoTest: Test generation**Bernd Schoeller * * Test: s1, s2: SET s2 s1 *SET +SET2 +SET1 + + *: Deferred +: Effective**AutoTest: Test extraction**• Turn a failed execution into a test case**Who finds what faults?**I.Ciupa, A. Leitner,M.Oriol, A. PretschnerICST 2008 (best paper) On a small EiffelBase subset,we compared: • AutoTest • Manual testing (students) (3 classes, 2 with bugs seeded) • User reports from the field • Largely separate kinds of faults • AutoTest: 62% specification, 38% implementation • User reports: 36% specification, 64% implementation**CITADEL: Daikon for Eiffel**Nadia PolikarpovaIlincaCiupaISSTA 2009 How do programmer-written and inferred contracts compare? Can contract inference be used to improve the quality of programmer-written contracts?**CITADEL results**• A high proportion of inferred contract clauses are correct (90%) and relevant (64%) • Contract inference does not find all programmer-written contracts (only 59%) • Programmers do not write all inferred contracts (25%) • Contract inference can strengthen programmer-written postconditions and invariants and find missing precondition clauses**AutoFix: programs that fix themselves**Yi Wei, Yu Pei, Andreas Zeller, ISSTA 2010 Yi Wei, Yu Pei, Carlo Furia, ASE 2011 (etc.) Passing & failing test cases Difference Fix to minimize the difference • 16 faults fixed out of 42 • Some of the fixes are exactly the same as those proposed by professional programmers**- 3.1 -**Towardscompletecontracts**Typical contract: list insertion**count index 1 v -- Previous elements unchanged! put_right (v: G) -- Insert v to the right of cursor require index <= count ensure i_th (index + 1) = v count =oldcount + 1 index =oldindex end**Complete contracts**Bernd Schoeller, Tobias Widmer (2008)Nadia Polikarpova (VSTTE 2010) Contracts are typically incomplete (unlike those of fully formal approaches such as Z) Our solution: • Use models A model is a mathematical interpretation of the structures Model library: MML (Mathematical Model Library) Fully applicative (no side effects, attributes, assignment etc.) But: expressed in Eiffel (preserving executability)**Typical contract: list insertion**count index 1 v -- Previous elements unchanged! put_right (v: G) -- Insert v to the right of cursor require index <= count ensure i_th (index + 1) = v count =oldcount + 1 index =oldindex end**List insertion with model-based contract**note model: sequence, index classLIST [G] ... sequence: MML_SEQUENCE [G] put_right (v: G) -- Insert v to the right of cursor. require index <= sequence.count ensure sequence = old(sequence.front (index).extended (x) + sequence.tail (index + 1)) index = oldindex end ...**Model-based contracts: applications**• On 7 of the most popular EiffelBase classes Testing found 4 “functional” faults by violation of model-based contracts • EiffelBase2: a data structures library with full contracts • Aim is to prove the code against these contracts • Now in progress: specifying application libraries (graphics, networking...)**- 3.2 -**Negative Variables& the Essenceof Object-Oriented Programming**O-O analysis, design & programming**“Principle of general relativity”: every operation in a class is expressed relative to a “current object” (self, this, Current), known only at execution time class C feature r do x := 1 x . f (...) end ... end Need to model this property in the logic**Simplified denotational rule for routines (unqualified call)**Notations: • r: body of r • r : list of formal arguments of r • F [v:e] : Substitution of v for e in F (for variable v and expression e; generalized to vectors of variables and expressions) • Distributed dot (used next), for a list v =<u, v, w, …>: xv = <xu, x v, x v, …> (call r (c)) (s) = r (s [r : c])**Classic axiomatic rule for routines**• {P} r {Q} • {P [r: c]} call r (c) {Q [r: c]} (Ignoring recursion)**Tentative rule for qualified (O-O) calls**• {P and INV} r {Q and INV} • {xP [r: c]} callx r (c) {xQ [r: c]} Standard solution (e.g. Peter Müller’s thesis):in r, replace all occurrences of an attribute a by Current.a (Inadequate!)**Object-oriented calls**Known by the target as x’ y • Negative variables: Current y • x Current= x • Current x = x • x’ x = Current • x x’ = Current • Current’ = Current x’ x Target object call x r (c)**Denotational rule for calls**= xr(s [r: x’c]) • Then for any property P(such that “” distributes over P): • P (call x r (c)) = xP(call r (x’c)) = x(P(r) [r: x’c]) Unqualified calls: (call r (c)) (s) = r (s [r : c]) Qualified calls: (call x r (c)) (s) = xcall r (x’ c) (s)**Adequate Hoare rule for qualified calls**• {P and INV} r {Q and INV} • {xP [r: c]} callx r (c) {xQ [r: c]} Proper rule • {P and INV} r {Q and INV} • {xP [r: x’c]} callx r (c) {xQ [r: x’c]} Reminder: inadequate rule**- 3.3 -**Framing:a language convention**The frame issue:**• Specifying what does not change In a class ACCOUNT: deposit (sum: INTEGER) require ... do... ensure balance = old balance + sum end What about other queries other than balance, e.g. owner, account_number, overdraft_limit...? What about descendant classes, e.g. MAFIA_ACCOUNT?**“Modifies” clauses**• ESC-Java, JML, Spec#: specify what a command can change • Theoretical problems: • This can be an infinite set • Requires careful design for inheritance • Practical problem: tedious to express • Pragmatic observation from survey of JMLcode: • Every item listed in a modifiesclause also appears in the postcondition!**The proposed language convention**• deposit (sum: INTEGER ) • require ... • do... • ensure • balance = old balance + sum • end ; cur: CURRENCY old has_penalty = (cur /= currency) involved (is_overdraft) This also avoids introducing a special language notation to express that a routine is pure A routine may only modify queries listed (outside of an oldexpression) in its postcondition**- 3.4 -**The Alias Calculus**The question asked by the Alias Calculus**Broy volume (IJSI, 2011) Given expressions e and f, and a program location p: Can e and f ever be attached to the same object at p?**An example** x y **Why is this important?**Alias analysis can be a key tool for proving O-O programs (as an alternative to separation logic)**Applications of alias analysis**b -- c = c, i.e. True a ? set_a (c) x.set_a (c) Understand as x.a := c -- y.a = b -- y.a = b x -- x.a = c -- x.a = c y c 1. Without it, cannot apply standard proof techniques to programs involving pointers 2. Concurrent program analysis, in particular deadlock 3. Program optimization**Alias relations**A binary relation is an alias relationif it is symmetric andirreflexive Can alias x to y and y to z but not x to z Relation of interest: “In the computation, x might become aliased to y” Definition: Not necessarily transitive: ifcthen x := y else y := z end**Describing an alias relation**• Set of binary relations on E; formally: P(E x E) D Identity on E = Set difference {[x, y], [y, x], [y, z], [z, y]} {[x, y], [y, x], [x, z], [z, x], [y, z], [z, y]} • If r is a relation in E E, the following is an alias relation: rr r-1― Id [E] • Example: {[x, x], [x, y], [y, z]}= • Generalized to sets: • {x, y, z} =**Canonical form and alias diagrams**x, y, y, z, x, u, v {x, y} { y, z} {x, u, v} x , y y y • Alias diagram: y, z • (not canonical) y y x, u, v x x • Make it canonical: Canonical form of an alias relation: e.g. , meaning None of the sets of variablesis a subset of another**Alias diagrams (non-O-O)**Value nodes Value nodes Value nodes Source node In canonical form: no label is subset of another; each label has at least two variables x , y y, z x, u, v