Formal software verification
This presentation is the property of its rightful owner.
Sponsored Links
1 / 36

Formal Software Verification PowerPoint PPT Presentation


  • 64 Views
  • Uploaded on
  • Presentation posted in: General

Christelle Scharff http://www.csis.pace.edu/~scharff May 2004. Formal Software Verification. DPS, Pace University. References. Use of the slides of Dr. Clark Barrett (with permission) ITR NSF Little Engines of Proof with Stanford Research Institute and Clarkson University, 2003

Download Presentation

Formal Software Verification

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Formal software verification

Christelle Scharff

http://www.csis.pace.edu/~scharff

May 2004

Formal Software Verification

DPS, Pace University


References

References

  • Use of the slides of Dr. Clark Barrett (with permission)

  • ITR NSF Little Engines of Proof with Stanford Research Institute and Clarkson University, 2003

  • Automated Deduction with Constraints and Simplification in Equational Theories, Christelle Scharff, PhD, 1999

  • ICS (I can Solve or Integrated Canonizer and Solver), Harald Ruess, 2002

  • Abstract Congruence Closure, Leo Bachmair, Ashish Tiwari, and Laurent Vigneron, 2003

  • Unsound Theorem Proving, Christopher Lynch, 2004

  • Direct Combination of Completion and Congruence Closure, Christelle Scharff and Leo Bachmair, 2002 and 2004

  • On prototyping Deduction with Constraints and Simplification in a Rewriting Language, Christelle Scharff, 2004


Therac 25

  • Between 1985 and 1987, at least 6 accidental radiation overdoses were administered.

  • All the victims were injured, and 3 of them later died.

Therac-25


Ariane 5 rocket

Ariane 5 Rocket

  • On June 4, 1996, an unmanned Ariane 5 rocket launched by the European Space Agency exploded just 40 seconds after its lift-off.

  • Value of rocket and cargo: $500 million


Blackout

Blackout

  • In August, 2003, the largest blackout in our country’s history occurred.

  • Estimated cost to New York City alone: $1.1 billion.


What do these events have in common

What do these events have in common?

Caused by Software Bugs!

  • Each of the overdoses from the Therac-25 was the result of a bug in the controlling software.

  • The Ariane 5 explosion was the result of an unsafe floating point to integer conversion in the rocket’s software system.

  • A software bug caused an alarm system failure at FirstEnergy in Akron, Ohio. An early response to those alarms would likely have prevented the blackout.


More horror stories

More horror stories

  • Software Reliability: Principles and Practice, p 25, by G. J. Myers

    • Appolo 8 spacecraft erased part of the computer's memory.

    • Eighteen errors were detected during the 10-day flight of Apollo 14.

    • An error in a single FORTRAN statement resulted in the loss of the first American probe to Venus.


Why must we suffer so

Why must we suffer so?

  • Engineers are supposed to be good at building things.

  • When we build a building, we don’t expect it to crumble and have to be rebuilt twice a week.

  • Why don’t we have software engineers who build better software?

    • It’s not all Microsoft’s fault.

    • Software may be the most complex thing ever created.

    • Getting software right is a very, very difficult task.

    • Existing engineering techniques are inadequate.


Outline

Outline

  • What is Formal Verification?

  • Model Checking

  • Theorem Proving

  • Systems and Tools

  • Demonstrations (Eclipse, ICS, GACC)

  • Conclusions and Future Work


What is formal verification

What is Formal Verification?

  • “[Formal] software verification … has been the Holy Grail of computer science for many decades” – Bill Gates

  • Formal Verification (Proving) versus Validation (Testing)

  • Create a mathematical model of the system

    • An inaccurate model can introduce or mask bugs.

    • Fortunately, this can often be done automatically.

  • Specify formally what the properties of the system should be

  • Prove that the model has the desired properties

    • Much better than any testing method

    • Covers all possible cases

    • This is the hard part

  • There are a variety of tools and techniques


Proof techniques

Proof techniques

  • Model Checking

    • Typically relies on low-level Boolean logic

    • Proof is automatic

    • Does not always scale to large systems

  • Theorem Proving

    • Typically uses more expressive logic (higher order logic)

    • Proof is automatic in first order logic

    • Proof is manually directed (in higher order logic)

    • Unlimited scalability

  • Advanced techniques combine elements of both


Outline1

Outline

  • What is Formal Verification?

  • Model Checking

  • Theorem Proving

  • Systems and Tools

  • Demonstrations (ICS, GACC, Eclipse)

  • Conclusions and Future Work


Formal models

  • Consider this simple program:

  • The states of this program are all possible pairs of the variables x and y.

  • Fortunately, we can restrict our attention to thereachablestates.

int x, y;

x = 0;

y = 0;

while (x < 4) {

x++;

y = y + x;

}

Formal Models

  • Typically, a formal model is a graph in which each vertex represents a state of the program, and each edge represents a transition from one state to another.


Reachable states and checking properties

x ≥ 0

x ≥ 0

0,0

0,0

x ≥ y

int x, y;

x = 0;

y = 0;

while (x < 4) {

x++;

y = y + x;

}

1,0

1,0

1,1

2,1

2,3

3,3

3,6

Reachable States and Checking Properties

  • We can check a property by verifying that it is true in every reachable state. If the property is false, then there is a bug.

Initial State

Final State


Outline2

Outline

  • What is Formal Verification?

  • Model Checking

  • Theorem Proving

  • Systems and Tools

  • Demonstrations (Eclipse, ICS, GACC)

  • Conclusions and Future Work


Theorem proving

Theorem Proving

  • Theorem proving relies on human ingenuity and symbolic manipulations to prove that a program satisfies some properties.

  • Typically, proving a single property about a program will require proving many other properties as well.

  • Theorem Prover = Software based on Inference Rules + Strategy

    • Inference Rules: How to deduce new data? How to remove data?

    • Strategy: How to apply the inference rules?


Theorem proving1

Theorem Proving

  • What are the properties of theorem provers?

    • Soundness (Mandatory Property)

      • Do not prove that True = False

      • Prove only true formulae

    • Completeness(Optional Property)

      • If a formula is true, it can be proved.

    • Termination (Undecidable Property)

      • Does the application of the rules with the strategy terminate?


Theorem proving2

[x=y]

x = x + 1

y = y + 1

[x=y]

Theorem Proving

  • One approach is to annotate the program with theorems to be proved (e.g. pre-conditions, post-conditions, invariants, assertions), and then prove that each theorem really does hold.

    • Hoare Logic

  • Consider a simple program:

  • Suppose we wish to prove that, if x = y before the execution of this program, then at the end of its execution, x = y. We can annotate the beginning and the end of the program with these properties.


Theorem proving3

[x=y]

x = x + 1

[x = y + 1]

y = y + 1

[x=y]

Theorem Proving

  • What is the condition that will guarantee that x = y after executing y = y + 1 ?

  • To find out, propagate the assertions upward through the program. We imagine trying to prove x = y using primed variables to represent the values after the execution of y = y + 1, and unprimed variables for the values before it:

    (?) y’ = y + 1 x’ = x x’ = y’

    (?) x = y + 1


Theorem proving4

[x=y]

[x + 1 = y + 1]

x = x + 1

[x = y + 1]

y = y + 1

[x=y]

Theorem Proving

  • What is the condition that will guarantee that x = y + 1 after executing x = x + 1 ?

  • To find out, we imagine trying to prove

    x = y + 1 using primed variables to represent the values after the execution of x = x + 1, and unprimed variables for the values before it:

    (?) x’ = x + 1 y’ = y x’ = y’ + 1

    (?) x + 1 = y + 1


Theorem proving5

[x=y]

[x + 1 = y + 1]

x = x + 1

[x = y + 1]

y = y + 1

[x=y]

Theorem Proving

  • We prove that the program is correct by proving that:

    • x = y implies x + 1 = y + 1

    • using a theorem prover.


Outline3

Outline

  • What is Formal Verification?

  • Model Checking

  • Theorem Proving

  • Systems and Tools

  • Demonstrations (Eclipse, ICS, GACC)

  • Conclusions and Future Work


Systems and tools

Systems and Tools

  • Model checkers

    • SMV

    • SPIN

  • Theorem provers

    • Little Engines of proof

      • Automated domain-specific theorem provers (uninterpreted functions, linear arithmetic, arrays, bitvectors...)

      • ICS, CVC lite

      • GACC (Graph-based Abstract Congruence Closure)

    • Other Engines of proof

      • PVS, Isabelle, HOL

      • BaCCS (Basic Completion with Constraint and Simplification)


Microsoft slam

Microsoft SLAM

  • Clever combination of model checking and automated theorem proving

    • An abstract program is created in which all conditions are replaced with Boolean variables

    • Resulting Boolean program is model checked

    • If model checking fails, the potential error path is checked in the original program using an automated theorem prover

  • Successfully used to find bugs in Windows drivers.

    • …reducing the frequency of “Blue Screens of Death”!


Outline4

Outline

  • What is Formal Verification?

  • Model Checking

  • Theorem Proving

  • Systems and Tools

  • Demonstrations (Eclipse, ICS, GACC)

  • Conclusions and Future Work


Eclipse

Eclipse

  • Eclipse is an open source IDE developed by IBM

  • Eclipse is a “kind of universal tool platform – an open extensible IDE for anything and nothing in particular.”

  • Eclipse can be used to create applications as diverse as web sites, java programs, C++ programs...

  • An extension to Eclipse is called a plugin.

  • http://www.eclipse.org


Demo of ics i can solve or integrated canonizer and solver

Demo of ICS (I can solve or Integrated Canonizer and Solver)

  • Developed at SRI. http://www.ics.com

  • Decides formulae in a useful combination of theories (non interpreted formulae, bitvectors, reals, arrays…)

  • & means AND, | means OR, and ~ means NOT.

  • Example 1:

    • Ics> sat x = 2 & x = 3;

    • Answer: unsat

  • Example 2:

    • Ics> sat [x = 2 | x = 3] & x < 3;

    • Answer: sat, model [x = 2; 3 + - 1 * x > 0]


Demo of ics i can solve or integrated canonizer and solver1

Demo of ICS (I can solve or Integrated Canonizer and Solver)

  • If we want to prove: p  q, then we use refutational theorem proving.

    In fact, we assume that the hypotheses are true, and that the conclusion is false. Then, if p  q is true, we will derive a contradiction (unsat). Otherwise, the resulting model provides us with a counter-example.

  • Example 3:

    • If x = 2 and y = 4, then x + y = 6 ????

      ics> sat x = 2 & y = 4 & ~ x + y = 6;

      Answer: unsat

    • If x = 2 or x = 5, then x < 4 ????

      ics> sat [x = 2 | x = 5] & ~ x < 4;

      Answer: sat, model [-4 + x >= 0; x = 5]


Demo of gacc graph based abstract congruence closure

Demo of GACC (Graph-based Abstract Congruence Closure)

  • Congruence Closure is a method for deciding ground equalities.

    • Example: From f(a) = b and a = c, we can deduce f(c) =b.

  • The system GACC implements a new algorithm of congruence closure based on the use of graph with labeled vertices and edges.

  • GACC was implemented by Eugene Kipnis from Pace University under my supervision, and is available for online experimentation at: http://www.csis.pace.edu/~scharff/SOFTWARE


Outline5

Outline

  • What is Formal Verification?

  • Model Checking

  • Theorem Proving

  • Systems and Tools

  • Demonstrations (Eclipse, ICS, GACC)

  • Conclusions and Future Work


Conclusions

Conclusions

  • Formal Software Verification is starting to become practical

  • Still lots of work to be done

  • How can it make you a better programmer?

    • Document your code with the properties and invariants that you think should be true

    • When you modify code, convince yourself that you are not breaking any invariants

  • Hopefully, someday software will be as safe and reliable as the other objects built by engineers!


Environment

Environment

  • CAFME Center create by Dr. Skevoulis

  • CS 851: Software Validation and Verification

    • New course offered in the Fall 2004

    • Part of the degree in Software Design and Development

    • Validation = Testing

    • Verification = Proving

  • ITR NSF Grant “Little Engines of Proof” with Stanford Research Institute, and Clarkson University


Future work 1 eclipse plugin

Future Work 1: Eclipse Plugin

  • A java program is annotated with assertions to be proved

  • Assertions are written in the ICS syntax

  • Some assertions are computed. Assertions are propagated up through the program line by line

  • ICS will be called from the java program, and the assertions will be checked

  • Parts of the program will be proved correct, or bugs will be discovered


Future work 2 strategies

Future Work 2: Strategies

  • Rewriting languages (e.g. ELAN, MAUDE, OBJ, ASF+SDF) are very expressive, and very convenient to prototype theorem provers

  • Rewriting languages share many features with functional programming languages (no side-effect, use of recursion, pattern matching…), but they are based on rewriting instead of lambda calculus

  • Can we develop a Java API that would be used to prototype theorem provers? Can we define the strategy on the rules in an XML document?


Future work 3 correctness of abstract congruence closure

Future Work 3: Correctness of Abstract Congruence Closure

  • Abstract congruence closure is a framework that defines a set of inference rules and different strategies for congruence closure. These strategies include strategies that simulate Nelson-Oppen, Shostak and Downey-Sethi-Tarjan algorithms

  • The rules and the strategies can be implemented easily using a rewriting language such as ELAN

  • This implementation would be a formal framework to prove the correctness of the abstract congruence closure algorithm, and, besides, would permit intensive benchmarking on criteria to be defined


Future work 4 unsound theorem proving

Future Work 4: Unsound Theorem Proving

  • Many problems given to a theorem prover will be false conjectures. It is important to be able to find quickly bugs in programs.

  • An inference system for unsound theorem proving (or theorem (dis)proving) has been defined by Christopher Lynch.

  • The theorem (dis)prover can be prototyped in a rewriting language such as ELAN.

  • This framework could be used to show the correctness of the procedure, and benchmark the method on criteria to be defined


  • Login