Model Checking and Testing combined

1. Model Checking and Testingcombined Doron Peled, University of Warwick

2. Want to verify hardware and code. Want to perform verification automatically. Manual methods are time consuming, difficult. Restricting to finite state systems. Willing to give upexhaustiveness. Checking a (mathematical) model of a system, not the system itself. Want to obtain counterexamples. Why “model checking”?

3. A transition system • A (finite) set of variables V. • A set of states . • A (finite) set of transitions T, each transition et has • an enabling condition e and • a transformation t. • An initial condition p. • Denote that s’ is a successor of s by R(s,s’).

4. The interleaving model • An execution is a finite or infinite sequence of states s0, s1, s2, … • The initial state satisfies the initial condition, i.e., p(s0). • Moving from one state si to si+1 is by executing a transition et: • e(si), i.e., si satisfies e. • si+1 is obtained by applying t to si.

5. L0:While True do NC0:wait(Turn=0); CR0:Turn=1 endwhile || L1:While True do NC1:wait(Turn=1); CR1:Turn=0 endwhile T0:PC0=L0PC0=NC0 T1:PC0=NC0/\Turn=0 PC0:=CR0 T2:PC0=CR0 (PC0,Turn):=(L0,1) T3:PC1==L1PC1=NC1 T4:PC1=NC1/\Turn=1 PC1:=CR1 T5:PC1=CR1 (PC1,Turn):=(L1,0) Initially: PC0=L0/\PC1=L1

6. Turn=0 L0,L1 Turn=1 L0,L1 Turn=0 L0,NC1 Turn=0 NC0,L1 Turn=1 L0,NC1 Turn=1 NC0,L1 Turn=0 NC0,NC1 Turn=0 CR0,L1 Turn=1 L0,CR1 Turn=1 NC0,NC1 Turn=0 CR0,NC1 Turn=1 NC0,CR1 The state space

7. Turn=0 L0,L1 Turn=1 L0,L1 Turn=0 L0,NC1 Turn=0 NC0,L1 Turn=1 L0,NC1 Turn=1 NC0,L1 Turn=0 NC0,NC1 Turn=0 CR0,L1 Turn=1 L0,CR1 Turn=1 NC0,NC1 Turn=0 CR0,NC1 Turn=1 NC0,CR1 Invariant: (PC0=CR0/\PC1=CR1)

8. How can we check the model? • The model is a graph. • The specification should refer the the graph representation. • Apply graph theory algorithms.

9. What properties can we check without using temporal specification? • Invariants: a property that needs to hold in each state. • Deadlock detection: can we reach a state where the program is blocked? • Dead code: does the program have parts that are never executed.

10. How to perform the check? • Apply a search strategy (Depth first search, Breadth first search). • Check states/transitions during the search. • If property does not hold, report counterexample! • DFS – on-the-fly verification. • BFS – finds the shortest counterexample.

11. If it is so good, why learn deductive verification methods? • Model checking works for finite state systems. Would not work with • Unconstrained integers. • Unbounded message queues. • General data structures: • queues • trees • stacks • parametric algorithms and systems. • Some new algorithms for infinite systems.

12. The state space explosion • Need to represent the state space of a program in the computer memory. • Each state can be as big as the entire memory! • Many states: • Each integer variable has 2^32 possibilities. Two such variables have 2^64 possibilities. • In concurrent protocols, the number of states usually grows exponentially with the number of processes.

13. If it is so constrained, is it of any use? • Many protocols are finite state. • Many programs or procedures are finite state in nature. Can use abstraction techniques. • Sometimes it is possible to decompose a program, and prove part of it by model checking and part by theorem proving. • Many techniques to reduce the state space explosion (BDDs, Partial Order Reduction).

14. Program DFS For each s such that q(s) dfs(s) end DFS Procedure dfs(s) for each s’ such that R(s,s’) do If new(s’) then dfs(s’) end dfs. Depth First Search

15. Start from an initial state Hash table: q1 q1 q2 q3 Stack: q1 q4 q5

16. Continue with a successor Hash table: q1 q1 q2 q2 q3 Stack: q1 q2 q4 q5

17. One successor of q2. Hash table: q1 q1 q2 q4 q2 q3 Stack: q1 q2 q4 q4 q5

18. Backtrack to q2(no new successors for q4). Hash table: q1 q1 q2 q4 q2 q3 Stack: q1 q2 q4 q5

19. Backtracked to q1 Hash table: q1 q1 q2 q4 q2 q3 Stack: q1 q4 q5

20. Second successor to q1q4 has been already visited. Hash table: q1 q1 q2 q4 q3 q2 q3 Stack: q1 q3 q4 q5

21. Backtrack again to q1. Hash table: q1 q1 q2 q4 q3 q2 q3 Stack: q1 q4 q5

22. How can we check properties with DFS? • Invariants: check that all reachable statessatisfy the invariant property. If not, showa path from an initial state to a bad state. • Deadlocks: check whether a state where noprocess can continue is reached. • Dead code: as we progress with the DFS, mark all the transitions that are executed at least once.

23. Want to do more! • Want to check more properties. • Want to have a unique algorithm to deal with all kinds of properties. • This is done by writing specification is temporal logics. • Temporal logic specification can be translated into graphs (finite automata).

24. Temporal Logic • First order logic or propositional assertions describe a state. • Modalities: • <>p means p will happen eventually. • []p means p will happen always. p p p p p p p p

25. More temporal logic • p U q– p has to hold until q holds. p p p p q • []<>p --- its always the case that there is a later p, i.e., p happens infinitely often. • <>[]p --- At some point p will hold forever, i.e., p is stable. • <>p/\<>q both p and q would happen eventually.Note, this is not the same as <>(p/\q).

26. Turn=0 L0,L1 Turn=1 L0,L1 Turn=0 L0,NC1 Turn=0 NC0,L1 Turn=1 L0,NC1 Turn=1 NC0,L1 Turn=0 NC0,NC1 Turn=0 CR0,L1 Turn=1 L0,CR1 Turn=1 NC0,NC1 Turn=0 CR0,NC1 Turn=1 NC0,CR1 [](Turn=0<>Turn=1)

27. Correctness condition • We want to find a correctness condition for a model to satisfy a specification. • Language of a model: L(Model) • Language of a specification: L(Spec). • We need: L(Model)  L(Spec).

28. Correctness Sequencessatisfying Spec Program executions All sequences

29. Counterexamples Sequencessatisfying Spec Program executions All sequences Incorrectness • Counterexamples are sometimes more interesting and useful than finding that the program is “correct” due to: • Underspecification • Modeling errors • Algorithm and tool limitation

30. How to check correctness? • Show that L(Model)  L(Spec). • Equivalently: ______Show that L(Model)  L(Spec) = Ø. • Also: can obtain L(Spec) by translating from LTL!

31. What do we need to know? • How to intersect two automata? • How to complement an automaton?Well, not really, if the specification is given in LTL, we can negate the specification and then translate. • How to translate from LTL to an automaton?

32. Büchi automata (w-automata) S - finite set of states. S0S - initial states. S - finite alphabet. d S  SS - transition relation. F S - accepting states. Accepting run: passes a state in F infinitely often. System automata: F=S.

33. a a a, a Example: check a <>a

34. Example: check <>a a <>a a a a

35. Example: check <>a a, a <>a a a Use automatic translation algorithms, e.g., [Gerth,Peled,Vardi,Wolper 95]

36. init Turn=0 L0,L1 Turn=1 L0,L1 Turn=0 L0,L1 Turn=1 L0,L1 • Add an initial node. • Propositions are attached to incoming nodes. • All nodes are accepting. Technicality…

37. a s1 s2 c b s3 System All states are accepting! = no fairness conditions

38. a s1 s2 a s1,q1 s2,q1 c b b a s3 s1,q2 s3,q2 c Every element in the product is a counter example for the checked property. a q1 a a a q2 Acceptance isdetermined byautomaton P.

39. a s1,q1 s2,q1 b a s1,q2 s3,q2 c How to check for (non)emptiness?

40. Nonemptiness... Need to check if there exists an accepting run, i.e., infinite sequence that passes through an accepting state infinitely often.

41. Finding accepting runs If there is an accepting run, then at least one accepting state repeats on it forever. This state must appear on a cycle. So, find a reachable accepting state on a cycle.

42. Equivalently... A strongly connected component: a maximal set of nodes where each node is reachable by a path from each other node. Find a reachable strongly connected component with an accepting node.

43. How to complement? • Complementation is hard! • Can ask for the negated property (the sequences that should never occur). • Can translate from LTL formula  to automaton A, and complement A. But: can translate ¬ into an automaton directly!

44. Model Checking under Fairness Express the fairness as a property φ.To prove a property ψ under fairness,model check φψ. Counter example Fair (φ) Bad (¬ψ) Program

45. Model Checking under Fairness Specialize model checking. For weak process fairness: search for a reachable strongly connected component, where for each process P either • it contains on occurrence of a transition from P, or • it contains a state where P is disabled.

46. Conformance Testing • Unknown deterministic finite state system B. • Known: n states and alphabet . • An abstract model C of B. C satisfies all the properties we want from B. C has m states. • Check conformance of B and C. • Another version: only a bound n on the number of states l is known. 

47. Given Finite state system B. Transition relation of B known. Property represent by automaton P. Check if L(B)  L(P)=. Graph theory or BDD techniques. Complexity: polynomial. Unknown Finite state system B. Alphabet and number of states of B or upper bound known. Specification given as an abstract system C. Check if B C. Complexity: polynomial if number states known. Exponential otherwise. Model Checking / Testing

48. Property represent by automaton P. Check if L(B)  L(P)=. Graph theory techniques. Unknown Finite state system B. Alphabet and upper bound on number of states of B known. Complexity: exponential. Black box checking  

49. Combination lock automaton Accepts only words with a specific suffix (cdab in the example). c d a b s1 s2 s3 s4 s5 Any other input

50.  b b a a b a b a  a b Cannot distinguish if reduced or not. Conformance testing