260 likes | 401 Views
Trace Verification for Parallel Systems Vijay K. Garg Department of Electrical and Computer Engineering The University of Texas at Austin Austin, TX 78712 email: garg@ece.utexas.edu. Talk Outline. Motivation and Overview Instrumentation Clock : Tracking Dependency Property Checking
E N D
Trace Verification for Parallel SystemsVijay K. GargDepartment of Electrical and Computer EngineeringThe University of Texas at AustinAustin, TX 78712email: garg@ece.utexas.edu
Talk Outline • Motivation and Overview • Instrumentation • Clock : Tracking Dependency • Property Checking • Sensor : Detecting Global Properties • Slicer : Computation Slicing
Motivation: Reliable System • Concurrent systems are prone to errors. • Concurrency, nondeterminism, process and channel failures Techniques to ensure correctness • Modeling: Model Checking and Formal Verification • Bug Hunting: Simulation, Debugging and Verification • Fault-Tolerance
Observe Program Monitor Slicer Predicate Paradise Environment Control
Talk Outline • Motivation and Overview • Instrumentation • Clock : Tracking Dependency • Property Checking • Sensor : Detecting Global Properties • Slicer : Computation Slicing
Trace Model: Total Order vs Partial Order • Total order: interleaving of events in a trace • Partial order: Lamport’s happened-before model Successful Trace ¬CS1 ¬CS2 CS1 CS2 Specification: CS1 ΛCS2 f1 f2 e2 e1 Partial Order Trace Faulty Trace ¬ CS1 CS1 P1 e1 e2 ¬CS1 CS2 CS1 ¬CS2 ¬CS2 CS2 P2 f1 e2 f2 e1 f2 f1
Tracking Dependency computation: a set of events ordered by “happened before” relation • Problem: Timestamp events to answer • e happened before f ? • e concurrent with f ?
Clocks in a Distributed System Result:s happened before t i the vector at s is less than the vector at t. Vector Clocks [Fidge 89, Mattern 89] (1,0,0) (2,1,0) (3,1,0) P1 (0,1,0) (0,2,0) P2 (0,0,1) (0,0,2) (2,1,3) P3
a b c d e f g h Dynamic Chain Clocks • Problem with vector clocks: scalability, dynamic process structure • Idea: Computing the “chains” in an online fashion [Aggarwal and Garg PODC 05] for relevant events a P1 P2 P3 P4 b c e d h f g The relevant subcomputation A computation with 4 processes
Experimental Results Simulation of a computation with 1% relevant events Measured • number of components vs number of threads • total time overhead vs number of threads
Talk Outline • Motivation and Overview • Instrumentation • Clock : Tracking Dependency • Property Checking • Sensor : Detecting Global Properties • Slicer : Computation Slicing
G1 G2 P1 P2 Critical section Global Property Detection Predicate: A global condition expressed using variables on processes • e.g., more than one process is in critical section, there is no token in the system Problem: find a global state that satisfies the given predicate
The Main Difficulty in Partial Order Algorithm for general predicate [Cooper and Marzullo 91] {e2, e1, f2, f1, ┴ e1 e2 P1 {e2, e1, f1, ┴} {e1, f2, f1, ┴} {e2, e1, ┴} ┴ {e1, f1, ┴} T P2 {f1, ┴} {e1, ┴} f1 f2 {┴} Too many global states : A computation may contain as many as O(kn) global states • k: maximum number of events on a process • n: number of processes
Efficient Predicate Detection for Special Cases • stable predicate:[Chandy and Lamport 85] • once the predicate becomes true, it stays true e.g., deadlock • unstable predicate: • observer independent predicate[Charron-Bost et al 95] occurs in one interleaving occurs in all interleavings e.g., any disjunction of local predicate • linear predicate[Chase and Garg 95] e.g., conjunctive predicates such as there is no leader in the system • relational predicate: x1 + x2 +…+ xn ≥ k [Chase and Garg 95] e.g., violation of k-mutual exclusion
Algorithms for Conjunctive Predicates Centralized Algorithm [Garg and Waldecker 92] Each non-checker process maintains its local vector and sends to the checker process the chain clock whenever • local predicate is true • at most once in each message interval. Time complexity: Checker requires at most O(n2m) comparisons. • token based algorithm [Garg and Chase 95] • completely distributed algorithm [Garg and Chase 95] • keeping queues shorter [Chiou and Korfhage 95] • avoiding control messages [Hurfin, Mizuno, Raynal, Singhal 96]
Other Special Classes of Predicates • Relational Predicates • Let xi: number of token at Pi • Σxi < k: loss of tokens • Algorithms: max-flow techniques [Groselj 93, Chase and Garg 95, Wu and Chen 98] • Dilworth's partition [Tomlinson and Garg 96]
Talk Outline • Motivation and Overview • Instrumentation • Clock : Tracking Dependency • Property Checking • Sensor : Detecting Global Properties • Slicer : Computation Slicing
The Main Idea of Computation Slicing state explosion Partial order trace keep all red global states slicing slice
How does Computation Slicing Help? Partial order trace check b1Λb2 satisfy b1 retain all global states satisfying b1 slicing for b1 slice check b2
1 4 4 2 2 0 1 2 -1 1 0 3 d a b c e f x h w v u g x P1 {a,e,f,u,v} {b} y P2 {w} {g} z P3 Example • Detect predicate (x*y + z < 5) Λ (x ≥1) Λ (z ≤3) Slice with respect to (x ≥1) Λ (z ≤3) Computation
Computation Slice computation slice: a sub-computation such that: [Mittal and Garg 01] • it contains allglobal states of the computation satisfying the given predicate, and • it contains the least number of global states
POTA Architecture [Sen Dissertation 04] Predicate (Specification) Program Analyzer yes/ witness Slice Instrumentor Slicer Predicate Detector no/ counter example Trace Slice Instrumented Program Promela yes Execute Program Trace Translator Execute SPIN no/ counter example Specification
Results Efficient polynomial-time algorithms for computing the slice for: • linear predicates: [Garg and Mittal 01] • time-complexity: O(n2m) • general predicate: • Theorem: Given a computation, if a predicate b can be detected efficiently then the slice for b can also be computed efficiently. [Mittal,Sen and Garg 03] • combining slices: Boolean operators • temporal logic operators: EF, AG, EG • approximate slice: For arbitrary boolean expression • n: number of processes • m: number of events
Experiments: Dining Philosophers Trace Verification • POTA: Partial Order Trace Analyzer (based on slicing) [Sen and Garg 03] • SPIN: A widely used model checking tool [Holzmann 97] • SPIN: 250 seconds for n = 6, runs out of memory for n > 6. • POTA: can handle n= 200. Used 400 seconds. Predicate: Two neighboring dining philosophers do not eat concurrently
Conclusions • Bug-hunting in concurrent systems • Total order vs. Partial Order • Abstraction like slicing to combat state space explosion problem
Questions • ?