1 / 47

From Symptom to Cause: Localizing Errors in Counterexample Traces

From Symptom to Cause: Localizing Errors in Counterexample Traces. Mayur Naik, Purdue University Thomas Ball, Microsoft Research Sriram K. Rajamani, Microsoft Research. Model Checking. + Fully automatic: Does not require the user to provide annotations.

sorley
Download Presentation

From Symptom to Cause: Localizing Errors in Counterexample Traces

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. From Symptom to Cause: Localizing Errors in Counterexample Traces Mayur Naik, Purdue University Thomas Ball, Microsoft Research Sriram K. Rajamani, Microsoft Research

  2. Model Checking + Fully automatic: Does not require the user to provide annotations. + Transparent: Produces a source-level error trace (counterexample). −An error trace represents a symptom of the error as opposed to its cause. − State-of-the-art model checkers report only one error trace.

  3. The Problem • How do we localize the cause of the errorin an error trace? • How do we produce multiple error traces having distinct causes? Note: Problem is relevant to other error-detection techniques as well.

  4. What is a “Cause”? • We define a “cause” to be those parts of an error trace not contained in any correct trace. • The program fragments containing the cause are rendered unreachable. • The model checker is invoked again to produce additional error traces.

  5. Example main() { AcquireLock(); if (...) ReleaseLock(); else { ... } AcquireLock(); if (...) ReleaseLock(); else { ... } return; }

  6. Error #1: Lock acquired in succession main() { AcquireLock(); if (...) ReleaseLock(); else { ... } AcquireLock(); if (...) ReleaseLock(); else { ... } return; }

  7. Correct Trace Computation main() { AcquireLock(); if (...) ReleaseLock(); else { ... } AcquireLock(); if (...) ReleaseLock(); else { ... } return; }

  8. Error Cause Localization main() { AcquireLock(); if (...) ReleaseLock(); else { ... } AcquireLock(); if (...) ReleaseLock(); else { ... } return; }

  9. Unreachable from entry of main in future runs of the model checker Inserthalt Error Recovery main() { AcquireLock(); if (...) ReleaseLock(); else { ... } AcquireLock(); if (...) ReleaseLock(); else { ... } return; }

  10. Error #2: Lock held on exit main() { AcquireLock(); if (...) ReleaseLock(); else { halt; ... } AcquireLock(); if (...) ReleaseLock(); else { ... } return; }

  11. Correct Trace Computation main() { AcquireLock(); if (...) ReleaseLock(); else { halt; ... } AcquireLock(); if (...) ReleaseLock(); else { ... } return; }

  12. Error Cause Localization main() { AcquireLock(); if (...) ReleaseLock(); else { halt; ... } AcquireLock(); if (...) ReleaseLock(); else { ... } return; }

  13. Unreachable from entry of main in future runs of the model checker Inserthalt Error Recovery main() { AcquireLock(); if (...) ReleaseLock(); else { halt; ... } AcquireLock(); if (...) ReleaseLock(); else { ... } return; }

  14. Final (Error-Free) Program main() { AcquireLock(); if (...) ReleaseLock(); else { halt; ... } AcquireLock(); if (...) ReleaseLock(); else { halt; ... } return; }

  15. Our Results • A technique that exploits correct traces for error cause localization. • Efficient algorithm for computing correct traces. • Experimental results in the context of the SLAM toolkit.

  16. Transitions and Edges 1 AcquireLock(); 2 if (...) 3 ReleaseLock(); else { 4 ... } 5 AcquireLock();

  17. Transitions and Edges 1 AcquireLock(); 2 if (...) 3 ReleaseLock(); else { 4 ... } 5 AcquireLock(); <(3, L), (5, U)>

  18. Transitions and Edges 1 AcquireLock(); 2 if (...) 3 ReleaseLock(); else { 4 ... } 5 AcquireLock(); project(<(3, L), (5, U)>) = (3, 5)

  19. High-Level Algorithm while truedo switch ModelCheck(G, ve) of //ve is of the form assert(e) case FAILURE(T): let C = GetCorrectTransitions(G, ve) and K = project(T) \ project(C) in if K = Ø then break for each (vi, vj) in K do insert a halt statement between vi and vj case SUCCESS: break

  20. Computing Correct Transitions 1 AcquireLock(); 2 if (...) 3 ReleaseLock(); else { 4 ... } 5 AcquireLock(); (1,U) (2,L) (3,L) (4,L) ve≡assert(s==U) (5,U) (5,L) Reachable state-space computed by model checker <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> Transitions in Error Trace (T)

  21. Computing Correct Transitions 1 AcquireLock(); 2 if (...) 3 ReleaseLock(); else { 4 ... } 5 AcquireLock(); (1,U) (2,L) (3,L) (4,L) ve≡assert(s==U) (5,U) (5,L) <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> Transitions in Error Trace (T) Correct Transitions (C)

  22. Computing Correct Transitions 1 AcquireLock(); 2 if (...) 3 ReleaseLock(); else { 4 ... } 5 AcquireLock(); (1,U) (2,L) (3,L) (4,L) ve≡assert(s==U) (5,U) (5,L) <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(3, L), (5, U)> Transitions in Error Trace (T) Correct Transitions (C)

  23. Computing Correct Transitions 1 AcquireLock(); 2 if (...) 3 ReleaseLock(); else { 4 ... } 5 AcquireLock(); (1,U) (2,L) (3,L) (4,L) ve≡assert(s==U) (5,U) (5,L) <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(2, L), (3, L)> <(3, L), (5, U)> Transitions in Error Trace (T) Correct Transitions (C)

  24. Computing Correct Transitions 1 AcquireLock(); 2 if (...) 3 ReleaseLock(); else { 4 ... } 5 AcquireLock(); (1,U) (2,L) (3,L) (4,L) ve≡assert(s==U) (5,U) (5,L) <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(1, U), (2, L)> <(2, L), (3, L)> <(3, L), (5, U)> Transitions in Error Trace (T) Correct Transitions (C)

  25. (1,U) (1,U) (2,L) (2,L) (3,L) (4,L) (5,U) (5,L) (6,U) (6,L) Example 1: An omission error 1 AcquireLock(); 2 if (...) 3 ReleaseLock(); else { 4 ... } 5 ... 6 AcquireLock(); ... <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(5, L), (6, L)> <(1, U), (2, L)> <(2, L), (3, L)> <(3, L), (5, U)> <(5, U), (6, U)> Transitions in Error Trace (T) Correct Transitions (C)

  26. (1,U) (1,U) (2,L) (2,L) (3,L) (4,L) (5,U) (5,L) (6,U) (6,L) Error Cause Localization 1 AcquireLock(); 2 if (...) 3 ReleaseLock(); else { 4 ... } 5 ... 6 AcquireLock(); ... <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(5, L), (6, L)> <(1, U), (2, L)> <(2, L), (3, L)> <(3, L), (5, U)> <(5, U), (6, U)> K = project (T) \ project (C) Transitions in Error Trace (T) Correct Transitions (C)

  27. (1,U) (1,U) (2,L) (2,L) (3,L) (4,L) (5,U) (5,L) (6,U) (6,L) Error Cause Localization 1 AcquireLock(); 2 if (...) 3 ReleaseLock(); else { 4 ... } 5 ... 6 AcquireLock(); ... <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(5, L), (6, L)> <(1, U), (2, L)> <(2, L), (3, L)> <(3, L), (5, U)> <(5, U), (6, U)> K = project (T) \ project (C) Transitions in Error Trace (T) Correct Transitions (C)

  28. (1,U) (1,U) (2,L) (2,L) (3,L) (4,L) (5,U) (5,L) (6,U) (6,L) Error Cause Localization 1 AcquireLock(); 2 if (...) 3 ReleaseLock(); else { 4 ... } 5 ... 6 AcquireLock(); ... <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(5, L), (6, L)> <(1, U), (2, L)> <(2, L), (3, L)> <(3, L), (5, U)> <(5, U), (6, U)> K = project (T) \ project (C) Transitions in Error Trace (T) Correct Transitions (C)

  29. (1,U) (2,L) (3,L) (5,U) (6,U) Error Cause Localization 1 AcquireLock(); 2 if (...) 3 ReleaseLock(); else { 4 ... } 5 ... 6 AcquireLock(); ... (1,U) (2,L) (4,L) (5,L) (6,L) <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(5, L), (6, L)> <(1, U), (2, L)> <(2, L), (3, L)> <(3, L), (5, U)> <(5, U), (6, U)> K = project (T) \ project (C) = { (2, 4), (4, 5) } Transitions in Error Trace (T) Correct Transitions (C)

  30. Experimental Results

  31. Experimental Results

  32. Experimental Results

  33. (1,(S,S)) (2,(S,S)) (4,(S,S)) (5,(S,F)) (6,(S,F)) Error Trace Example 2: A variable-value error main() { 1 int status = S; 2 if (*) 3 status = foo(); else { 4 foo(); 5 status = S; } 6 assert(status==x); } (1,(S,S)) (2,(S,S)) (3,(S,S)) (6,(S,S)) (6,(F,F)) enum { S, F } x = S; int foo() { if (*) x = S; else x = F; return x; } Correct Traces Program state is of the form (status, x)

  34. (1,(S,S)) (1,(S,S)) (2,(S,S)) (2,(S,S)) (3,(S,S)) (4,(S,S)) (4,(S,S)) (5,(S,S)) (5,(S,F)) (6,(S,S)) (6,(S,F)) (6,(F,F)) Correct Traces Error Trace Example 2: A variable-value error main() { 1 int status = S; 2 if (*) 3 status = foo(); else { 4 foo(); 5 status = S; } 6 assert(status==x); } enum { S, F } x = S; int foo() { if (*) x = S; else x = F; return x; } Program state is of the form (status, x)

  35. (1,(S,S)) (1,(S,S)) (2,(S,S)) (2,(S,S)) (3,(S,S)) (4,(S,S)) (4,(S,S)) (5,(S,S)) (5,(S,F)) (6,(S,S)) (6,(S,F)) (6,(F,F)) Correct Traces Error Trace Error Cause Localization main() { 1 int status = S; 2 if (*) 3 status = foo(); else { 4 foo(); 5 status = S; } 6 assert(status==x); } enum { S, F } x = S; int foo() { if (*) x = S; else x = F; return x; } K = project (T) \ project (C) = Ø

  36. High-Level Algorithm while truedo switch ModelCheck(G, ve) of //ve is of the form assert(e) case FAILURE(T): let C = GetCorrectTransitions(G, ve) and K = project(T) \ project(C) in if K = Ø then break for each (vi, vj) in K do insert a halt statement between vi and vj case SUCCESS: break

  37. High-Level Algorithm while truedo switch ModelCheck(G, ve) of //ve is of the form assert(e) case FAILURE(T): let C = GetCorrectTransitions(G, ve) and K = project(T \ C) in if K = Ø then break for each (vi, vj) in K do insert a halt statement between vi and vj case SUCCESS: break

  38. (1,(S,S)) (1,(S,S)) (2,(S,S)) (2,(S,S)) (3,(S,S)) (4,(S,S)) (4,(S,S)) (5,(S,S)) (5,(S,F)) (6,(S,S)) (6,(S,F)) (6,(F,F)) Correct Traces Error Trace Example 2: A variable-value error main() { 1 int status = S; 2 if (*) 3 status = foo(); else { 4 foo(); 5 status = S; } 6 assert(status==x); } enum { S, F } x = S; int foo() { if (*) x = S; else x = F; return x; } Program state is of the form (status, x)

  39. (1,(S,S)) (2,(S,S)) (3,(S,S)) (4,(S,S)) (5,(S,S)) (6,(S,S)) (6,(F,F)) Correct Traces Error Cause Localization main() { 1 int status = S; 2 if (*) 3 status = foo(); else { 4 foo(); 5 status = S; } 6 assert(status==x); } (1,(S,S)) (2,(S,S)) (4,(S,S)) (5,(S,F)) (6,(S,F)) enum { S, F } x = S; int foo() { if (*) x = S; else x = F; return x; } Error Trace K = project (T \ C) = { (4, 5), (5, 6) }

  40. (1,U) (1,U) (2,L) (2,L) (3,L) (4,L) (5,U) (5,L) (6,U) (6,L) Example 1: An omission error 1 AcquireLock(); 2 if (...) 3 ReleaseLock(); else { 4 ... } 5 ... 6 AcquireLock(); ... <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(5, L), (6, L)> <(1, U), (2, L)> <(2, L), (3, L)> <(3, L), (5, U)> <(5, U), (6, U)> Transitions in Error Trace (T) Correct Transitions (C)

  41. (1,U) (1,U) (2,L) (2,L) (3,L) (4,L) (5,U) (5,L) (6,U) (6,L) Error Cause Localization 1 AcquireLock(); 2 if (...) 3 ReleaseLock(); else { 4 ... } 5 ... 6 AcquireLock(); ... <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(5, L), (6, L)> <(1, U), (2, L)> <(2, L), (3, L)> <(3, L), (5, U)> <(5, U), (6, U)> K = project (T \ C) Transitions in Error Trace (T) Correct Transitions (C)

  42. (1,U) (1,U) (2,L) (2,L) (3,L) (4,L) (5,U) (5,L) (6,U) (6,L) Error Cause Localization 1 AcquireLock(); 2 if (...) 3 ReleaseLock(); else { 4 ... } 5 ... 6 AcquireLock(); ... <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(5, L), (6, L)> <(1, U), (2, L)> <(2, L), (3, L)> <(3, L), (5, U)> <(5, U), (6, U)> K = project (T \ C) Transitions in Error Trace (T) Correct Transitions (C)

  43. (1,U) (1,U) (2,L) (2,L) (3,L) (4,L) (5,U) (5,L) (6,U) (6,L) Error Cause Localization 1 AcquireLock(); 2 if (...) 3 ReleaseLock(); else { 4 ... } 5 ... 6 AcquireLock(); ... <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(5, L), (6, L)> <(1, U), (2, L)> <(2, L), (3, L)> <(3, L), (5, U)> <(5, U), (6, U)> K = project (T \ C) Transitions in Error Trace (T) Correct Transitions (C)

  44. (1,U) (1,U) (2,L) (2,L) (3,L) (4,L) (5,U) (5,L) (6,U) (6,L) Error Cause Localization 1 AcquireLock(); 2 if (...) 3 ReleaseLock(); else { 4 ... } 5 ... 6 AcquireLock(); ... <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(5, L), (6, L)> <(1, U), (2, L)> <(2, L), (3, L)> <(3, L), (5, U)> <(5, U), (6, U)> K = project (T \ C) = { (2, 4), (4, 5), (5, 6) } Transitions in Error Trace (T) Correct Transitions (C)

  45. Limitations • Control-based approach fails to localize the cause when every edge in the error trace is contained in some correct trace. • Transition-based approach localizes the cause to a suffix of the error trace. • Model Imprecision: Infeasible paths can misguide error cause localization using either approach.

  46. Related Work Multiple Counterexamples • Verisim [Bhargavan et al., TSE ’02] Error Cause Localization • Explaining counterexamples [Jin et al., TACAS ’02] • Explaining type errors [Wand, POPL ’86; Johnson & Walz, POPL ’86; Beaven & Stansifer, LOPLAS ’93; Duggan & Bent, SCP ’96; Chitil, ICFP ’01; Tip & Dinesh, TOSEM ’01] • Program Slicing [Weiser, TSE ’84] • Algorithmic Debugging [Shapiro, Ph.D. thesis ’82] • Delta Debugging [Zeller, FSE ’99] Anomaly Detection • Static: Meta-Level Compilation [Hallem et al., PLDI ’02] • Dynamic: Daikon [Ernst et al., TSE ’01], DIDUCE [Hangal & Lam, ICSE ’02]

  47. Conclusions • We have presented a technique for localizing the causes of errors in counterexample traces. • A combination of the control-based and transition-based approaches appears promising. • Our technique is quite general and should be applicable to error detection tools based on data-flow analysis as well.

More Related