1 / 45

What Went Wrong?

Learn how to use counterexamples to identify the true cause of errors in Java code. Discover techniques for generating additional traces and analyzing positive and negative cases to pinpoint the root problem.

Download Presentation

What Went Wrong?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. What Went Wrong? Alex Groce Carnegie Mellon University Willem Visser NASA Ames Research Center

  2. Java Code Bytecode void add(Object o) { buffer[head] = o; head = (head+1)%size; } Object take() { … tail=(tail+1)%size; return buffer[tail]; } JAVAC JVM 0: iconst_0 1: istore_2 2: goto #39 5: getstatic 8: aload_0 9: iload_2 10: aaload Model Checker Java PathFinder Special JVM

  3. Counterexamples • When the model checker finds a bug, it reports a counterexample. • However, even an exact trace of the program failing is not what we really want for most bugs.

  4. Counterexamples • How can we get more information from a counterexample? • Perhaps if we had some successful runs to compare to, or other failing runs in which to look for common elements… • A common way of debugging by hand • Except we can automate it using the model checker

  5. Counterexamples The Error

  6. Counterexamples Real cause “The Error”

  7. Counterexamples • Idea: generate other traces that can give you more information about what is going wrong. • How? Real cause “The Error”

  8. Positives and Negatives Real cause assert (x < 5); (and x = 8) “The Error”

  9. Positives and Negatives Same location, same error: NEGATIVE (-) Real cause Real cause assert (x < 5); (and x = 8) assert (x < 5); (and x = 10) “The Error” An Error

  10. Positives and Negatives Same control flow location, no error condition: POSITIVE (+) Same location, same error: NEGATIVE (-) Real cause Real cause assert (x < 5); (and x = 3) assert (x < 5); (and x = 8) assert (x < 5); (and x = 10) No Error “The Error” An Error

  11. The Basic Search Real cause “The Error”

  12. The Basic Search Real cause

  13. The Basic Search Real cause

  14. The Basic Search Real cause

  15. The Basic Search Real cause

  16. The Basic Search Real cause

  17. The Basic Search Real cause

  18. The Basic Search

  19. The Basic Search

  20. The Basic Search

  21. The Basic Search

  22. The Search in Detail Real cause Depth limit “The Error” Positives Negatives

  23. The Search Real cause Already visited Depth limit “The Error” Positives Negatives

  24. The Search Real cause Depth limit “The Error” Positives Negatives

  25. The Search Real cause Depth limit “The Error” “push-through” Positives Negatives

  26. The Search Real cause Already visited Depth limit “The Error” Positives Negatives

  27. After the Search • Positives that are prefixes of some negative are removed • While negatives are a true subset of complete negatives, positives are an approximation based on observed behavior • This can be useful: in some reactive systems, every trace can be extended into an error trace Real cause “The Error”

  28. Positives Negatives Analysis of the Traces Now what do we do with these? • Report on code that appears • only in positives/negatives • in all positives/negatives • only and all positives/negatives (causal) • Transform positives into negatives • Find difference in invariants across positives and negatives

  29. Analysis by Code Lines • For our example trace, the line(s) in the real cause will be: • in all negative traces and • only appear in negative traces • thus will be identified as genuine cause • Includes which thread executed the code and any nondeterministic choices made • Can custom define equivalences to allow for looser definition of “the same code” Real cause Automatically identified “The Error”

  30. Common Ground Shared prefix Real cause Transformation “How to make it break” Shared suffix (control flow only) “The Error” (Can set a minimum shared prefix or suffix size)

  31. Transformation Analysis • Sorted so that minimal way to break (transformation from each positive to nearest negative) is identified • Rerun code line analysis over transformation code only (very useful in some cases) • Can identify type of error: • If any positive can be transformed into a negative by only altering thread scheduling, the error is essentially concurrency based

  32. Invariant Differences • Pick certain control locations at which to observe data values in our traces • Run a dynamic invariant detection algorithm to compute invariants across these locations for all negatives and then all positives • Compare the discovered invariants

  33. A Stack Example public IntStack (int s) { stack = new int[s]; top = 0; } public void push (int i) { stack[top++] = i; } public int pop () { top--; return stack[top]; }

  34. A Stack Example public IntStack (int s) { stack = new int[s]; top = 0; } public void push (int i) { stack[top++] = i; } public int pop () { top--; return stack[top]; } A simple stack with an interfacethat initializes the stack to a randommaximum size then randomlypushes and pops data.

  35. A Stack Example public IntStack (int s) { stack = new int[s]; top = 0; } public void push (int i) { stack[top++] = i; } public int pop () { top--; return stack[top]; } A simple stack with an interfacethat initializes the stack to a randommaximum size then randomlypushes and pops data. ORIGINAL COUNTEREXAMPLE: Stack overflow for stack size 1 (array out of bounds on stack)

  36. A Stack Example public IntStack (int s) { stack = new int[s]; top = 0; } public void push (int i) { stack[top++] = i; } public int pop () { top--; return stack[top]; } A simple stack with an interfacethat initializes the stack to a randommaximum size then randomlypushes and pops data. Run analysis with a search depth of 50 and compute invariants. ORIGINAL COUNTEREXAMPLE: Stack overflow for stack size 1 (array out of bounds on stack)

  37. A Stack Example public IntStack (int s) { stack = new int[s]; top = 0; } public void push (int i) { stack[top++] = i; } public int pop () { top--; return stack[top]; } Code line analysis Code in the push method appears in all negatives.

  38. A Stack Example public IntStack (int s) { stack = new int[s]; top = 0; } public void push (int i) { stack[top++] = i; } public int pop () { top--; return stack[top]; } Invariant analysis For positives, on entry to the push method, this invariant holds: stack.length > top For negatives, however the invariant is: stack.length >= top

  39. A Stack Example public IntStack (int s) { stack = new int[s]; top = 0; } public void push (int i) { stack[top++] = i; } public int pop () { top--; return stack[top]; } • Transformations from positive to negative • Change stack size from 3 to 1 • … • Change stack size from 4 to 2 • Insert a push • … • Remove a pop • …

  40. DEOS Error Analysis • DEOS • Real time operating system • Allows threads to be created and deleted • Has time budgets for various threads

  41. DEOS Error Analysis • DEOS • Real time operating system • Allows threads to be created and deleted • Has time budgets for various threads • Original counterexample • Assertion about the time budget of a thread fails

  42. DEOS Error Analysis • DEOS • Real time operating system • Allows threads to be created and deleted • Has time budgets for various threads • Original counterexample • Assertion about the time budget of a thread fails • Run analysis with size limit of 10

  43. DEOS Error Analysis Transformations • Change a wait until next period into a thread deletion • Insert an interrupt and system clock ticks (before a deletion already in the positive) • Change a wait until next period into an interrupt and system clock ticks and afterwards a deletion • … The DEOS error requires deletion (which appears in the set of code found in all negatives) but also relies upon specific timing, as indicated by the transformations.

  44. Related Work • “Fate and Free Will in Error Traces” (TACAS 02) • SLAM group independently investigated very similar concepts this summer (submitted to POPL) • Approach quite similar in spirit, lacks invariants and transformation approaches, but has some niceties with respect to source code analysis • Andreas Zeller: work on isolating error-causing thread schedules and cause-effect chains

  45. Future Work • More efficient algorithms for generation of the traces (try bounded model checking) • Different approaches to analysis: • Generation of automata • Characterization of error classes • Checking of causality…

More Related