1 / 23

Backstop: A Tool for Debugging Runtime Errors

Backstop: A Tool for Debugging Runtime Errors. Chris Murphy, Eunhee Kim, Gail Kaiser, Adam Cannon Columbia University. Observation. Novice/CS1 Java programmers have difficulty dealing with uncaught exceptions and the resulting stack traces.

berenicee
Download Presentation

Backstop: A Tool for Debugging Runtime Errors

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Backstop: A Tool for Debugging Runtime Errors Chris Murphy, Eunhee Kim, Gail Kaiser, Adam Cannon Columbia University

  2. Observation • Novice/CS1 Java programmers have difficulty dealing with uncaught exceptions and the resulting stack traces java.lang.NumberFormatException: For input string: "4.5" at java.lang.NumberFormatException.forInputString (NumberFormatException.java:48) at java.lang.Integer.parseInt(Integer.java:456) at java.lang.Integer.parseInt(Integer.java:497) at Test.main(Test.java:124)

  3. Problems • Students are often confused by runtime errors in Java • Stack traces were not developed with the introductory-level CS student in mind! • Few, if any, tools exist to help novice programmers interpret and debug runtime errors

  4. Our Solution: Backstop • A command-line tool to produce detailed and helpful error messages when an uncaught runtime error (exception) occurs • Also provides debugging support by displaying the lines of code that are executed as a program runs, as well as the values of any variables on those lines

  5. Hypothesis • “Backstop allows students to understand and debug runtime errors more easily than just the Java stacktrace” • Backstop is part of Columbia University’s ongoing efforts to develop tools to help CS1 students work with Java

  6. Overview • Related Work • Handling of Runtime Errors • Usability Study • Debugging Support • Conclusions and Future Work

  7. Related Work • Programming environments with easier-to-understand compiler errors • BlueJ, ProfessorJ, DrJava, … • Identify most common compiler errors • Gauntlet, Expresso, HiC, … • Identifying logical errors • InSTEP, DEBUG, … • Debugging • CMeRun

  8. Handling of Runtime Errors • Designed to catch any uncaught exceptions in the student’s program • Once the exception (or any other Java runtime error) is caught, interpret its meaning and provide a user-friendly message that also seeks to provide some enlightenment as to what caused the error

  9. Handling of Runtime Errors • BackstopRuntime “wraps” the student’s (compiled) Java program • Uses reflection to invoke the “main” method • Catches any uncaught exception and dispatches it to the appropriate handler • StackTraceElements and source code are used to produce a friendlier, more helpful error message that is appropriate for the type of exception

  10. ********************************************************* ************************ ERROR ************************** ********************************************************* An error occurred at line 124 of Test.java in the method "main". I'm sorry, the program cannot continue. The line of code that caused the error is int myNum = Integer.parseInt(arg); It seems that the code wants to convert a String to an integer. However, the String "4.5" appears to be a floating point value, not an integer. Don’t forget, an integer can’t contain a decimal point. You may want to use a different datatype to store the value, like float. This is called a NumberFormatException. The full error is as follows: java.lang.NumberFormatException: For input string: "4.5" at java.lang.NumberFormatException.forInputString (NumberFormatException.java:48) at java.lang.Integer.parseInt(Integer.java:456) at java.lang.Integer.parseInt(Integer.java:497) at Test.main(Test.java:124) Refer to “the code”, not “you” Reinforce what is already known Make a suggestion Also use Java terms

  11. Usability Study • Research Question: “Is the information provided by Backstop helpful for understanding runtime errors?” • 17 students (8 male, 9 female) who had just completed a CS1 summer course • Students were asked to find/fix a runtime error in code they did not write • After completion, were asked subjective questions about the experience

  12. Usability Study • Subject was shown the code for a program that reads a String and displays the number of occurrences of each letter • Subject was then told how the algorithm works • It was then suggested that the code would produce an error for some inputs • Subject was suggested to try a sentence like “hello world” • Backstop error appeared

  13. Problem Code // loop through and increment the values for each char for (int i = 0; i < s.length(); i++) { // get the character at index i char c = s.charAt(i); // increment the value in the array of occurrences // figure out its index by subtracting 'a' from it occur[(int)(c - 'a')]++; }

  14. **************************************************** ********************** ERROR *********************** **************************************************** An error occurred at line 30 of StringCount.java in the method "main". I'm sorry, the program cannot continue. The line of code that caused the error is occur[(int)(c - 'a')]++; It seems that the code tried to use an illegal value as an index in an array. The code was trying to access an element at index -65 of the array called "occur". The expression "(int)(c - 'a')" had the value -65 when the error occurred. Remember, you cannot have a negative index. Be sure that the index is always positive. This error is called an ArrayIndexOutOfBoundsException. The full error is as follows: java.lang.ArrayIndexOutOfBoundsException: -65 at StringCount.main(StringCount.java:30)

  15. Questions • Was it more useful than the stacktrace? • All said “yes” • Did the output mislead you? • Two said “yes”; one admitted he didn’t read it • Describe the tone (vs. stack trace) • Six said it was “friendly” • Four said “too long” • Two said “too pedantic for a general audience” • Three did not notice it

  16. Observations • Most common complaint was that the output is too long • One possibility is giving the option to see more details or user-adjusted verbosity levels • Average time to complete the task with Backstop was around 5.8 minutes • Understanding the cause took 1-2 minutes

  17. General Debugging Support • Display each line of code as it gets executed • File name • Line number • Source code • Also display the values of any variables

  18. Original source code sum = x + y; y = x; Line number printed before code is executed, in case of exception Modified source code System.out.print("\nFib.java:27: sum = x + y;"); System.out.print(" (x was " + x);System.out.print(" , y was " + y );System.out.print(")"); sum = x + y; System.out.print(" (sum is now " + sum + ")"); System.out.print("\nFib.java:29: y = x;");System.out.print(" (x was " + x);System.out.print(")");y = x; System.out.print(" (y is now " + y + ")"); Everything on one line to retain original line number Verb tense indicates change of variable values Output at runtime Fib.java:27: sum = x + y; (x was 2 , y was 1) (sum is now 3) Fib.java:29: y = x; (x was 2) (y is now 2)

  19. StringCount.java:27: char c = s.charAt(i); (i was 0) (c is now h) StringCount.java:30: occur[(int)(c - 'a')]++; (occur[(int)(c - 'a')] is now 1) StringCount.java:27: char c = s.charAt(i); (i was 1) (c is now e) StringCount.java:30: occur[(int)(c - 'a')]++; (occur[(int)(c - 'a')] is now 1) StringCount.java:27: char c = s.charAt(i); (i was 2) (c is now l) StringCount.java:30: occur[(int)(c - 'a')]++; (occur[(int)(c - 'a')] is now 1) StringCount.java:27: char c = s.charAt(i); (i was 3) (c is now l) StringCount.java:30: occur[(int)(c - 'a')]++; (occur[(int)(c - 'a')] is now 2) StringCount.java:27: char c = s.charAt(i); (i was 4) (c is now o) StringCount.java:30: occur[(int)(c - 'a')]++; (occur[(int)(c - 'a')] is now 1) StringCount.java:27: char c = s.charAt(i); (i was 5) (c is now !) StringCount.java:30: occur[(int)(c - 'a')]++; ************************************************************************* ********************************* ERROR ********************************* ************************************************************************* An error occurred at line 30 of StringCount.java in the method "main". I'm sorry, the program cannot continue. The line of code that caused the error is occur[(int)(c - 'a')]++; It seems that the code tried to use an illegal value as an index to an array. The code was trying to access an element at index -64 of the array called "occur". The variable "(int)(c - 'a')" had the value -64 when the error occured. Remember, you cannot have a negative index. Be sure that the index is always positive. This error is called an ArrayIndexOutOfBoundsException. The full error is as follows: java.lang.ArrayIndexOutOfBoundsException: -64 at StringCount.main(StringCount.java:30) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

  20. Questions • Was it helpful? • All but one said “yes” • Did the output mislead you? • Five said “yes”, but four said “only at first” • Describe the appearance • Five said more spacing would make it easier to read • Four said more spacing would make it harder to read • Three said the spacing would make no difference

  21. Evaluation • Backstop helped students identify and fix errors in code they did not write • Backstop automates/mimics the way that many CS1 students debug a Java program on the command line • Still need a study involving the use of both parts of Backstop to debug a runtime error

  22. Conclusion and Future Work • Our contribution: Backstop • Makes the error messages produced by uncaught Java exceptions easier to understand (compared to the Java stacktrace) • Provides friendlier and more useful information about how to fix the error • Automates simple debugging approach • Future work could include integration with an IDE, and further studies to measure Backstop’s effectiveness

  23. Backstop: A Tool for Debugging Runtime Errors Chris Murphy, Eunhee Kim, Gail Kaiser, Adam Cannon Columbia University http://www.cs.columbia.edu/~cmurphy/backstop cmurphy@cs.columbia.edu

More Related