BugHint : A Visual Debugger Based on Graph Mining

1. BugHint: A Visual Debugger Based on Graph Mining Jennifer Leopold, Nathan Eloe, and Patrick Taylor

2. Outline • Background and related work - Debugging with beginning programmers - Visualization in debugging • BugHint - Algorithm - Graphic user interface • Experiments - Human test study and analysis of results • Future work • Questions and comments

3. Debugging with Beginning Programmers Overwhelming!!! Last thing students want to learn is a sophisticated debugger like GDB Syntax, editor, compiler, operating system commands, …

4. Debugging with Beginning Programmers Ex: Easier to see what is going on in a loop However, some intro programming classes have successfully incorporated debuggers into the teaching of various constructs

5. Visualization in Debugging Visualizationfacilitates understanding of code and data structures Java debugger jBixbe

6. Our Motivation Create an easy-to-use debugging tool that would give a beginning programmer a hint about where the bug might be in his/her program

7. BugHint: The Algorithm Input: • A program (in C or C++) • At least 1 test case that produces correct results • At least 1 test case that produces incorrect results Output: The line(s) in the code that are likely causing the bug

8. BugHint: The Algorithm How the analysis is done: (1) Construct a control flow graph (CFG) of the input program node = basic block of code edge = flow from one block to another Simple example: if (x != y) // line 1 { x = y; // line 2 y = x; // line 3 } cout << x << y; // line 4 B1 B2 B1 = line 1 B2 = lines 2, 3 B4 = line 4 B4

9. BugHint: The Algorithm How the analysis is done: (2) Construct a CFG for each input test case Case 1 Case 2 Case 3 B1 B1 B2 B1 B2 B1 = line 1 B2 = lines 2, 3 B4 = line 4 B4 B4 B4

10. BugHint: The Algorithm How the analysis is done: (3) Remove non-discriminant edges (and nodes) Case 1 Case 2 Case 3 B1 B1 B2 B1 B2 B1 = line 1 B2 = lines 2, 3 B4 = line 4 B4 B4 B4

11. BugHint: The Algorithm How the analysis is done: (4) Find the largest connected subgraph that exists in the (majority of) the incorrect graphs and doesn’t exist in the (majority of) the correct graphs B2 B2 Case 1 Case 2 Case 3 B1 = line 1 B2 = lines 2, 3 B4 = line 4 empty

12. BugHint: The Algorithm How the analysis is done: This corresponds to the block of code where the bug likely occurs In this example, the swap is not being done correctly Simple example: if (x != y) // line 1 { x = y; // line 2 y = x; // line 3 } cout << x << y; // line 4 B1 B2 B1 = line 1 B2 = lines 2, 3 B4 = line 4 B4

13. BugHint: The Algorithm Some cases are more difficult to analyze! Example: if (x <= y) // line 1 min = y; // line 2 else min = y; // line 3 cout << min; // line 4 B1 = line 1 B2 = line 2 B3 = line 3 B4 = line 4 B1 B1 B2 B2 B3 B3 B1 B2 B1 Case 2 Case 1 Case 3 B4 B4 B4 B4

14. BugHint: The Algorithm Some cases are more difficult to analyze! Example: if (x <= y) // line 1 min = y; // line 2 else min = y; // line 3 cout << min; // line 4 B1 = line 1 B2 = line 2 B3 = line 3 B4 = line 4 B1 B1 B2 B2 B3 B3 B1 B2 B1 Case 2 Case 1 Case 3 B4 B4 B4 B4

15. BugHint: The Algorithm Relax the rule that the subgraph in the incorrect case(s) does not occur in all correct case(s) Example: if (x <= y) // line 1 min = y; // line 2 else min = y; // line 3 cout << min; // line 4 Conclude that the bug is in B2, which is line 2 (should be min = x;) B2 B3 B2 Case 2 Case 1 Case 3 B1 = line 1 B2 = line 2 B3 = line 3 B4 = line 4

16. BugHint: The Algorithm How the analysis is done: • Sometimes it will be the opposite situation (i.e., it will be code that is executed in the correct cases that is not executed in the incorrect cases) • Sometimes no hint can be given because the control path will be the same for any inputs Ex: int sum = 5, total = 10; float avg; avg = sum / total; // gives 0, not 0.5

17. BugHint: The GUI Clicking on a block in a trace will highlight node and lines in code Lines that are the bug hint Can step through a selected test case execution Trace for a selected test case Specify test cases Input values for selected test case Bug hint

18. Experiments Wanted to test the hypothesis that BugHint would improve debugging skill • 163 students who had just completed CS1 • Treatment group: intro to BugHint, pre-training exercises with it • Control group: tips on manual debugging, pre-training exercises using those tips • Both groups then given programs with a bug and asked to fix it; treatment group given a bug hint

19. Experiments Wanted to test the hypothesis that BugHint would improve debugging skill Treatment group: • Better able to come up with additional test cases • Better able to add comments to the code they were debugging • Said it was easier to find and fix the bug • Didn’t take significantly less time to find and fix the bug than the control group

20. Future Work • Add ability to debug user-defined functions • Prioritize subgraphs that contain statements with multiple operators and/or particular operators (e.g., && and ||) • More usability and usefulness studies, including scalability

21. Questions? Comments?Ideas?