1 / 30

AutoFLox: An Automatic Fault Localizer for Client-Side JavaScript

AutoFLox: An Automatic Fault Localizer for Client-Side JavaScript. University of British Columbia. Frolin S. Ocariza, Jr. Karthik Pattabiraman Ali Mesbah. Web 2.0 Applications: JavaScript. JavaScript: Implementation of ECMAScript standard Client-Side JavaScript: used to develop web apps

evan
Download Presentation

AutoFLox: An Automatic Fault Localizer for Client-Side JavaScript

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. AutoFLox: An Automatic Fault Localizer for Client-Side JavaScript University of British Columbia Frolin S. Ocariza, Jr. Karthik Pattabiraman Ali Mesbah

  2. Web 2.0 Applications: JavaScript JavaScript: Implementation of ECMAScript standard Client-Side JavaScript: used to develop web apps Executes at client’s browser Web application’s functionality Dependability problem: Average of 4 errors per web app in the wild [ISSRE11] 2

  3. JavaScript Dependability: Document Object Model (DOM) html body head table div p script Text: “Hello world” tr p Element removed by JavaScript Element added by JavaScript 3

  4. JavaScript Dependability: Document Object Model (DOM) JavaScript code: var x = document.getElementById(“elem”); var x = document.getElementById(“elme”); Inexistent ID Will return null DOM-related JavaScript error div DOM: id: elem 4

  5. Fault Localization Web application testing What to do after we find errors? Need to fix them… Fault localization: Find the root cause of the error 5

  6. Contributions/Outline Discussion of JavaScript fault localization challenges Automated technique to localize DOM-related JavaScript faults Empirical studies Highlight prevalence of DOM-related JavaScript errors Evaluate accuracy and performance of technique 6

  7. Running Example • Goal: Show a fading banner that cycles through four images every 5 seconds 1 function changeBanner(bannerID) { 2 clearTimeout(changeTimer); 3 changeTimer = setTimeout(changeBanner, 5000); 4 5 prefix = “banner_”; 6 currBannerElem = document.getElementById(prefix+currentBannerID); 7 bannerToChange = document.getElementById(prefix + bannerID); 8 currBannerElem.removeClassName(“active”); 9 bannerToChange.addClassName(“active”); 10 currentBannerID = bannerID; 11 } 12 currentBannerID = 1; 13 changeTimer = setTimeout(changeBanner, 5000); 1 function changeBanner(bannerID) { 2 clearTimeout(changeTimer); 3 changeTimer = setTimeout(changeBanner, 5000); 4 5 prefix = “banner_”; 6 currBannerElem = document.getElementById(prefix+currentBannerID); 7 bannerToChange = document.getElementById(prefix + bannerID); 8 currBannerElem.removeClassName(“active”); 9 bannerToChange.addClassName(“active”); 10 currentBannerID = bannerID; 11 } 12 currentBannerID = 1; 13 changeTimer = setTimeout(changeBanner, 5000); bannerID will be set randomly Would return null if bannerID out of range Passed with no argument (even though changeBanner needs one argument) NULL EXCEPTION! setTimeout call 7

  8. JavaScript Fault Localization: Challenges Multiple JS execution sequences executing asynchronously 1 function changeBanner(bannerID) { 2 clearTimeout(changeTimer); 3 changeTimer = setTimeout(changeBanner, 5000); 4 5 prefix = “banner_”; 6 currBannerElem = document.getElementById(prefix+currentBannerID); 7 bannerToChange = document.getElementById(prefix + bannerID); 8 currBannerElem.removeClassName(“active”); 9 bannerToChange.addClassName(“active”); 10 currentBannerID = bannerID; 11 } 12 currentBannerID = 1; 13 changeTimer = setTimeout(changeBanner, 5000); 1 function changeBanner(bannerID) { 2 clearTimeout(changeTimer); 3 changeTimer = setTimeout(changeBanner, 5000); 4 5 prefix = “banner_”; 6 currBannerElem = document.getElementById(prefix+currentBannerID); 7 bannerToChange = document.getElementById(prefix + bannerID); 8 currBannerElem.removeClassName(“active”); 9 bannerToChange.addClassName(“active”); 10 currentBannerID = bannerID; 11 } 12 currentBannerID = 1; 13 changeTimer = setTimeout(changeBanner, 5000); 1 function changeBanner(bannerID) { 2 clearTimeout(changeTimer); 3 changeTimer = setTimeout(changeBanner, 5000); 4 5 prefix = “banner_”; 6 currBannerElem = document.getElementById(prefix+currentBannerID); 7 bannerToChange = document.getElementById(prefix + bannerID); 8 currBannerElem.removeClassName(“active”); 9 bannerToChange.addClassName(“active”); 10 currentBannerID = bannerID; 11 } 12 currentBannerID = 1; 13 changeTimer = setTimeout(changeBanner, 5000); Sequence 2 Sequence 1 8

  9. JavaScript Fault Localization: Challenges (2) DOM-related JS Errors – fault can be in JS code or DOM – find direct DOM access 1 function changeBanner(bannerID) { 2 clearTimeout(changeTimer); 3 changeTimer = setTimeout(changeBanner, 5000); 4 5 prefix = “banner_”; 6 currBannerElem = document.getElementById(prefix+currentBannerID); 7 bannerToChange = document.getElementById(prefix + bannerID); 8 currBannerElem.removeClassName(“active”); 9 bannerToChange.addClassName(“active”); 10 currentBannerID = bannerID; 11 } 12 currentBannerID = 1; 13 changeTimer = setTimeout(changeBanner, 5000); 1 function changeBanner(bannerID) { 2 clearTimeout(changeTimer); 3 changeTimer = setTimeout(changeBanner, 5000); 4 5 prefix = “banner_”; 6 currBannerElem = document.getElementById(prefix+currentBannerID); 7 bannerToChange = document.getElementById(prefix + bannerID); 8 currBannerElem.removeClassName(“active”); 9 bannerToChange.addClassName(“active”); 10 currentBannerID = bannerID; 11 } 12 currentBannerID = 1; 13 changeTimer = setTimeout(changeBanner, 5000); direct DOM access 9

  10. Scope of Technique Types of DOM-related JS errors Code-terminating DOM-related JS errors element = $(“elem”); b = element.getAttribute(“badAttr”) element.innerHTML = “text”; b.value = “newValue”; exception Output DOM-related JS errors function changeToBlue(elem) { elem.style.color = “red”; } Wrong colour change 10

  11. Block Diagram Web Application Trace Collection Phase Instrument JS Code Run Web Application Generate Traces Trace file Extract Relevant Sequence Trace Analysis Phase Analyze backward slice Partition into Sequences direct DOM access 11

  12. Trace Collection Intercept JS code in web app and instrument each line Run web app – instrumentation will execute for each line that executes Generate a dynamic trace of all executed lines Web Application Instrument JS Code Run Web Application Generate Traces Trace file 1 function changeBanner(bannerID) { 2 clearTimeout(changeTimer); 3 changeTimer = setTimeout(changeBanner, 5000); 4 5 prefix = “banner_”; 6 currBannerElem = document.getElementById(prefix+currentBannerID); 7 bannerToChange = document.getElementById(prefix + bannerID); 8 currBannerElem.removeClassName(“active”); 9 bannerToChange.addClassName(“active”); 10 currentBannerID = bannerID; 11 } 12 currentBannerID = 1; 13 changeTimer = setTimeout(changeBanner, 5000); 1 function changeBanner(bannerID) { 2 clearTimeout(changeTimer); trace(); 3 changeTimer = setTimeout(changeBanner, 5000); trace(); 4 5 prefix = “banner_”; trace(); 6 currBannerElem = document.getElementById(prefix+currentBannerID);trace(); 7 bannerToChange = document.getElementById(prefix + bannerID); trace(); 8 currBannerElem.removeClassName(“active”); trace(); 9 bannerToChange.addClassName(“active”); trace(); 10 currentBannerID = bannerID; trace(); 11 } 12 currentBannerID = 1; trace(); 13 changeTimer = setTimeout(changeBanner, 5000); trace(); 1 function changeBanner(bannerID) { 2 clearTimeout(changeTimer); trace(); 3 changeTimer = setTimeout(changeBanner, 5000); trace(); 4 5 prefix = “banner_”; trace(); 6 currBannerElem = document.getElementById(prefix+currentBannerID);trace(); 7 bannerToChange = document.getElementById(prefix + bannerID); trace(); 8 currBannerElem.removeClassName(“active”); trace(); 9 bannerToChange.addClassName(“active”); trace(); 10 currentBannerID = bannerID; trace(); 11 } 12 currentBannerID = 1; trace(); 13 changeTimer = setTimeout(changeBanner, 5000); trace(); Trace Record Prefix: changeBanner:::4 Variables: currentBannerID (global): 1 changeTimer (global): 2 bannerID (local): -11 prefix (local): none currBannerElem (local): none bannerToChange (local): none • Trace contains line number, function name, and values of in-scope variables • Error marker also in trace 12

  13. Trace Analysis Trace is divided into sequences Relevant sequence – execution sequence that contains error marker Analyze backward slice of last variable that held null to see where null value originated. The line where null value originated is the direct DOM access. Trace file Extract Relevant Sequence Analyze backward slice Partition into Sequences direct DOM access Web Application Instrument JS Code Run Web Application Generate Traces 13

  14. Trace Analysis: Example 1 function changeBanner(bannerID) { 2 clearTimeout(changeTimer); 3 changeTimer = setTimeout(changeBanner, 5000); 4 5 prefix = “banner_”; 6 currBannerElem = document.getElementById(prefix+currentBannerID); 7 bannerToChange = document.getElementById(prefix + bannerID); 8 currBannerElem.removeClassName(“active”); 9 bannerToChange.addClassName(“active”); 10 currentBannerID = bannerID; 11 } 12 currentBannerID = 1; 13 changeTimer = setTimeout(changeBanner, 5000); 1 function changeBanner(bannerID) { 2 clearTimeout(changeTimer); 3 changeTimer = setTimeout(changeBanner, 5000); 4 5 prefix = “banner_”; 6 currBannerElem = document.getElementById(prefix+currentBannerID); 7 bannerToChange = document.getElementById(prefix + bannerID); 8 currBannerElem.removeClassName(“active”); 9 bannerToChange.addClassName(“active”); 10 currentBannerID = bannerID; 11 } 12 currentBannerID = 1; 13 changeTimer = setTimeout(changeBanner, 5000); 1 function changeBanner(bannerID) { 2 clearTimeout(changeTimer); 3 changeTimer = setTimeout(changeBanner, 5000); 4 5 prefix = “banner_”; 6 currBannerElem = document.getElementById(prefix+currentBannerID); 7 bannerToChange = document.getElementById(prefix + bannerID); 8 currBannerElem.removeClassName(“active”); 9 bannerToChange.addClassName(“active”); 10 currentBannerID = bannerID; 11 } 12 currentBannerID = 1; 13 changeTimer = setTimeout(changeBanner, 5000); 1 function changeBanner(bannerID) { 2 clearTimeout(changeTimer); 3 changeTimer = setTimeout(changeBanner, 5000); 4 5 prefix = “banner_”; 6 currBannerElem = document.getElementById(prefix+currentBannerID); 7 bannerToChange = document.getElementById(prefix + bannerID); 8 currBannerElem.removeClassName(“active”); 9 bannerToChange.addClassName(“active”); 10 currentBannerID = bannerID; 11 } 12 currentBannerID = 1; 13 changeTimer = setTimeout(changeBanner, 5000); Technique outputs this line as direct DOM access bannerToChange being assigned return value of DOM access function Last variable to take on null value Error Marker Sequences.: (1) line2 -> line3 -> line5 -> line6 -> line7 -> line8 -> line9 (2) line12 -> line13 Relevant Seq.: line12 -> line13 -> line2 -> line3 -> line5 -> line6 -> line7 -> line8 -> line9 14

  15. Implementation of AutoFLox Trace Collection: Modified versions of existing tools (InvarScope [Groeneveld et al.] and Crawljax [Mesbah et al.]) Trace Analysis: Written from scratch 15

  16. Evaluation: Research Questions RQ1: How prominent are DOM-related JavaScript errors in web applications? RQ2: What is the fault localization accuracy of AutoFLox? RQ3: What is the performance overhead of AutoFLox? 16

  17. Prominence of DOM-related JS Errors RQ1 Approach: Study bug reports of four web apps (TUDU, TaskFreak, WordPress, and Google) Only JS-related bugs marked valid or fixed are studied 17

  18. Research Questions RQ1: How prominent are DOM-related JavaScript errors in web applications? RQ2: What is the fault localization accuracy of AutoFLox? RQ3: What is the performance overhead of AutoFLox? 18

  19. Accuracy of AutoFLox RQ2 Approach: Fault injection experiment on TUDU, TaskFreak, and WordPress Faults injected: Null exceptions Mutate the parameters of getElementById and getAttribute methods 19

  20. Accuracy of AutoFLox: Results WordPress: anonymous functions not supported by AutoFLox at the moment Direct DOM access in Tumblr also found 20

  21. Research Questions How prominent are DOM-related JavaScript errors in web applications? What is the fault localization accuracy of AutoFLox? What is the performance overhead of AutoFLox? 21

  22. Performance RQ3 Approach: Measure trace collection overhead in Tumblr; measure time it takes for trace analysis to complete Tumblr used because production code more complex than development code Results Trace collection incurred 35% overhead Trace analysis took 0.115 seconds to complete 22

  23. Conclusion DOM-related JS errors are prevalent Created fault localization tool for JS called AutoFLox http://ece.ubc.ca/~frolino/projects/autoflox/ Evaluated accuracy and performance of AutoFLox Future work Relax simplifying assumptions Target Output DOM-related JS errors Automatically fixing DOM-related JS errors 23

  24. Backup Slides

  25. Web 2.0 Applications: Document Object Model - Defines elements contained in webpage 25

  26. Dependability Problems in JavaScript Big contributing factor: JS interaction with Document Object Model (DOM) DOM JavaScript Code DOM-related JavaScript error DIV id: elem var x = document.getElementById(“elem”); var x = document.getElementById(“elme”); Will return null Inexistent ID 26

  27. Fault Localization Web application testing What to do after we find errors? Need to fix them… Fault localization: Find the root cause of the error x = document.getElementById(“elme”); changeText(x); ….. function changeText(domElement) { y = domElement; y.innerHTML = ‘New Text’; } root cause exception 27

  28. Running Example 1 function changeBanner(bannerID) { 2 clearTimeout(changeTimer); 3 changeTimer = setTimeout(changeBanner, 5000); 4 5 prefix = “banner_”; 6 currBannerElem = document.getElementById(prefix+currentBannerID); 7 bannerToChange = document.getElementById(prefix + bannerID); 8 currBannerElem.removeClassName(“active”); 9 bannerToChange.addClassName(“active”); 10 currentBannerID = bannerID; 11 } 12 currentBannerID = 1; 13 changeTimer = setTimeout(changeBanner, 5000); bannerID will be set randomly Would return null if bannerID out of range Passed with no argument (even though changeBanner needs one argument) NULL EXCEPTION! 28

  29. AutoFLox: Scope Types of DOM-related JS errors Code-terminating DOM-related JavaScript Errors: Errors that occur when DOM access function returns null, undefined, or incorrect value Output DOM-related JavaScript Errors: Errors that occur when DOM update function sets DOM element’s property to wrong value AutoFLox is concerned with code-terminating DOM-related JavaScript errors 29

  30. Limitations User has to know how to replicate error User required to specify elements to click during crawling Multiple JS errors currently not supported 30

More Related