1 / 35

Dynamic Bug Detection & Tolerance in Software Development Dreams

Explore a comprehensive approach to detecting, reporting, and tolerating bugs in high-performance software, emphasizing dynamic optimization, self-healing systems, and prevention of memory errors. Learn how to maintain robust, fast-running applications with easy implementation and correct specifications.

fay
Download Presentation

Dynamic Bug Detection & Tolerance in Software Development Dreams

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Dynamic Bug Detection & Tolerance Kathryn S McKinley The University of Texas at Austin

  2. Software Developer Dreams

  3. Software Developer Dreams • Easy to implement specifications • Easy to get correct • Robust to errors of all sorts • Easy to maintain • Runs fast

  4. Reality thanks to David Callahan

  5. Software Developer Dreamspeople-who-want-computers-to-do-things • Easy to implement specifications • Easy to get correct • Robust to errors of all sorts • Easy to maintain • Runs fast

  6. Software Developer Dreamspeople-who-want-computers-to-do-things • Easy to implement specifications • Easy to get correct learn & get results • Robust to errors of all sorts • Easy to maintain • Runs fast

  7. Software Developer Dreamspeople-who-want-computers-to-do-things • Easy to implement specifications • Easy to get correct learn & get results • Robust to errors of all sorts • Easy to maintain • Runs fast Operating Environment Survives No unrecoverable data loss No observable implementation artifacts No violations of security or privacy policies

  8. Software Developer Dreamspeople-who-want-computers-to-do-things • Easy to implement specifications • Easy to get correct learn & get results • Robust to errors of all sorts • Easy to maintain share • Runs fast enough and consistently Operating Environment Survives No unrecoverable data loss No observable implementation artifacts No violations of security or privacy policies

  9. Impediments to Reality & the Dream Applications are more complicated • In 10 years, Windows grew from 10M to 50M lines • Application knowledge spread thin • Testing is not enough Architectures are more complicated • Transistors on a chip [Moore’s law]: • 180,000 transistors - First CMOS Vax 1987 • 291,000,000 transistors - Core2 Duo 2007 • Many core The glue is more complicated • Dynamic optimization & runtime systems • Coordination of multiple languages and runtime systems • Impossible to test all scenarios Difficult to understand programs or runtime behavior

  10. Multi-pronged Approach to Correct High Performance Software • Better languages • Java and C# are not the last programming languages • Validation when possible • We probably will not be able to validate substantial parallel applications any time soon • Is application growth outpacing validation advances? • Analysis and development tools • Static bug finding tools • Dynamic bug finding tools • Self healing & dynamically updatable systems • Don’t crash • Run Fast • New optimization opportunities • move code & data around • use JIT to specialize to input data • Evaluation

  11. Detect, Report, & Tolerate Bugs • Memory errors • find leaks [Bond ASPLOS 2006, Jump POPL 2007] • tolerate leaks [Bond in progress] • Null pointer exceptions • store a PC canary instead of zero for null [Bond et al. OOPSLA 2007] • keep running [Kent/Bond, in progress] • Untested control paths & security holes • low cost probabilistic calling context [Bond OOPSLA 2007] • Random semantic errors • data structure repair [Elkarablieh et al. OOPLSA 2007] • heap analysis [Jump in progress] • keep running [Kent/Bond in progress] • Dynamic software updating • update software while it is running [Subramanian in progress]

  12. Memory Leaks Maria Jump [POPL 2007] • Best case: increases GC workload • Worst case: systematic heap growth causes crash after days of execution A memory leakin a garbage-collected language occurs when a program inadvertently maintains references to objects that it no longer needs, preventing the collector from reclaiming space. Corkaccurately pinpoints systematic heap growth during production runs

  13. Cork’s Solution: Find growing data structures • Summarize heap objects by class & reference relationships • Calculate class points-from graph • Piggyback on full-heap object scan • Difference the graphs • Generate debugging reports • Candidate Report • Data Structure Slice Report • Allocation Site Report

  14. Class Points-From Graph =instance =class =HashTable =Queue =Queue =Company =People Heap CPFG 2 3 1 3 1 1 4 4 1 2

  15. Difference CPFGs 1 1 1 CPFGi 1 1 2 2 1 2

  16. Difference CPFGs 1 1 1 CPFGi 1 1 2 2 1 2 2 1 2 2 CPFGi+1 1 1 3 3 1

  17. Difference TPFGs 1 1 1 CPFGi 1 1 2 2 1 2 2 2 1 2 2 CPFGi+1 1 3 3 3 1 1

  18. Difference TPFGs 1 1 1 CPFGi 1 1 2 2 1 2 2 2 1 2 2 CPFGi+1 1 3 3 3 1 1 1 1 1 1 CPFGi+2 1 4 4 1

  19. Eclipse 115789 Heap Occupancy (MB) Time (MB of allocation)

  20. Eclipse 115789: CPFG • 3365 classes loaded (1773 in Eclipse) • 667 nodes • 4090 edges

  21. Eclipse 115789: Candidates Identify as candidates if rt < rthres 7 candidates String[] Object[] File Path ArrayList Folder ResourceCompareInput$ FilteredBufferedResourceNode

  22. Eclipse 115789: Slice Calculate slice from each candidate: Set of all paths (n0…nn) s.t. rn(k+1)n(k)<0 String[] Folder File Object[] Path ArrayList ResourceCompareInput$ FilteredBufferedResourceNode

  23. Slice Diagram Candidate Non-candidate String[] File IPath[] Path Folder Object[] ElementTree$ ChildIDsCache ElementTree ResourceCompareInput$ FilteredBufferedResourceNode ArrayList CompareConfiguration HashMap$ HashEntry ResourceCompareInput$ MyDiffNode JarFile HashMap$ HashEntry[] ResourceCompareInput PropertyListenerList HashMap RuleBasedCollator

  24. Eclipse 115789 Heap Occupancy (MB) Time (MB of allocation)

  25. Tolerating Memory Leaks[Mike Bond: work in progress] • Problem: • Identified leaks take time to fix • In the mean time, they degrade application and GC performance, or crash the program • Solution: • Managed languages provide an opportunity to reorganize memory

  26. Tolerating Memory Leaks Roots In-Use Space Stale Space A B C E F D G

  27. Tolerating Memory Leaks Roots In-Use Space Stale Space A B C E F D G (1) Identify stale memory

  28. Tolerating Memory Leaks Roots In-Use Space Stale Space A B C E D Bscion Bstub G F (2) Move stale memory out of application & collector working set

  29. Tolerating Memory Leaks Roots In-Use Space Stale Space A B C E D Bscion Bstub G F (3) OS pages stale space to disk & compresses it

  30. Tolerating Memory Leaks Roots In-Use Space Stale Space A B C E D Bscion Bstub G F (4) Activate object if touched

  31. Tolerating Memory Leaks Roots In-Use Space Stale Space A B C E D Bscion Bstub G F (4) Activate object if touched

  32. Tolerating Memory Leaks Roots In-Use Space Stale Space A B C E D Bscion Bstub G F Illusion of fixed leak!

  33. Throughput of Eclipse Leak

  34. Multi-pronged Approach to Correct High Performance Software • Better languages • Java and C# are not the last programming languages • Validation when possible • We probably will not be able to validate substantial parallel applications any time soon • Is application growth outpacing validation advances? • Analysis and development tools • Static bug finding tools • Dynamic bug finding tools • Self healing & dynamically updatable systems • Don’t crash • Run Fast • New optimization opportunities • move code & data around • use JIT to specialize to input data • Evaluation

  35. Thank you!www.dacapogroup.org

More Related