in defense of unsoundness n.
Skip this Video
Loading SlideShow in 5 Seconds..
In Defense of Unsoundness PowerPoint Presentation
Download Presentation
In Defense of Unsoundness

Loading in 2 Seconds...

play fullscreen
1 / 12

In Defense of Unsoundness - PowerPoint PPT Presentation

  • Uploaded on

In Defense of Unsoundness. Ben Livshits, Manu Sridharan, Yannis Smaragdakis, and Ondřej Lhoták. April 14 – 19, 2013, Dagstuhl Seminar 13162 Pointer Analysis. Are Static Analysis (papers) Sound?. Sound: capture all program behavior Must analysis results hold during program execution?

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'In Defense of Unsoundness' - knox

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
in defense of unsoundness

In Defense of Unsoundness

Ben Livshits, Manu Sridharan, Yannis Smaragdakis, and Ondřej Lhoták

are static analysis papers sound
Are Static Analysis (papers) Sound?
  • Sound: capture all program behavior
    • Must analysis results hold during program execution?
  • Of course not!
  • Virtually all recent whole program analyses for realistic languages are unsound
unsoundness is everywhere
Unsoundness is Everywhere!
  • Omit conservative handling for common language features
  • Unsoundness lurks in the shadows
    • caveats only mentioned off-hand in an “implementation” or “evaluation” section.
nasty language features
Nasty Language Features
  • Typical (published) whole-program analysis extolls its scalability virtues and briefly mentions its soundness caveats.
    • Java: reflection and JNI
    • JavaScript: eval and dynamically computed properties
    • C/C++: assumptions about memory region and pointer arithmetic
can these language features be ignored
Can These Language Features be Ignored?
  • Most of the time the answer is no
  • These language features are nearly ubiquitous in practice.
  • "Assuming the features away" excludes the majority of input programs.
  • For example, very few JavaScript programs larger than a certain size omit at least occasional calls to eval.
could all these features be modeled soundly
Could all these Features be Modeled Soundly?
  • In principle, yes.
  • In practice, destroys the precision of the analysis
    • Must be highly over-approximate.
    • Huge imprecise result = useless.
    • Imprecision destroys scalability
soundness is not even necessary
Soundness is not Even Necessary!
  • Many clients can tolerate unsoundness.
    • IDEs (auto-complete systems, code navigation)
    • General purpose bug detectors
    • Automated refactoring tools
    • Even hints for runtime optimization
should we even try
Should We Even Try?
  • Soundness is extremely hard to achieve for a whole-program analysis in a realistic, modern language, due to programming language features that are very hard or even impossible to analyze precisely.
  • Even if achieved the precision is likely to be destroyed.
  • What is a reasonable middle ground?
  • Sound modulo inevitable unsoundness
    • “best-effort soundness”
    • “sound except for the things we all know about”
middle ground soundiness
Middle Ground: Soundiness
  • We draw a distinction between
    • a soundy analysis, which aims to capture all dynamic behaviors within reason, and
    • an unsound analysis that deliberately ignores certain behaviors
  • We argue that soundiness is a good line in the sand to draw in order to avoid abuse of the observation that "everyone's analysis is unsound."
moving forward
Moving Forward
  • Soundy is the new sound, de facto, given the research literature of the past decades.
  • Papers on unsound analyses should explain the implications of their unsoundness.
  • For nasty features, more studies should be published to characterize how extensively they are used in typical programs.
  • As a community, we should provide guidelines on how to write unsound analysis papers.

The PL research community should embrace unsound analysis techniques and tune its soundness expectations.