Bugbench a benchmark for evaluating bug detection tools
This presentation is the property of its rightful owner.
Sponsored Links
1 / 6

BugBench: A Benchmark for Evaluating Bug Detection Tools PowerPoint PPT Presentation


  • 89 Views
  • Uploaded on
  • Presentation posted in: General

BugBench: A Benchmark for Evaluating Bug Detection Tools. Shan Lu, Zhenmin Li , Feng Qin, Lin Tan, Pin Zhou and Yuanyuan Zhou University of Illinois, Urbana-Champaign. Content of This Talk. Share our experience Bug/application characteristics analysis BugBench has been used by

Download Presentation

BugBench: A Benchmark for Evaluating Bug Detection Tools

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Bugbench a benchmark for evaluating bug detection tools

BugBench: ABenchmarkfor EvaluatingBug Detection Tools

Shan Lu, Zhenmin Li, Feng Qin, Lin Tan, Pin Zhou and Yuanyuan Zhou

University of Illinois, Urbana-Champaign


Content of this talk

Content of This Talk

  • Share our experience

  • Bug/application characteristics analysis

  • BugBench has been used by

    • Our previous work [Micro’04, ISCA’04, HPCA’05]

    • Other research groups: UCSD, Purdue, NCSU, etc.


Current benchmark suite

Current Benchmark Suite

memory related

multi-thread related

semantic

Other type of bugs: In searching …


Functionality

Functionality

Valgrind

Purify

CCured

miss stack buffer overflow

miss moderate global-buffer overflow

miss stack buffer overflow

miss 1 Byte global-buffer overflow

Failed to apply


Overhead

NCOM

BC

Memory Alloc Freq. (# per MInst)

769

0

138

480

NCOM

BC

BC

NCOM

Mem. Access Freq. (# per Instruction)

.48

.5

.52

.55

.62

.85

.65

.69

Overhead

  • Valgrind: 6.4X (NCOM) ~ 119X (BC)

  • Purify: 28% (POLY) ~ 76X (BC)

  • CCured: 4% (POLY) ~ 3.7X (GZIP)

1.35X

69%

18%

28%

4%

Heap Usage Ratio[Heap/(Heap+Stack)]

85.1%

99%

0%

23.9%

76.6%


Experience summary

Experience Summary

  • Building benchmark is a time-consuming and long-term work

    • Motivate automatic tools to extract bugs

  • Bug/application characteristics are important for selecting applications

  • Need cooperation from entire community


  • Login