Mining behavior graphs for backtrace of noncrashing bugs
Download
1 / 30

Mining Behavior Graphs for “Backtrace” of Noncrashing Bugs - PowerPoint PPT Presentation


  • 111 Views
  • Uploaded on

Mining Behavior Graphs for “Backtrace” of Noncrashing Bugs. Chao Liu, Xifeng Yan, Hwanjo Yu, Jiawei Han University of Illinois at Urbana-Champaign Philip S. Yu IBM T. J. Watson Research Presented by: Chao Liu. Outline . Motivations Related Work Classification of Program Executions

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Mining Behavior Graphs for “Backtrace” of Noncrashing Bugs' - harris


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Mining behavior graphs for backtrace of noncrashing bugs

Mining Behavior Graphs for “Backtrace” of Noncrashing Bugs

Chao Liu, Xifeng Yan, Hwanjo Yu, Jiawei Han

University of Illinois at Urbana-Champaign

Philip S. Yu

IBM T. J. Watson Research

Presented by: Chao Liu


Outline
Outline Bugs

  • Motivations

  • Related Work

  • Classification of Program Executions

  • Extract “Backtrace” from Classification Dynamics

  • Case Study

  • Conclusions


Motivations
Motivations Bugs

  • Software is full of bugs

    • Windows 2000, 35M LOC

      • 63,000 known bugs at the time of release, 2 per 1000 lines

  • Software failure costs

    • Ariane 5 explosion is due to “errors in the software of the inertial reference system” (Ariaen-5 flight 501inquiry board report http://ravel.esrin.esa.it/docs/esa-x-1819eng.pdf)

    • A study by the National Institute of Standards and Technology found that software errors cost the U.S.economy about $59.5 billion annuallyhttp://www.nist.gov/director/prog-ofc/report02-3.pdf

  • Testing and debugging are laborious and expensive

    • “50% of my company employees are testers, and the rest spends 50% of their time testing!”

      --Bill Gates, in 1995

Courtesy to CNN.com


Bug localization
Bug Localization Bugs

  • Automatically circle out the most suspicious places

  • Two kinds of bugs w.r.t. symptoms

    • Crashing bugs

      • Typical symptoms: segmentation faults

      • Reasons: memory access violations

    • Noncrashing bugs

      • Typical symptoms: smooth executions but unexpected outputs

      • Reasons: logic or semantic errors

      • An example


Running example
Running Example Bugs

void subline(char *lin, char *pat, char *sub)

{

int i, lastm, m;

lastm = -1;

i = 0;

while((lin[i] != ENDSTR)) {

m = amatch(lin, i, pat, 0);

if (m >= 0){

putsub(lin, i, m, sub);

lastm = m;

}

if ((m == -1) || (m == i)){

fputc(lin[i], stdout);

i = i + 1;

} else

i = m;

}

}

void subline(char *lin, char *pat, char *sub)

{

int i, lastm, m;

lastm = -1;

i = 0;

while((lin[i] != ENDSTR)) {

m = amatch(lin, i, pat, 0);

if ((m >= 0) && (lastm != m) ){

putsub(lin, i, m, sub);

lastm = m;

}

if ((m == -1) || (m == i)){

fputc(lin[i], stdout);

i = i + 1;

} else

i = m;

}

}

  • Subject program

    • replace: perform regular expression matching and substitutions

    • 563 lines of C code

    • 17 functions are involved

  • Execution behaviors

    • 130 out of 5542 test cases fail to give correct outputs

    • No incorrect executions incur segmentation faults

  • Debug method

    • Step-by-step tracing



Bug localization via backtrace
Bug Localization via Backtrace Bugs

  • Backtrace for noncrashing bugs?

  • Major challenges

    • No abnormality is visible on the surface.

    • When and where the abnormality happens.


Outline1
Outline Bugs

  • Motivations

  • Related Work

  • Classification of Program Executions

  • Extract “Backtrace” from Classification Dynamics

  • Case Study

  • Conclusions


Related work
Related Work Bugs

  • Crashing bugs

    • Memory access monitoring

      • Purify [HJ92], Valgrind [SN00], GDB …

  • Noncrashing bugs

    • Static program analysis

    • Traditional model checking

    • Model checking source code


Static program analysis
Static Program Analysis Bugs

  • Methodology

    • Examine source code directly

    • Enumerate all the possible execution paths without running the program

    • Check user-specified properties, e.g.

      • free(p) …… (*p)

      • lock(res) …… unlock(res)

      • receive_ack() … … send_data()

  • Strengths

    • Check all possible execution paths

  • Problems

    • Shallow semantics

    • Properties should be directly mapped to source code structure

  • Tools

    • ESC [DRL+98], LCLint [EGH+94], ESP [DLS02], MC Checker [ECC00] …

×


Traditional model checking
Traditional Model Checking Bugs

  • Methodology

    • Model program computation as finite state machines

    • It is described with a particular description language

    • Exhaustively explore all the reachable states in checking desired or undesired properties

  • Strengths

    • Model deeper semantics

    • Naturally fit in checking event-driven systems, like protocols

  • Problems

    • Significant amount of manual efforts in modeling

    • State space explosion

  • Tools

    • SMV [M93], SPIN [H97], Murphi [DDH+92] …


Model checking source code
Model Checking Source Code Bugs

  • Methodology

    • Execute the real program in a sandbox (e.g., virtual machine)

    • Manipulate event happenings, e.g.,

      • Message incomings

      • Return value of memory allocation

  • Strengths

    • Less significant manual specification

  • Problems

    • Application restrictions, e.g.,

      • Event-driven programs (still)

      • Clear mapping between source code and logic event

  • Tools

    • CMC [MPC+02], Verisoft [G97], Java PathFinder [BHP+-00] …


Summary of related work
Summary of Related Work Bugs

  • In summary,

    • Semantic inputs are necessary

      • Program model

      • Properties to be checked (all three methods)

    • Restricted application domain

      • Event-driven model

      • Properties are also event-related.


Example revisited
Example Revisited Bugs

void subline(char *lin, char *pat, char *sub)

{

int i, lastm, m;

lastm = -1;

i = 0;

while((lin[i] != ENDSTR)) {

m = amatch(lin, i, pat, 0);

if (m >= 0){

putsub(lin, i, m, sub);

lastm = m;

}

if ((m == -1) || (m == i)){

fputc(lin[i], stdout);

i = i + 1;

} else

i = m;

}

}

void subline(char *lin, char *pat, char *sub)

{

int i, lastm, m;

lastm = -1;

i = 0;

while((lin[i] != ENDSTR)) {

m = amatch(lin, i, pat, 0);

if (m > 0){

putsub(lin, i, m, sub);

lastm = m;

}

if ((m == -1) || (m == i)){

fputc(lin[i], stdout);

i = i + 1;

} else

i = m;

}

}

void subline(char *lin, char *pat, char *sub)

{

int i, lastm, m;

lastm = -1;

i = 0;

while((lin[i] != ENDSTR)) {

m = amatch(lin, i, pat, 0);

if (m >= 0){

putsub(lin, i, m, sub);

lastm = m;

}

if ((m == -1) || (m == i)){

fputc(lin[i], stdout);

i = i + 1;

} else

i = m;

}

}

void subline(char *lin, char *pat, char *sub)

{

int i, lastm, m;

lastm = -1;

i = 0;

while((lin[i] != ENDSTR)) {

m = amatch(lin, i, pat, 0);

if ((m >= 0) && (lastm != m) ){

putsub(lin, i, m, sub);

lastm = m;

}

if ((m == -1) || (m == i)){

fputc(lin[i], stdout);

i = i + 1;

} else

i = m;

}

}

  • No memory violations

  • Not event-driven program

  • No explicit error properties


Outline2
Outline Bugs

  • Motivations

  • Related Work

  • Classification of Program Executions

  • Extract “Backtrace” from Classification Dynamics

  • Case Study

  • Conclusions


Synopsis of program execution
Synopsis of Program Execution Bugs

  • Program behavior graphs

    • Function-level abstraction of program behaviors

    • Function calls and transitions

    • First-order sequential information about function interactions

int main(){

...

A();

...

B();

}

int A(){ ... }

int B(){ ... C() ... }

int C(){ ... }


Identification of incorrect executions
Identification of Incorrect Executions Bugs

  • A two-class classification problem

    • Every execution gives one behavior graph

    • Edges and closed frequent subgraphs as features

  • Is classification useful?

    • Classification itselfdoes not work for bug localization

      • Classifier only labels each run as either correct or incorrect as a whole

      • It does not tell when and where abnormality happens

  • Observations

    • Good classifiers know the differences between correct and incorrect execution

      • Difference, a kind of abnormality?

    • Where and when does abnormality happens?

      • Incremental classification

?


Outline3
Outline Bugs

  • Motivations

  • Related Work

  • Classification of Program Executions

  • Extract “Backtrace” from Classification Dynamics

  • Case Study

  • Conclusions


Incremental classification
Incremental Classification Bugs

  • Classification works only when instances from two classes are different.

  • Precision as a measure of the difference.

  • Incremental classification

  • Observe accuracy dynamics


Illustration precision boost

main Bugs

main

A

E

A

E

B

C

F

B

C

F

H

D

D

G

G

Illustration: Precision Boost

One Correct Execution

One Incorrect Execution


Bug relevance
Bug Relevance Bugs

  • Precision boost

    • For each function F:

      • Precision boost = Exit precision - Entrance precision.

    • Intuition & heuristics

      • Differences take place within the execution of F

      • Abnormality happens while F is in the stack

      • The larger the boost, the more likely F is relevant to the bug

  • Bug-relevant function


Outline4
Outline Bugs

  • Related Work

  • Classification of Program Executions

  • Extract “Backtrace” from Classification Dynamics

  • Case Study

  • Conclusions


Case study
Case Study Bugs

void subline(char *lin, char *pat, char *sub)

{

int i, lastm, m;

lastm = -1;

i = 0;

while((lin[i] != ENDSTR)) {

m = amatch(lin, i, pat, 0);

if (m >= 0){

putsub(lin, i, m, sub);

lastm = m;

}

if ((m == -1) || (m == i)){

fputc(lin[i], stdout);

i = i + 1;

} else

i = m;

}

}

void subline(char *lin, char *pat, char *sub)

{

int i, lastm, m;

lastm = -1;

i = 0;

while((lin[i] != ENDSTR)) {

m = amatch(lin, i, pat, 0);

if ((m >= 0) && (lastm != m) ){

putsub(lin, i, m, sub);

lastm = m;

}

if ((m == -1) || (m == i)){

fputc(lin[i], stdout);

i = i + 1;

} else

i = m;

}

}

  • Subject program

    • replace: perform regular expression matching and substitutions

    • 563 lines of C code

    • 17 functions are involved

  • Execution behaviors

    • 130 out of 5542 test cases fail to give correct outputs

    • No incorrect executions incur segmentation faults

  • Task

    • Can we circle out the backtrace for this bug?




Outline5
Outline Bugs

  • Motivations

  • Related Work

  • Classification of Program Executions

  • Extract “Backtrace” from Classification Dynamics

  • Case Study

  • Conclusions


Conclusions

Mining into Bugs

Software and

Systems?

Conclusions

  • Identify incorrect executions from program runtime behaviors.

  • Classification dynamics can give away “backtrace” for noncrashing bugs without any semantic inputs.

  • Data mining can contribute to software engineering and system researches in general.


References
References Bugs

  • [DRL+98] David L. Detlefs, K. Rustan, M. Leino, Greg Nelson and James B. Saxe. Extended static checking, 1998

  • [EGH+94] David Evans, John Guttag, James Horning, and Yang Meng Tan. LCLint: A tool for using specifications to check code. In Proceedings of the ACM SIG-SOFT '94 Symposium on the Foundations of Software Engineering, pages 87-96, 1994.

  • [DLS02] Manuvir Das, Sorin Lerner, and Mark Seigle. Esp: Path-sensitive program verication in polynomial time. In Conference on Programming Language Design and Implementation, 2002.

  • [ECC00] D.R. Engler, B. Chelf, A. Chou, and S. Hallem. Checking system rules using system-specic, programmer-written compiler extensions. In Proceedings of the Fourth Symposium on Operating Systems Design and Implementation, October 2000.

  • [M93] Ken McMillan. Symbolic Model Checking. Kluwer Academic Publishers, 1993

  • [H97] Gerard J. Holzmann. The model checker SPIN. Software Engineering, 23(5):279-295, 1997.

  • [DDH+92] David L. Dill, Andreas J. Drexler, Alan J. Hu, and C. Han Yang. Protocol verication as a hardware design aid. In IEEE International Conference on Computer Design: VLSI in Computers and Processors, pages 522-525, 1992.

  • [MPC+02] Madanlal Musuvathi, David Y.W. Park, Andy Chou, Dawson R. Engler and David L. Dill. CMC: A Pragmatic Approach to Model Checking Real Code. In Proceedings of the fifth Symposium on Operating Systems Design and Implementation, 2002.


References cont d
References (cont’d) Bugs

  • [G97] P. Godefroid. Model Checking for Programming Languages using VeriSoft. In Proceedings of the 24th ACM Symposium on Principles of Programming Languages, 1997

  • [BHP+-00] G. Brat, K. Havelund, S. Park, and W. Visser. Model checking programs. In IEEE International Conference on Automated Software Engineering (ASE), 2000.

  • [HJ92] R. Hastings and B. Joyce. Purify: Fast Detection of Memory Leaks and Access Errors. 1991. in Proceeding of the fthe Winter 1992 USENIX Conference, pages 125-138. San Francisco, California

  • [SN00] Julian Seward and Nick Nethercote. Valgrind, an open-source memory debugger for x86-GNU/Linux http://valgrind.org/

  • [LLM+04] Zhenmin Li, Shan Lu, Suvda Myagmar, Yuanyuan Zhou. CP-Miner: A Tool for Finding Copy-paste and Related Bugs in Operating System Code, in Proceeding of the 6th Symposium of Operating Systems Design and Implementation, 2004

  • [LCS+04] Zhenmin Li, Zhifeng Chen, Sudarshan M. Srinivasan, Yuanyuan Zhou. C-Miner: Mining Block Correlations in Storage Systems. In proceeding of the 3rd usenix conferences on file and storage technologies, 2004


Q & A Bugs

Thank You!


ad