CS527: Advanced Topics in Software Engineering (Software Testing and Analysis) - PowerPoint PPT Presentation

CS527: Advanced Topics in Software Engineering(Software Testing and Analysis)

Darko Marinov

September 11, 2007

• Is everyone on mailing list and Wiki?
• Coursework
• Project [50%]
• Presentation [20%]
• Participation (reports and discussion) [20%]
• Problem set(s) [10%]
• Fair warning: this is NOT easy
• Project requires that you explore some topic in great depth

• Aug 28 & 30: Darko and Brett
• Sep 4 & 6: Danny Dig and Shan Lu
• Sep 11 & 13: Darko
• Software testing and model checking
• Sep 18 & 20: Darko
• Static analysis and (likely) software testing
• Sep 25 on: your paper presentations
• Sep 25: initial project proposals

• Test thoroughly a piece of software
• Apply some technique(s)/tool(s) that we read about on some piece of software
• Reimplement a tool (e.g., for another language)
• Evaluate some techniques/tools on a number of case studies
• Develop a new technique/tool
• …

• Accurate exploitable buffer overflow detection with static analysis
• Compare effectiveness and efficiency of several tools on some pieces of code
• What tools to use? What code to use?
• Other topics mentioned on the list
• Automated GUI testing
• Stress testing of web apps
• Testing embedded code/virtual machines

• One good point
• One bad point
• One potential project
• One question for discussion
• Looking for depth
• Length is not important; answers that are too long can make it hard to find the depth

• Due by lecture time, 2pm CST
• Please follow these rules
• Email Brett (bdaniel3) & presenter
• Put cs527 in the subject line
• ASCII text (no PDF/DOC files)
• No attachments

• Feedback-directed random test generation by Carlos Pacheco, Shuvendu K. Lahiri, Michael D. Ernst, and Thomas Ball(ICSE 2007)
• Slides courtesy of Carlos Pacheco

• On papers: How much background to assume about paper audience?
• Most papers written for experts in the field but not necessarily experst on specific topic
• E.g., software engineering but not software testing
• You should follow references for papers you present or base your projects on
• On tool: Is there a RANDOOP web site?
• Some other tools: JPF, Korat, ASTGen

• Evaluation considered mostly good, but
• Why evaluate on data structures?
• Why not compare with more other tools?
• What kind of bugs were found?
• Any feedback from developers on bugs?
• When is this technique good/bad?
• What to use to complement RANDOOP?
• Why is this solution better than others?

• What about non-primitive inputs?
• Can RANDOOP generate input programs?
• What about testing business systems?
• How to reduce # of illegal sequences?
• How many seeds to use in runs?
• Is time a good stopping condition?
• What are general pros/cons of random testing?