1 / 14

CS527 Topics in Software Engineering (Software Testing and Analysis)

CS527 Topics in Software Engineering (Software Testing and Analysis). Darko Marinov September 13, 2011. Schedule. First few lectures to help you select projects Shared memory: CHESS, IMUnit , CAPP Message passing: Setac Comment analysis: iComment

maggy-mooney
Download Presentation

CS527 Topics in Software Engineering (Software Testing and Analysis)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS527Topics in Software Engineering(Software Testing and Analysis) Darko Marinov September 13, 2011

  2. Schedule • First few lectures to help you select projects • Shared memory: CHESS, IMUnit, CAPP • Message passing: Setac • Comment analysis: iComment • Today: Regression testing, note: survey paper • Sep 15: Model-based testing • Sep 20: Mock-based testing? • Sep 22: ? • Sep 27-29: Project proposals due • From Oct 4: Your presentations

  3. Projects • By far the most important part of the course • Meet or email me and/or Shin Hwei • Our discussion can help you select project/paper • You need to explore some topic in depth • “I don't know anything, but I do know that everything is interesting if you go into it deeply enough” Richard Feymanquote • Mostly for I2CS students: Can use the technology from your work or learn some new • Need to be able to show us some results • Preferably work in groups of 2-3 students

  4. Paper Today • Regression Testing Minimization, Selection and Prioritization: A Surveyby Shin Yoo and Mark Harman (STVR 2010) • Survey paper • Presents a broad summary of an area • Regression testing • Test Suite Minimization • Test Case Selection • Test Case Prioritization • Follows: brief overview, projects, questions

  5. … tn t2 t3 tn-1 t1 Minimization • Select a subset of T to cover all requirements r1 x x x r2 x x r3 x x … rk x x

  6. … tn t2 t3 tn-1 t1 in-1 in i2 i3 … tn-1 t1 t3 δ code 1 code 2 selection prioritization Selection and Prioritization i1

  7. Future Directions Listed • Orchestrating regression testing techniques with test data generation • Multi-objective regression testing • Problem of test oracle and its cost • Consideration of other domains • Non-functional testing • Tool support • Little text • Big issue in practice • Often ignored in research

  8. Questions for Discussion (1) • Any actual industrial pieces of software out there that can [do this]? [KN] • What is used at Microsoft/Google/Apple? [SO] • Should we consider [sequential] criteria when testing multithread programs? [YL] • Why did the text based approach work so well with simple diffs? [SB] • In which situations should we use change-aware techniques? [SO] • How [to] ensure that a discarded test case … would not detect faults? [AD]

  9. Questions for Discussion (2) • … a correlation between use of automated refactorings and test case selection? [CM] • … worth it compared to just buying more hardware and running tests in parallel? [DeG] • …the place is of survey papers in CS? [DwG] • … could we pick only one of the approaches listed in this survey? [XW] • Which regression test selection approach offers the best scalability/safety ratio? [AK] • … in a multithreaded regression test, what info would be useful to keep (POR)? [FS]

  10. Old Questions for Discussion, 1 • Correlation between fault detection and requirement coverage? [AL] • How is selection affected by refactoring? [MG] • Many techniques are proposed, are there some tools? [MG] • What's the conclusion? Is minimization technique useful? [HY] • What is minimum hitting set problem in terms of suite minimization? [JN] • Regression testing for porting systems to different OS? (Porting testing?) [DM]

  11. Old Questions for Discussion, 2 • What to do for multithreaded code? [QL] • Can minimization and selection produce same results? [ST] • Are redundant test cases a real problem? • Doesn’t the need for minimization mean that we generated counter-productive tests? • Should we minimize if that can miss faults? • How much redundancy to have in testing? • What are best practices for regression testing of GUI's?

  12. Old Sample Project Ideas (1) • [AA], [AS], [CR], [JC], [MK] • Writing a test suite for faults vs. to meet requirements [AL] • Extend a tool X to generate tests only for modified parts of the code [MG] • Compare existing approaches for test selection [MG] • Perform a large study on all apps from some open-source code base [HY] • Extend or evaluate RETEST [DM]

  13. Old Sample Project Ideas (2) • Develop system based on regression testing minimization techniques to instead prepare sets of non-regression, related test suites that have overlaps in functionality [JN] • Integrate test selection/prioritization with version control [QL] • Combine several approach (e.g., integer programming and data-flow analysis) [ST] • A project on test case minimization would be challenging… (types of experiments in CS/SE)

  14. Old Sample Project Ideas (3) • Study number of redundant or obsolete test cases in open source (how about disabled?) • Implement and compare some approchaes • Extend the CDG-based technique • Generate test cases that target changes • Integrate some of these strategies with capture-replay testing tools • Apply these techniques on web applications • Apply in other new domains • Test power consumption of embedded systems

More Related