1 / 18

Understanding Software Testing Termination: Cost-Effectiveness and Quality Assurance

Testing software can be costly and time-consuming, raising the question of when to terminate testing. One effective approach is Fault Seeding, where known errors are introduced into the code to estimate total faults based on detected seeded errors. Key measures of software quality include reliability, availability, and maintainability, assessed via metrics such as MTTF, MTTR, and MTBF. Factors influencing the need to ship include market drivers and resource constraints. Properly balancing testing and quality assurance is crucial to ensure a usable product while also being mindful of business necessities.

fisk
Download Presentation

Understanding Software Testing Termination: Cost-Effectiveness and Quality Assurance

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Software Test Termination 1209

  2. Testing is expensive

  3. When do we terminate testing?

  4. Finding all the bugs

  5. Fault Seeding Seed (add) errors to code Use standard techniques to identify errors Estimate total errors based on fraction of seeded errors found

  6. Fault Seeding ST: total seeded faults SD: Seeded faults detected NT: Total faults that were not seeded ND: Detected faults that were not seeded SD/ST = Fraction of seeded faults found ND/NT = Fraction of non-seeded faults found SD/ST = ND/NT

  7. Example: We seed 20 faults in a system. The testing group finds 38 faults, 8 of which were seeded. How many unseeded faults remain?

  8. Problems with this approach

  9. Software is good enough

  10. Measures of Software Quality: Usability Ship when product is usable enough based on customer usability studies

  11. Measures of Software Quality: Reliability, Availability, Maintainability • Reliability: • Availability: • Maintainability:

  12. Measures • MTTF: Mean time to failure • MTTR: Mean time to repair • MTBF: Mean time between failure MTBF = MTTF + MTTR Failure Failure System repaired Time TTF TTR TBF

  13. Software Reliability Models • Statistical models of reliability • Track the first N failures • Look for a probability distribution that matches this data • Example models include • Basic execution time • Logarithmic Poisson Execution Time

  14. Failures, not faults • Time • Execution time (more accurate) • Calendar time (more useful)

  15. Non-static failure behavior • Faults are repaired when the system fails • The probability of failure changes over time

  16. Developing models • Track failures and execution time • Time to failure • Time between failure • Fit to statistical distribution • Estimate distribution parameters • Estimate reliability for some future time

  17. Need to Ship: Factors • Market-driven need (need to be first to market) • Business need • Need to market product to generate revenue • Lack of resources (money) to continue testing • Cost/benefit analysis is positive • Political factors

  18. Retire the System • “retirement” here may mean “terminate maintenance”

More Related