slide1 l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Estimating the number of components with defects post-release that showed no defects in testing C. Stringfellow A. Andre PowerPoint Presentation
Download Presentation
Estimating the number of components with defects post-release that showed no defects in testing C. Stringfellow A. Andre

Loading in 2 Seconds...

play fullscreen
1 / 19

Estimating the number of components with defects post-release that showed no defects in testing C. Stringfellow A. Andre - PowerPoint PPT Presentation


  • 209 Views
  • Uploaded on

Estimating the number of components with defects post-release that showed no defects in testing C. Stringfellow A. Andrews C. Wohlin H. Peterson Jeremy Mange. For nearly any product, defects will appear after release

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Estimating the number of components with defects post-release that showed no defects in testing C. Stringfellow A. Andre' - agrata


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Estimating the number of components with defects post-release that showed no defects in testing

C. Stringfellow

A. Andrews

C. Wohlin

H. Peterson

Jeremy Mange

motivation

For nearly any product, defects will appear after release

The defects that were not detected in testing point to problems in the test process

We would like to know how many of these post-release defects to expect

Motivation
slide3
Using these estimates, release / re-test decisions can be made

The authors wish to compare methods of post-release defect estimation

Motivation
defect estimation

The paper compares three methods of this type of estimation

Experience-based

Capture-recapture

Curve-fitting

Experience-based requires historical data,Capture-Recapture and Curve-fitting do not

Defect Estimation
experience based estimation

Requires some sort of historical data:

Defect data

Change data

Historical data can be from previous products or prior releases of the same product

If from previous products, they must be similar for these estimations to work

Experience-based Estimation
experience based estimation6

Defect data

Number of faults in a module can be estimated using the defect history of that module

Change data

Number of faults in a module can be estimated using the number of file changes

Recent changes can be weighted more heavily

Experience-based Estimation
capture recapture estimation

A type of model originally designed for reviews and inspections

Compares the number and types of defects found from multiple test sites

Applies statistical estimation to the data based on assumptions about test sites

Capture-recapture Estimation
capture recapture estimation8

Differences between test sites are accounted for in two areas:

Ability to find defects

Probabilities of finding specific defects

This yields four models:

Same ability, same probabilities

Same ability, different probabilities

Different ability, same probabilities

Different ability, different probabilities

Capture-recapture Estimation
curve fitting estimation

Fits a mathematical curve to the data points to estimated remaining defects

Two basic types:

Decreasing – plot number of test sites that found each defect, sorted in decreasing order

Increasing – plot cumulative number of defects found by each testing event

Curve-fitting Estimation
curve fitting estimation11

With both increasing and decreasing models, the fitted curve is used to predict the number of remaining defects

Both exponential and linear prediction models exist for each type

Curve-fitting Estimation
1 collect the data for each discovered defect count the number of test sites that discovered it
1. Collect the data

For each discovered defect, count the number of test sites that discovered it

Approach
approach

2. Apply non-historical methods

Both capture-recapture and curve-fitting models are applied

These provide estimates of the number of total defects

Approach
approach14

3. Apply experience-based method

Use historical data to estimate the number of post-release defects

For each past project (or release), calculate:

Approach
approach15

4. Using the calculations, estimate post- release defects

For non-historical models, subtract the number of defects found in testing from the calculated total defects estimate

Approach
approach16

5. Compare this estimate to a decision threshold value

If the number of expected post-release defects is acceptable, the product can be released

If not, further testing must be performed

Of course, the actual release decision will be based on many criteria, this is just one of those

Approach
results

This approach was carried out in a case study on a medical record system

Results:

Capture-recapture and curve-fitting: 5%-20% error

Experience-based:3%-5% error

Results
conclusion

If data for similar projects is available, experience-based models provide the best estimations for post-release defects

However, non-historical (capture-recapture and curve-fitting) models also provide fairly accurate estimates independent of past data

Post-release defect estimates should be used in release decisions

Conclusion