Projecting quality
This presentation is the property of its rightful owner.
Sponsored Links
1 / 18

Projecting Quality PowerPoint PPT Presentation


  • 51 Views
  • Uploaded on
  • Presentation posted in: General

Projecting Quality. Previously, we used Defect Removal Effectiveness as a guidance to formulate a projection mechanism: MP : Problems Removed via Inspection(s) PTR: Problems Removed via Testing(s) (with the above 2 assumptions ---- then)

Download Presentation

Projecting Quality

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Projecting quality

Projecting Quality

  • Previously, we used Defect Removal Effectiveness as a guidance to formulate a projection mechanism:

    • MP : Problems Removed via Inspection(s)

    • PTR: Problems Removed via Testing(s)

      (with the above 2 assumptions ---- then)

    • Q : Field Problems = PTR2/ (MP-PTR)

  • Are there other models we can use ?


Projecting field problems

Projecting Field Problems

  • 2 Basic Models :

    • Static Model :

      • # of Field Problem or Q = f ( x1, x2, - - - - -, xn)

        where x’s are a set of parameters such as size of the software, skills of the people, development schedule, etc.

      • The coefficients of x’s used in the computation come from previous, similar projects.

      • The computation of # of Field Problems for the project is assumed to be one from a population of similar projects.


Projecting field problems1

Projecting Field Problems

  • Dynamic Model :

    • Estimate is based on multiple measurements taken from the project of interest, not from other projects, but from the current project

    • It is more specific to the project or product.

    • It is often expressed in terms of time.

  • We will use one such model to simulate the complete product life cycle.


Weibull distribution family

Weibull Distribution family

  • Has been used by other disciplines for reliability analysis and projecting the defect as a function of time.

  • The probability density function :

    f(t) = (m/t) * ((t/c)**m) * (e ** -((t/c)**m) )

    = (m/t) (t/c)m e (–t/c) **m

    • m is a parameter that shapes the curve

    • t is the time

    • c is a parameter which is a function of t when the curve reaches maximum point.

  • The cumulative distribution function : (integrate the above)

    F(t) = 1 – (e** -((t/c)**m) )


  • Weibull curves with differing m s

    Weibull Curves (with differing m’s)

    f(t) = (m/t) * ((t/c)**m) * ( e** (-(t/c)**m) )

    m= 8

    problems

    found

    m= 2

    m= 4

    m= 1

    t: some unit of time


    Rayleigh curve

    Rayleigh Curve

    • A special case of Weibull curve where m=2 is called the Rayleigh curve

      f(t) = (2/t) * ((t/c)**2) * ( e** (-(t/c)**2) )

    At tmax , the area under

    the curve is ~39.3%

    problems

    found

    m= 2

    tmax

    t: some unit of time


    Rayleigh curve1

    Rayleigh Curve

    • The area under the f(t) curve (normalized to a total of 1) is the (% of)accumulative total problems found through some time. The integral of f(t) is the cumulative distribution function:

      F(t) = 1 - (e** (-(t/c)**2) )

    total

    problems

    found

    t : time units


    Applying rayleigh curve concept to discrete measurements

    Applying Rayleigh Curve concept to Discrete Measurements

    • Let’s map software defect removal activities into the Rayleigh Curve: (Some projects have found the mapping to hold)

    Io = requirement inspection

    I1 = design inspection

    I2 = code inspection

    UT= unit testing

    CT = component testing

    ST = system testing

    FP = field problems

    problems

    found

    I

    I

    CT

    ST

    FP or Q

    I

    UT

    0

    1

    2

    t : product life phases


    Projection of field problems with rayleigh curve

    Projection of Field Problems with Rayleigh Curve

    • If your project appears to show a Rayleigh curve through the first couple of inspections

      • Continue to track against the curve

      • See if the peak point is close to UT phase. Assign that phase or time as the point where f(t) reaches the max point.

      • Use that max time to compute the value of “c”for the Rayleigh curve for your product: c= t * 2 (see next chart)

      • f(t) = (2/t) * ((t/c)**2) * (e** (-(t/c)**2))

    max


    Solving for the c parameter

    Solving for the “c” parameter

    • A Way to solve for the parameter, c, in Rayleigh curve

      • Take the first derivative of f(t) and set it to 0 to find the time at which f(t) peaks.

      • d[f(t)] = d[(2/t) * ((t/c)**2) * (e**(-(t/c)**2)) ]

      • = d[ (2t/(c**2)) * (e**(-(t/c)**2)) ]

      • = (2t/(c**2) ) d[e**(-(t/c)**2)] + d[2t/(c**2)](e**(-(t/c)**2))

      • = (2t/(c**2)) [(e**(-(t/c)**2)) d[-(t/c)**2] + {second part}

      • = (2t/(c**2))[(e**(-(t/c)**2)) * (-2t/c**2)] + {second part}

      • = ((-4t**2)/(c**4)) (e**(-(t/c)**2)) + {second part}

      • = {first part} + (2/(c**2)) (e**(-(t/c)**2)

      • = (e**(-(t/c)**2)) *[( (-4t**2)/c**4)) + (2/c**2)] = 0

      • (-4t**2)/(c**4) +(2/c**2) = 0

      • (-4t**2)/(c**4) = - (2/c**2)

      • -(2t**2) = - (c**2)

      • t**2 = (c**2)/2

      • t = c / (sqrt(2))

    look for ‘max’


    Try an example with rayleigh curve

    Try an Example with Rayleigh Curve

    • Assume the following information (collected data)

      • I0 : at time 1 with 9 problems found

      • I1 : at time 2 with 12 problems found

      • I2 : at time 3 with 17 problems found

      • UT : at time 4 with 10 problems found

      • CT : at time 5 with 6 problems found

      • ST : at time 6 with 3 problems found

      • FP : problems in remaining time (field problems)

    • tmax is approximated at time unit 3, or I2, in this case

    = total of 57 problems


    Example continued

    Example Continued

    • Using the previous equation : tmax = c /sqrt(2)

    • c = tmax * sqrt(2) = 3 * 1.41 = 4.23

    • Use the accumulative F(t)= 1 – e**(-(t/c)**2)

    • F(6) = 1 – e**(-(6/4.23)**2) = 1 – e**(-1.99) = 1- (.135) = .865

    • Since 1 is the total area under the curve, the remaining area under the curve is .135.

    • Also since .865 is approximately equal to the total problems found up to end of system test, which is 57 problems, then the total life cycle is estimated at 66 problems. (57/.865 = 66)

    • Thus the remaining problems in the field is estimated at 66-57 = 9 problems.

    • FP (field problems) is estimated at 9 problems


    Compare with the previous estimation method

    Compare with the Previous Estimation Method

    • Consider the previous estimation of Field Problems

      Q = PTR/(μ-1) = PTR / [(MP/PTR) -1] = PTR2/(MP-PTR)

    • Substituting the numbers :

      • Q = 192/(38 -19)

      • Q = 361/19 = 19 problems

    • Note the difference between the 2 estimates with the same assumed problems found rates :

      • Q with Rayleigh Curve approach estimated at 9 problems

      • Q with the PTR2/(MP-PTR) estimated at 19 problems.

        (Remember the “earlier” estimate assumed MP/TD = PTR/(TD-MP) which in this case 38/66 does NOT equal 19/28)


    Estimates

    Estimates

    • Remember that these are still estimates and not all the conditions are known, yet.

    • Two assumptions are used in using the Rayleigh curve to model quality of software:

      • The defect rate observed during development activities is highly correlated to the defect rate in the field.

      • Given the same error injection rate, then the more defects discovered early in the development cycle, the fewer will remain in the system at later stages.


    Correlating development phase i0 inspection defect rate with field defect rate

    Correlating Development Phase I0-inspection, Defect Rate with Field Defect rate

    Rank field defect

    rate by module

    Rank I0

    by module

    (Diff)2

    Diff

    -3

    -18

    -22

    -5

    .

    .

    .

    +13

    9

    324

    484

    25

    .

    .

    .

    169

    4

    20

    25

    9

    .

    .

    .

    52

    1

    2

    3

    4

    .

    .

    .

    65

    Spearman’s Rank Order Correlation coefficient = 1 – {[6* Σ(diff)2] / [n *(n2-1)]}

    Using this technique, book author found low correlation for I0, I1 to field defects

    and higher correlation of I2, CT, and ST to field defects


    More thoughts illustrated with rayleigh curve

    More “thoughts” illustrated with Rayleigh curve

    Using the same process of error removals for a set of modules,

    we have been noticing that if we find lots of bugs in a module, X, relative to other modules, then that same module, X, tends to be more error prone (have high # of bugs) in the field also.

    [ A uniform upward shift effect - no shape change]

    # of

    Defects

    found

    More error prone module

    A “typical” module

    product release

    day

    Time


    More thoughts illustrated with rayleigh curve1

    More “thoughts” illustrated with Rayleigh Curve

    We have also justified the concept of finding more bugs prior to release is

    a good scenario because less will be in the product after release. So if more bugs

    are found in a module, X, relative to other modules prior to release should be a good

    sign. How does that relate to the comments on a module with lots of bugs found prior to

    release tends to be the same module that is error prone in the field?

    [ A change in shape of the curve ---due to modified process ]

    Modified or improved process and the

    number of problems removed

    shifted to the front, resulting

    in less problems detected after release.

    Original process and the

    number of problem found

    Product release


    Reliability and predictive validity of models

    Reliability and Predictive Validity of Models

    • Reliability of the Model refers to the confidence we have of the model in terms of its estimates.

      • Dos the output from the model vary greatly from one input set to another input set?

    • Predictive Validity of the Model refers to the accuracy of the model estimates, especially compared to empirical data

      • The author found that using Rayleigh curve model across different systems such as AS/400, S/38/ and S/36 data showed that the field problems discovery rate is consistently underestimated. The model had to be adjusted to set m= 1.8 for the IBM mid-range systems.


  • Login