Introducing Bayesian Nets in AgenaRisk
This presentation is the property of its rightful owner.
Sponsored Links
1 / 58

Introducing Bayesian Nets in AgenaRisk An example based on Software Defect Prediction PowerPoint PPT Presentation


  • 155 Views
  • Uploaded on
  • Presentation posted in: General

Introducing Bayesian Nets in AgenaRisk An example based on Software Defect Prediction. Typical Applications. Predicting reliability of critical systems Software defect prediction Aircraft accident traffic risk Warranty return rates of electronic parts

Download Presentation

Introducing Bayesian Nets in AgenaRisk An example based on Software Defect Prediction

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Introducing bayesian nets in agenarisk an example based on software defect prediction

Introducing Bayesian Nets in AgenaRisk

An example based on

Software Defect Prediction


Typical applications

Typical Applications

  • Predicting reliability of critical systems

  • Software defect prediction

  • Aircraft accident traffic risk

  • Warranty return rates of electronic parts

  • Operational risk in financial institutions

  • Hazards in petrochemical industry


Typical applications1

Typical Applications

  • Predicting reliability of critical systems

  • Software defect prediction

  • Aircraft accident traffic risk

  • Warranty return rates of electronic parts

  • Operational risk in financial institutions

  • Hazards in petrochemical industry


Typical applications2

Typical Applications

  • Predicting reliability of critical systems

  • Software defect prediction

  • Aircraft accident traffic risk

  • Warranty return rates of electronic parts

  • Operational risk in financial institutions

  • Hazards in petrochemical industry


Typical applications3

Typical Applications

  • Predicting reliability of critical systems

  • Software defect prediction

  • Aircraft accident traffic risk

  • Warranty return rates of electronic parts

  • Operational risk in financial institutions

  • Hazards in petrochemical industry


Typical applications4

Typical Applications

  • Predicting reliability of critical systems

  • Software defect prediction

  • Aircraft accident traffic risk

  • Warranty return rates of electronic parts

  • Operational risk in financial institutions

  • Hazards in petrochemical industry


Typical applications5

Typical Applications

  • Predicting reliability of critical systems

  • Software defect prediction

  • Aircraft accident traffic risk

  • Warranty return rates of electronic parts

  • Operational risk in financial institutions

  • Hazards in petrochemical industry


A bayesian net for predicting air traffic incidents

A Bayesian Net for predicting air traffic incidents


A detailed example

A Detailed Example

  • What follows is a demo of a simplified version of a Bayesian net model to provide more accurate predictions of software defects

  • Many organisations worldwide have now used models based around this one


Predicting software defects

Operational defects

Predicting software defects

The number of operational defects (i.e. those found by customers) is what we are really interested in predicting


Predicting software defects1

Residual Defects

Operational defects

Predicting software defects

We know this is clearly dependent on the number of residual defects.


Predicting software defects2

Residual Defects

Operational usage

Operational defects

Predicting software defects

But it is also critically dependent on the amount of operational usage. If you do not use the system you will find no defects irrespective of the number there.


Predicting software defects3

Defects Introduced

Residual Defects

Operational usage

Operational defects

Predicting software defects

The number of residual defects is determined by the number you introduce during development….


Predicting software defects4

Defects Introduced

Defects found

and fixed

Residual Defects

Operational usage

Operational defects

Predicting software defects

…minus the number you successfully find and fix


Predicting software defects5

Defects Introduced

Defects found

and fixed

Residual Defects

Operational usage

Operational defects

Predicting software defects

Obviously defects found and fixed is dependent on the number introduced


Predicting software defects6

Defects Introduced

Problem

complexity

Defects found

and fixed

Residual Defects

Operational usage

Operational defects

Predicting software defects

The number introduced is influenced by problem complexity…


Predicting software defects7

Design process

quality

Defects Introduced

Problem

complexity

Defects found

and fixed

Residual Defects

Operational usage

Operational defects

Predicting software defects

….and design process quality


Predicting software defects8

Design process

quality

Defects Introduced

Problem

complexity

Testing Effort

Defects found

and fixed

Residual Defects

Operational usage

Operational defects

Predicting software defects

Finally, how many defects you find is influenced not just by the number there to find but also by the amount of testing effort


A model in action

A Model in action

Here is that very simple model with the probability distributions shown


A model in action1

A Model in action

We are looking at an individual software component in a system


A model in action2

A Model in action

The prior probability distributions represent our uncertainty before we enter any specific information about this component.


A model in action3

A Model in action

So the component is just as likely to have very high complexity as very low


A model in action4

A Model in action

and the number of defects found and fixed in testing is in a wide range where the median value is about 20.


A model in action5

A Model in action

As we enter observations about the component the probability distributions update


Introducing bayesian nets in agenarisk an example based on software defect prediction

Here we have entered the observation that this component had 0 defects found and fixed in testing


Introducing bayesian nets in agenarisk an example based on software defect prediction

Note how the other distributions changed.


Introducing bayesian nets in agenarisk an example based on software defect prediction

The model is doing forward inference to predict defects in operation…..


Introducing bayesian nets in agenarisk an example based on software defect prediction

..and backwards inference to make deductions about design process quality.


Introducing bayesian nets in agenarisk an example based on software defect prediction

but actually the most likely explanation is very low testing quality.


Introducing bayesian nets in agenarisk an example based on software defect prediction

…and lower than average complexity.


Introducing bayesian nets in agenarisk an example based on software defect prediction

But if we find out that the complexity is actually high…..


Introducing bayesian nets in agenarisk an example based on software defect prediction

https://intranet.dcs.qmul.ac.uk/courses/coursenotes/DCS235/

then the expected number of operational defects increases


Introducing bayesian nets in agenarisk an example based on software defect prediction

https://intranet.dcs.qmul.ac.uk/courses/coursenotes/DCS235/

and we become even more convinced of the inadequate testing


Introducing bayesian nets in agenarisk an example based on software defect prediction

https://intranet.dcs.qmul.ac.uk/courses/coursenotes/DCS235/

So far we have made no observation about operational usage.


Introducing bayesian nets in agenarisk an example based on software defect prediction

https://intranet.dcs.qmul.ac.uk/courses/coursenotes/DCS235/

If, in fact, the operational usage is high…


Introducing bayesian nets in agenarisk an example based on software defect prediction

Then we have an example of a component with no defects in test ..


Introducing bayesian nets in agenarisk an example based on software defect prediction

…but probably many defects in operation.


Introducing bayesian nets in agenarisk an example based on software defect prediction

But suppose we find out that the test quality was very high.


Introducing bayesian nets in agenarisk an example based on software defect prediction

Then we completely revise out beliefs


Introducing bayesian nets in agenarisk an example based on software defect prediction

We are now pretty convinced that the module will be fault free in operation


Introducing bayesian nets in agenarisk an example based on software defect prediction

…And the ‘explanation’ is that the design process is likely to be very high quality


A model in action6

A Model in action

we reset the model and this time use the model to argue backwards


A model in action7

A Model in action

Suppose we know that this is a critical component that has a requirement for 0 defects in operation…


Introducing bayesian nets in agenarisk an example based on software defect prediction

The model looks for explanations for such a state of affairs.


Introducing bayesian nets in agenarisk an example based on software defect prediction

The most obvious way to achieve such a result is to not use the component much.


Introducing bayesian nets in agenarisk an example based on software defect prediction

But if we know it will be subject to high usage…


Introducing bayesian nets in agenarisk an example based on software defect prediction

Then the model adjusts the beliefs about the other uncertain variables.


Introducing bayesian nets in agenarisk an example based on software defect prediction

A combination of lower than average complexity…..


Introducing bayesian nets in agenarisk an example based on software defect prediction

…Higher than average design quality…..


Introducing bayesian nets in agenarisk an example based on software defect prediction

and much higher than average testing quality …..


Introducing bayesian nets in agenarisk an example based on software defect prediction

But suppose we cannot assume our testing is anything other than average…


Introducing bayesian nets in agenarisk an example based on software defect prediction

Then better design quality …..


Introducing bayesian nets in agenarisk an example based on software defect prediction

..and lower complexity are needed …..


Introducing bayesian nets in agenarisk an example based on software defect prediction

But if complexity is very high …..


Introducing bayesian nets in agenarisk an example based on software defect prediction

…Then we are left with a very skewed distribution for design process quality.


Introducing bayesian nets in agenarisk an example based on software defect prediction

What the model is saying is that, if these are the true requirements for the component then you are very unlikely to achieve them unless you have a very good design process


Making better decisions

Making better decisions

  • That was a simplified version of model produced for Philips

  • Helped Philips make critical decisions about when to release software for electronic components

  • 95% accuracy in defect prediction – much better than can be achieved by traditional statistical methods


Model implementation

Model Implementation

In AgenaRisk

www.agenarisk.com


  • Login