1 / 12

Bayesian Graphical Models for Software Testing David A Wooff, Michael Goldstein, Frank P.A. Coolen

Bayesian Graphical Models for Software Testing David A Wooff, Michael Goldstein, Frank P.A. Coolen. Presented By Scott Young. Applying Detailed Metrics to Testing. Can provide insight for performing risk analysis

orinda
Download Presentation

Bayesian Graphical Models for Software Testing David A Wooff, Michael Goldstein, Frank P.A. Coolen

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Bayesian Graphical Models for Software TestingDavid A Wooff, Michael Goldstein, Frank P.A. Coolen Presented By Scott Young

  2. Applying Detailed Metrics to Testing • Can provide insight for performing risk analysis • Can provide concrete values to inform the customer or management, concerning software quality • May result in more efficient testing procedures

  3. Meaningful Metrics • Simple metrics concerning testing can inform concerning the success of the testing process • They also can provide insight for when the test process is reaching completion • More complex testing analysis needs to be done in order to locate points of inefficiency in testing, as well as providing more fine-grained knowledge about test results

  4. What is a Bayesian Graphical Model? • Also commonly called a Bayesian Belief Network • Is a directed graph, with nodes signifying indeterminate factors • Bayesian models are most commonly heard of today in relation to email spam filtering • They are used to calculate probability based on pre-defined knowledge and relationships between components

  5. Software Actions (SA’s) • A Software Action is an individual, fine grained component of the software project which accomplishes a single task. • An example of a software action in a system would be the processing of a credit card number.

  6. Specifying Nodes • A node should be a collection of operations with the same prior probability of failure, as well as the same change in probability of failure given a test covering that set of operations.

  7. Factors For Defining Probabilities • Level of code complexity • Reliability comparison with existing code which has been evaluated • Maturity of codebase • Typical reliability of author’s code • Similarities to existing code

  8. Updating The Model • As testing continues, the model must be update in stepwise fashion to follow changes to components as they occur. • The probability of an individual node can be updated according to multiple criteria (which are necessarily assumptions) about remaining defects.

  9. What The Results Provide • Tests should be arranged according to the software action(s) which they provide coverage for. Tests discovered to be redundant may be safely removed • Results demonstrate the perceived probability (or strength of belief/confidence) that there are no more existing faults within each SA

  10. What Does This Mean For V&V? • Software producers can demand a level of confidence for components from their testing according to the role of the software and potential financial impact of defects in specific components.

  11. Drawbacks • Informed knowledge is required in order to build a reliable model • This “informed knowledge” still consists of assumptions of relationships (though an assumption within an order of magnitude can still provide useful results) • The amount of additional work to formally track every SA may be prohibitive

  12. Resources • David A Wooff, Michael Goldstein, Frank P.A. Coolen, “Bayesian Graphical Models for Software Testing”. IEEE Transactions on Software Engineering, May 2002. • Murray Cumming, “Bayesian Belief Networks”. http://www.murrayc.com/learning/AI/bbn.shtml Date unknown. • Kevin Murphy, “A brief introduction to Bayes’ Rule”. http://www.ai.mit.edu/~murphyk/Bayes/bayesrule.html, Jan 2004.

More Related