using logic criterion feasibility to reduce test set size while guaranteeing fault detection l.
Download
Skip this Video
Download Presentation
Using Logic Criterion Feasibility to Reduce Test Set Size While Guaranteeing Fault Detection

Loading in 2 Seconds...

play fullscreen
1 / 22

Using Logic Criterion Feasibility to Reduce Test Set Size While Guaranteeing Fault Detection - PowerPoint PPT Presentation


  • 223 Views
  • Uploaded on

Using Logic Criterion Feasibility to Reduce Test Set Size While Guaranteeing Fault Detection . Gary Kaminski and Paul Ammann ICST 2009 March 24 Version. Motivation. Current logic criteria: generate large test sets (Combinatorial) or do not guarantee detecting logic faults (RACC) Goal:

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Using Logic Criterion Feasibility to Reduce Test Set Size While Guaranteeing Fault Detection' - roy


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
using logic criterion feasibility to reduce test set size while guaranteeing fault detection

Using Logic Criterion Feasibility to Reduce Test Set Size While Guaranteeing Fault Detection

Gary Kaminski and Paul Ammann

ICST 2009

March 24 Version

motivation
Motivation

Current logic criteria:

  • generate large test sets (Combinatorial)

or

  • do not guarantee detecting logic faults (RACC)

Goal:

- generate smaller test sets while still

guaranteeing fault detection

Assumption:

- restrict attention to minimal Disjunctive Normal Form (DNF) Boolean predicates tested in isolation

a word about infeasibility
A Word About Infeasibility
  • Infeasible Test Requirements are a hassle!
    • They can bloat test sets
    • They can thwart subsumption hierarchies
    • Example: RACC and CACC
      • May be infeasible to satisfy RACC, but feasible to satisfy CACC
      • RACC subsumes CACC, yet for a literal in a predicate, CACC may yield a test case when RACC does not
  • Coverage Criteria for Detecting Logic Faults
    • If all test requirements feasible, simple criteria are enough
    • More complex criteria needed to fill in the gaps
    • This paper analyzes feasibility at a “low” level
    • Result: Minimal, fault-detecting test sets
building test sets guaranteed to detect faults current
Building Test Sets Guaranteed to Detect Faults (Current)

Apply

Criterion 1

T1

Apply

Criterion 2

Predicate P:

ab + a!c

Test Set =

T1 + T2 + T3

T2

Apply

Criterion 3

T3

  • Apply Each Criterion to P, Component by Component
  • If criterion feasible on component, generate test
  • If criterion infeasible on component, satisfy as much as possible
  • Result: Tests from all Criteria on all Components
  • Criteria are all necessary; but individual tests may be unnecessary
analyzing criterion feasibility at component level
Analyzing Criterion Feasibility at Component Level

Extract

Components

Criterion 1 Feasible?

Yes

Apply

Criterion 1

T1’

No

Criterion 2 Feasible?

Yes

Test Set =

T1’ + T2’ + T3’

Apply

Criterion 2

T2’

No

Predicate P:

ab + a!c

Apply

Criterion 3

T3’

  • Criterion Feasibility Analyzed, Component by Component
  • If criterion feasible on component, generate test and FINISH
  • If criterion infeasible on component, partially satisfy and go to next criterion

Result: Every resulting test has a reason for being there

Note: Some details glossed over in this figure…

minimal dnf
Minimal DNF
  • Terms separated by OR, literals by AND

ab + a!c vs. a(b + !c)

  • Make each term true and other terms false

ab + ac vs. ab + abc

  • Impossible to remove a literal or term without changing the predicate

ab vs. abc + ab!c

minimal dnf logic faults
Minimal DNF Logic Faults

Original: ab + bc

  • Literal Insertion Fault: abc + b!c
  • Literal Insertion Fault: ab!c + b!c
  • Literal Reference Fault: ac + b!c
  • Literal Reference Fault: a!c + b!c
  • Literal Omission Fault: b + b!c

A test set detecting these faults also detects others

lau and yu s fault hierarchy

LIF

LOF

LRF

TOF

LNF

ORF.

ORF+

TNF

ENF

Lau and Yu’s Fault Hierarchy
  • A test set that guarantees detection of a source fault guarantees detection of a destination fault
  • Ignores effect of criterion feasibility
unique true points and near false points
Unique True Points and Near False Points
  • UTP: An assignment of values such that only one term evaluates to true.

ab + !ac: 110 and 111 are UTPs for ab

  • NFP: An assignment of values such that the predicate evaluates to false but when a literal is omitted, it evaluates to true.

ab + !ac: 100 and 101 are NFPs for b

mutp criterion
MUTP Criterion
  • Find UTP tests for each term such that all literals not in the term attain 0 and 1.
  • Detects LIF and if feasible, detects LRF
  • Inexpensive to satisfy
  • Feasible for ab + !ac

ab – 110, 111

!ac – 001, 011

  • Infeasible for ab + ac

ab – 110

cutpnfp criterion
CUTPNFP Criterion
  • Find a UTP - NFP pair such that only the literal of interest changes value.
  • Detects LOF and if feasible, detects LRF
  • More expensive to satisfy
  • Feasible for b in ab + ac

UTP for ab is 110

NFP for b in ab is 100

  • Infeasible for b in first term of ab + b!c + !bc

UTP for ab is 111

NFP for b in ab 100 (101 makes !bc true)

mnfp criterion
MNFP Criterion
  • Find NFP tests for each literal such that all literals not in the term attain 0 and 1.
  • Detects LOF and if feasible, detects LRF
  • Most expensive to satisfy
  • Feasible for a in first term of ab + ac

010, 011

  • Infeasible for a in first term of ab + !ac

010 (011 makes !ac true)

mumcut criterion
MUMCUT Criterion
  • Combine CUTPNFP, MNFP, and MUTP

- detects LIF, LRF, and LOF but expensive

- without considering feasibility need all 3 criteria to detect LRF

  • Other criteria require less inputs but do not guarantee fault detection (RACC)
  • Can we reduce MUMCUT test set size while still guaranteeing LRF detection?
mutp feasibility and lrf

For Each Term

MUTP

feasible?

Test Set =

MUTP + NFP

MUTP Feasibility and LRF

If MUTP is feasible for a term: Black – Green

  • MUTP detects LRF
  • CUTPNFP not needed to detect LRF
  • MNFP not needed to detect LRF

MNFP

CUTPNFP

feasible?

For Each Literal In Term

Test Set =

MUTP + MNFP

Test Set =

MUTP + CUTPNFP

cutpnfp feasibility and lrf

MNFP

CUTPNFP

feasible?

For Each Literal In Term

For Each Term

MUTP

feasible?

Test Set =

MUTP + MNFP

Test Set =

MUTP + NFP

Test Set =

MUTP + CUTPNFP

CUTPNFP Feasibility and LRF

If MUTP is infeasible for a term but CUTPNFP is feasible for a literal in the term: Black – Red – Black - Green

  • MUTP does not detect LRF
  • CUTPNFP detects LRF
  • MNFP not needed to detect LRF
mnfp feasibility and lrf

MNFP

CUTPNFP

feasible?

For Each Literal In Term

For Each Term

MUTP

feasible?

Test Set =

MUTP + MNFP

Test Set =

MUTP + NFP

Test Set =

MUTP + CUTPNFP

MNFP Feasibility and LRF

If MUTP is infeasible for a term and CUTPNFP is infeasible for a literal in the term: Black – Red – Black – Red – Black

  • MUTP does not detect LRF
  • CUTPNFP does not detect LRF
  • MNFP will detect LRF
minimal mumcut criterion
Minimal-MUMCUT Criterion
  • MUTP feasible  MUTP detects LRF
  • CUTPNFP feasible  CUTPNFP detects LRF
  • Both infeasible  MNFP detects LRF

Minimal-MUMCUT:

  • Always need MUTP tests to detect LIF
  • CUTPNFP tests only when MUTP infeasible
  • MNFP tests only when both are infeasible

“Minimal” means that every test in the test set is needed to

guarantee fault detection – not minimized

new fault hierarchy

LIF

LRF

TOF

LOF

LNF

ORF.

ORF+

TNF

ENF

New Fault Hierarchy
  • Black arrow: relation always holds
  • Green arrow: relation holds if MUTP is feasible
  • Red arrow: relation holds if MUTP is infeasible and CUTPNFP is feasible
case study
Case Study
  • Analyzed 19 Boolean predicates in an avionics software system (Weyuker, Chen, Lau, and Yu)
  • Number of unique literals range: 5 to 13
  • Determined MUTP feasibility for each term and CUTPNFP feasibility for each literal
  • Examined test set size for MUMCUT vs. Minimal-MUMCUT
case study results
Case Study Results
  • Minimal-MUMCUT size is 12% of MUMCUT size
  • Savings in test set size comes from

1) CUTPNFP feasible for all 853 literals: no MNFP

2) For 24% of literals, MUTP detects LRF: no CUTPNFP

3) 16 of 19 predicates had a MUTP feasible term

conclusion
Conclusion
  • Used criterion feasibility to reduce test set size without sacrificing fault detection
  • Modification of fault detection relations in Lau and Yu’s hierarchy based on criterion feasibility
  • Introduction of the Minimal-MUMCUT criterion based on minimal DNF
  • Applications for software testing of programs with large predicates