Using logic criterion feasibility to reduce test set size while guaranteeing fault detection
Download
1 / 22

Using Logic Criterion Feasibility to Reduce Test Set Size While Guaranteeing Fault Detection - PowerPoint PPT Presentation


  • 220 Views
  • Uploaded on

Using Logic Criterion Feasibility to Reduce Test Set Size While Guaranteeing Fault Detection . Gary Kaminski and Paul Ammann ICST 2009 March 24 Version. Motivation. Current logic criteria: generate large test sets (Combinatorial) or do not guarantee detecting logic faults (RACC) Goal:

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Using Logic Criterion Feasibility to Reduce Test Set Size While Guaranteeing Fault Detection' - roy


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Using logic criterion feasibility to reduce test set size while guaranteeing fault detection l.jpg

Using Logic Criterion Feasibility to Reduce Test Set Size While Guaranteeing Fault Detection

Gary Kaminski and Paul Ammann

ICST 2009

March 24 Version


Motivation l.jpg
Motivation While Guaranteeing Fault Detection

Current logic criteria:

  • generate large test sets (Combinatorial)

    or

  • do not guarantee detecting logic faults (RACC)

    Goal:

    - generate smaller test sets while still

    guaranteeing fault detection

    Assumption:

    - restrict attention to minimal Disjunctive Normal Form (DNF) Boolean predicates tested in isolation


A word about infeasibility l.jpg
A Word About Infeasibility While Guaranteeing Fault Detection

  • Infeasible Test Requirements are a hassle!

    • They can bloat test sets

    • They can thwart subsumption hierarchies

    • Example: RACC and CACC

      • May be infeasible to satisfy RACC, but feasible to satisfy CACC

      • RACC subsumes CACC, yet for a literal in a predicate, CACC may yield a test case when RACC does not

  • Coverage Criteria for Detecting Logic Faults

    • If all test requirements feasible, simple criteria are enough

    • More complex criteria needed to fill in the gaps

    • This paper analyzes feasibility at a “low” level

    • Result: Minimal, fault-detecting test sets


Building test sets guaranteed to detect faults current l.jpg
Building Test Sets Guaranteed to Detect Faults (Current) While Guaranteeing Fault Detection

Apply

Criterion 1

T1

Apply

Criterion 2

Predicate P:

ab + a!c

Test Set =

T1 + T2 + T3

T2

Apply

Criterion 3

T3

  • Apply Each Criterion to P, Component by Component

  • If criterion feasible on component, generate test

  • If criterion infeasible on component, satisfy as much as possible

  • Result: Tests from all Criteria on all Components

  • Criteria are all necessary; but individual tests may be unnecessary


Analyzing criterion feasibility at component level l.jpg
Analyzing Criterion Feasibility at Component Level While Guaranteeing Fault Detection

Extract

Components

Criterion 1 Feasible?

Yes

Apply

Criterion 1

T1’

No

Criterion 2 Feasible?

Yes

Test Set =

T1’ + T2’ + T3’

Apply

Criterion 2

T2’

No

Predicate P:

ab + a!c

Apply

Criterion 3

T3’

  • Criterion Feasibility Analyzed, Component by Component

  • If criterion feasible on component, generate test and FINISH

  • If criterion infeasible on component, partially satisfy and go to next criterion

Result: Every resulting test has a reason for being there

Note: Some details glossed over in this figure…


Minimal dnf l.jpg
Minimal DNF While Guaranteeing Fault Detection

  • Terms separated by OR, literals by AND

    ab + a!c vs. a(b + !c)

  • Make each term true and other terms false

    ab + ac vs. ab + abc

  • Impossible to remove a literal or term without changing the predicate

    ab vs. abc + ab!c


Minimal dnf logic faults l.jpg
Minimal DNF Logic Faults While Guaranteeing Fault Detection

Original: ab + bc

  • Literal Insertion Fault: abc + b!c

  • Literal Insertion Fault: ab!c + b!c

  • Literal Reference Fault: ac + b!c

  • Literal Reference Fault: a!c + b!c

  • Literal Omission Fault: b + b!c

    A test set detecting these faults also detects others


Lau and yu s fault hierarchy l.jpg

LIF While Guaranteeing Fault Detection

LOF

LRF

TOF

LNF

ORF.

ORF+

TNF

ENF

Lau and Yu’s Fault Hierarchy

  • A test set that guarantees detection of a source fault guarantees detection of a destination fault

  • Ignores effect of criterion feasibility


Unique true points and near false points l.jpg
Unique True Points and While Guaranteeing Fault Detection Near False Points

  • UTP: An assignment of values such that only one term evaluates to true.

    ab + !ac: 110 and 111 are UTPs for ab

  • NFP: An assignment of values such that the predicate evaluates to false but when a literal is omitted, it evaluates to true.

    ab + !ac: 100 and 101 are NFPs for b


Mutp criterion l.jpg
MUTP Criterion While Guaranteeing Fault Detection

  • Find UTP tests for each term such that all literals not in the term attain 0 and 1.

  • Detects LIF and if feasible, detects LRF

  • Inexpensive to satisfy

  • Feasible for ab + !ac

    ab – 110, 111

    !ac – 001, 011

  • Infeasible for ab + ac

    ab – 110


Cutpnfp criterion l.jpg
CUTPNFP Criterion While Guaranteeing Fault Detection

  • Find a UTP - NFP pair such that only the literal of interest changes value.

  • Detects LOF and if feasible, detects LRF

  • More expensive to satisfy

  • Feasible for b in ab + ac

    UTP for ab is 110

    NFP for b in ab is 100

  • Infeasible for b in first term of ab + b!c + !bc

    UTP for ab is 111

    NFP for b in ab 100 (101 makes !bc true)


Mnfp criterion l.jpg
MNFP Criterion While Guaranteeing Fault Detection

  • Find NFP tests for each literal such that all literals not in the term attain 0 and 1.

  • Detects LOF and if feasible, detects LRF

  • Most expensive to satisfy

  • Feasible for a in first term of ab + ac

    010, 011

  • Infeasible for a in first term of ab + !ac

    010 (011 makes !ac true)


Mumcut criterion l.jpg
MUMCUT Criterion While Guaranteeing Fault Detection

  • Combine CUTPNFP, MNFP, and MUTP

    - detects LIF, LRF, and LOF but expensive

    - without considering feasibility need all 3 criteria to detect LRF

  • Other criteria require less inputs but do not guarantee fault detection (RACC)

  • Can we reduce MUMCUT test set size while still guaranteeing LRF detection?


Mutp feasibility and lrf l.jpg

For Each Term While Guaranteeing Fault Detection

MUTP

feasible?

Test Set =

MUTP + NFP

MUTP Feasibility and LRF

If MUTP is feasible for a term: Black – Green

  • MUTP detects LRF

  • CUTPNFP not needed to detect LRF

  • MNFP not needed to detect LRF

MNFP

CUTPNFP

feasible?

For Each Literal In Term

Test Set =

MUTP + MNFP

Test Set =

MUTP + CUTPNFP


Cutpnfp feasibility and lrf l.jpg

MNFP While Guaranteeing Fault Detection

CUTPNFP

feasible?

For Each Literal In Term

For Each Term

MUTP

feasible?

Test Set =

MUTP + MNFP

Test Set =

MUTP + NFP

Test Set =

MUTP + CUTPNFP

CUTPNFP Feasibility and LRF

If MUTP is infeasible for a term but CUTPNFP is feasible for a literal in the term: Black – Red – Black - Green

  • MUTP does not detect LRF

  • CUTPNFP detects LRF

  • MNFP not needed to detect LRF


Mnfp feasibility and lrf l.jpg

MNFP While Guaranteeing Fault Detection

CUTPNFP

feasible?

For Each Literal In Term

For Each Term

MUTP

feasible?

Test Set =

MUTP + MNFP

Test Set =

MUTP + NFP

Test Set =

MUTP + CUTPNFP

MNFP Feasibility and LRF

If MUTP is infeasible for a term and CUTPNFP is infeasible for a literal in the term: Black – Red – Black – Red – Black

  • MUTP does not detect LRF

  • CUTPNFP does not detect LRF

  • MNFP will detect LRF


Minimal mumcut criterion l.jpg
Minimal-MUMCUT Criterion While Guaranteeing Fault Detection

  • MUTP feasible  MUTP detects LRF

  • CUTPNFP feasible  CUTPNFP detects LRF

  • Both infeasible  MNFP detects LRF

    Minimal-MUMCUT:

  • Always need MUTP tests to detect LIF

  • CUTPNFP tests only when MUTP infeasible

  • MNFP tests only when both are infeasible

    “Minimal” means that every test in the test set is needed to

    guarantee fault detection – not minimized


New fault hierarchy l.jpg

LIF While Guaranteeing Fault Detection

LRF

TOF

LOF

LNF

ORF.

ORF+

TNF

ENF

New Fault Hierarchy

  • Black arrow: relation always holds

  • Green arrow: relation holds if MUTP is feasible

  • Red arrow: relation holds if MUTP is infeasible and CUTPNFP is feasible


Case study l.jpg
Case Study While Guaranteeing Fault Detection

  • Analyzed 19 Boolean predicates in an avionics software system (Weyuker, Chen, Lau, and Yu)

  • Number of unique literals range: 5 to 13

  • Determined MUTP feasibility for each term and CUTPNFP feasibility for each literal

  • Examined test set size for MUMCUT vs. Minimal-MUMCUT


Case study results l.jpg
Case Study Results While Guaranteeing Fault Detection

  • Minimal-MUMCUT size is 12% of MUMCUT size

  • Savings in test set size comes from

    1) CUTPNFP feasible for all 853 literals: no MNFP

    2) For 24% of literals, MUTP detects LRF: no CUTPNFP

    3) 16 of 19 predicates had a MUTP feasible term


Test set size vs number of unique literals l.jpg
Test Set Size vs. While Guaranteeing Fault Detection Number of Unique Literals


Conclusion l.jpg
Conclusion While Guaranteeing Fault Detection

  • Used criterion feasibility to reduce test set size without sacrificing fault detection

  • Modification of fault detection relations in Lau and Yu’s hierarchy based on criterion feasibility

  • Introduction of the Minimal-MUMCUT criterion based on minimal DNF

  • Applications for software testing of programs with large predicates


ad