Loading in 5 sec....

Using Logic Criterion Feasibility to Reduce Test Set Size While Guaranteeing Fault Detection PowerPoint Presentation

Using Logic Criterion Feasibility to Reduce Test Set Size While Guaranteeing Fault Detection

- By
**roy** - Follow User

- 220 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about 'Using Logic Criterion Feasibility to Reduce Test Set Size While Guaranteeing Fault Detection' - roy

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

### Using Logic Criterion Feasibility to Reduce Test Set Size While Guaranteeing Fault Detection

Gary Kaminski and Paul Ammann

ICST 2009

March 24 Version

Motivation While Guaranteeing Fault Detection

Current logic criteria:

- generate large test sets (Combinatorial)
or

- do not guarantee detecting logic faults (RACC)
Goal:

- generate smaller test sets while still

guaranteeing fault detection

Assumption:

- restrict attention to minimal Disjunctive Normal Form (DNF) Boolean predicates tested in isolation

A Word About Infeasibility While Guaranteeing Fault Detection

- Infeasible Test Requirements are a hassle!
- They can bloat test sets
- They can thwart subsumption hierarchies
- Example: RACC and CACC
- May be infeasible to satisfy RACC, but feasible to satisfy CACC
- RACC subsumes CACC, yet for a literal in a predicate, CACC may yield a test case when RACC does not

- Coverage Criteria for Detecting Logic Faults
- If all test requirements feasible, simple criteria are enough
- More complex criteria needed to fill in the gaps
- This paper analyzes feasibility at a “low” level
- Result: Minimal, fault-detecting test sets

Building Test Sets Guaranteed to Detect Faults (Current) While Guaranteeing Fault Detection

Apply

Criterion 1

T1

Apply

Criterion 2

Predicate P:

ab + a!c

Test Set =

T1 + T2 + T3

T2

Apply

Criterion 3

T3

- Apply Each Criterion to P, Component by Component
- If criterion feasible on component, generate test
- If criterion infeasible on component, satisfy as much as possible

- Result: Tests from all Criteria on all Components
- Criteria are all necessary; but individual tests may be unnecessary

Analyzing Criterion Feasibility at Component Level While Guaranteeing Fault Detection

Extract

Components

Criterion 1 Feasible?

Yes

Apply

Criterion 1

T1’

No

Criterion 2 Feasible?

Yes

Test Set =

T1’ + T2’ + T3’

Apply

Criterion 2

T2’

No

Predicate P:

ab + a!c

Apply

Criterion 3

T3’

- Criterion Feasibility Analyzed, Component by Component
- If criterion feasible on component, generate test and FINISH
- If criterion infeasible on component, partially satisfy and go to next criterion

Result: Every resulting test has a reason for being there

Note: Some details glossed over in this figure…

Minimal DNF While Guaranteeing Fault Detection

- Terms separated by OR, literals by AND
ab + a!c vs. a(b + !c)

- Make each term true and other terms false
ab + ac vs. ab + abc

- Impossible to remove a literal or term without changing the predicate
ab vs. abc + ab!c

Minimal DNF Logic Faults While Guaranteeing Fault Detection

Original: ab + bc

- Literal Insertion Fault: abc + b!c
- Literal Insertion Fault: ab!c + b!c
- Literal Reference Fault: ac + b!c
- Literal Reference Fault: a!c + b!c
- Literal Omission Fault: b + b!c
A test set detecting these faults also detects others

LIF While Guaranteeing Fault Detection

LOF

LRF

TOF

LNF

ORF.

ORF+

TNF

ENF

Lau and Yu’s Fault Hierarchy- A test set that guarantees detection of a source fault guarantees detection of a destination fault
- Ignores effect of criterion feasibility

Unique True Points and While Guaranteeing Fault Detection Near False Points

- UTP: An assignment of values such that only one term evaluates to true.
ab + !ac: 110 and 111 are UTPs for ab

- NFP: An assignment of values such that the predicate evaluates to false but when a literal is omitted, it evaluates to true.
ab + !ac: 100 and 101 are NFPs for b

MUTP Criterion While Guaranteeing Fault Detection

- Find UTP tests for each term such that all literals not in the term attain 0 and 1.
- Detects LIF and if feasible, detects LRF
- Inexpensive to satisfy
- Feasible for ab + !ac
ab – 110, 111

!ac – 001, 011

- Infeasible for ab + ac
ab – 110

CUTPNFP Criterion While Guaranteeing Fault Detection

- Find a UTP - NFP pair such that only the literal of interest changes value.
- Detects LOF and if feasible, detects LRF
- More expensive to satisfy
- Feasible for b in ab + ac
UTP for ab is 110

NFP for b in ab is 100

- Infeasible for b in first term of ab + b!c + !bc
UTP for ab is 111

NFP for b in ab 100 (101 makes !bc true)

MNFP Criterion While Guaranteeing Fault Detection

- Find NFP tests for each literal such that all literals not in the term attain 0 and 1.
- Detects LOF and if feasible, detects LRF
- Most expensive to satisfy
- Feasible for a in first term of ab + ac
010, 011

- Infeasible for a in first term of ab + !ac
010 (011 makes !ac true)

MUMCUT Criterion While Guaranteeing Fault Detection

- Combine CUTPNFP, MNFP, and MUTP
- detects LIF, LRF, and LOF but expensive

- without considering feasibility need all 3 criteria to detect LRF

- Other criteria require less inputs but do not guarantee fault detection (RACC)
- Can we reduce MUMCUT test set size while still guaranteeing LRF detection?

For Each Term While Guaranteeing Fault Detection

MUTP

feasible?

Test Set =

MUTP + NFP

MUTP Feasibility and LRFIf MUTP is feasible for a term: Black – Green

- MUTP detects LRF
- CUTPNFP not needed to detect LRF
- MNFP not needed to detect LRF

MNFP

CUTPNFP

feasible?

For Each Literal In Term

Test Set =

MUTP + MNFP

Test Set =

MUTP + CUTPNFP

MNFP While Guaranteeing Fault Detection

CUTPNFP

feasible?

For Each Literal In Term

For Each Term

MUTP

feasible?

Test Set =

MUTP + MNFP

Test Set =

MUTP + NFP

Test Set =

MUTP + CUTPNFP

CUTPNFP Feasibility and LRFIf MUTP is infeasible for a term but CUTPNFP is feasible for a literal in the term: Black – Red – Black - Green

- MUTP does not detect LRF
- CUTPNFP detects LRF
- MNFP not needed to detect LRF

MNFP While Guaranteeing Fault Detection

CUTPNFP

feasible?

For Each Literal In Term

For Each Term

MUTP

feasible?

Test Set =

MUTP + MNFP

Test Set =

MUTP + NFP

Test Set =

MUTP + CUTPNFP

MNFP Feasibility and LRFIf MUTP is infeasible for a term and CUTPNFP is infeasible for a literal in the term: Black – Red – Black – Red – Black

- MUTP does not detect LRF
- CUTPNFP does not detect LRF
- MNFP will detect LRF

Minimal-MUMCUT Criterion While Guaranteeing Fault Detection

- MUTP feasible MUTP detects LRF
- CUTPNFP feasible CUTPNFP detects LRF
- Both infeasible MNFP detects LRF
Minimal-MUMCUT:

- Always need MUTP tests to detect LIF
- CUTPNFP tests only when MUTP infeasible
- MNFP tests only when both are infeasible
“Minimal” means that every test in the test set is needed to

guarantee fault detection – not minimized

LIF While Guaranteeing Fault Detection

LRF

TOF

LOF

LNF

ORF.

ORF+

TNF

ENF

New Fault Hierarchy- Black arrow: relation always holds
- Green arrow: relation holds if MUTP is feasible
- Red arrow: relation holds if MUTP is infeasible and CUTPNFP is feasible

Case Study While Guaranteeing Fault Detection

- Analyzed 19 Boolean predicates in an avionics software system (Weyuker, Chen, Lau, and Yu)
- Number of unique literals range: 5 to 13
- Determined MUTP feasibility for each term and CUTPNFP feasibility for each literal
- Examined test set size for MUMCUT vs. Minimal-MUMCUT

Case Study Results While Guaranteeing Fault Detection

- Minimal-MUMCUT size is 12% of MUMCUT size
- Savings in test set size comes from
1) CUTPNFP feasible for all 853 literals: no MNFP

2) For 24% of literals, MUTP detects LRF: no CUTPNFP

3) 16 of 19 predicates had a MUTP feasible term

Test Set Size vs. While Guaranteeing Fault Detection Number of Unique Literals

Conclusion While Guaranteeing Fault Detection

- Used criterion feasibility to reduce test set size without sacrificing fault detection
- Modification of fault detection relations in Lau and Yu’s hierarchy based on criterion feasibility
- Introduction of the Minimal-MUMCUT criterion based on minimal DNF
- Applications for software testing of programs with large predicates

Download Presentation

Connecting to Server..