Property based test generation li tan oleg sokolsky and insup lee university of pennsylvania
This presentation is the property of its rightful owner.
Sponsored Links
1 / 12

Property-Based Test Generation Li Tan, Oleg Sokolsky, and Insup Lee University of Pennsylvania PowerPoint PPT Presentation


  • 97 Views
  • Uploaded on
  • Presentation posted in: General

Property-Based Test Generation Li Tan, Oleg Sokolsky, and Insup Lee University of Pennsylvania. The Overview of Our Approach. LTL formulae F 1 , …, F n. Quasi linear (Lasso-shape) proof structure. Feature conflict detection. Model Checker. Test trace generator. Behavior specification.

Download Presentation

Property-Based Test Generation Li Tan, Oleg Sokolsky, and Insup Lee University of Pennsylvania

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Property based test generation li tan oleg sokolsky and insup lee university of pennsylvania

Property-Based Test GenerationLi Tan, Oleg Sokolsky, and Insup LeeUniversity of Pennsylvania


Property based test generation li tan oleg sokolsky and insup lee university of pennsylvania

The Overview of Our Approach

LTL formulae

F1, …, Fn

Quasi linear (Lasso-shape) proof structure

Feature conflict

detection

Model Checker

Test trace

generator

Behavior specification

LTL formulae

F1, …, Fn

Temporal Property

Translator

Traces {r1,…,rn}

Criteria not being covered

Specification Model

(CHARON)

Simulation-based

randomized

test generator

Traces

Coverage

Criteria

Testing result

System Modeling

Environ. Modeling

Interface Definition

Informal

System Specification

Hardware

Specification/limitation

Test Harness

Implementation

Test result


Property based test generation li tan oleg sokolsky and insup lee university of pennsylvania

Goal: Using model-checking technique to make test generation more efficient, flexible, and centered on the system-specific properties (features).

Step I. Preparing specifications

  • Properties (feature specification) as linear temporal logic formula

  • (optional) System specification system as CHARON (for hybrid systems) and EFSM (for discrete systems)

    Step II. Test generation using model checkers.

  • (For hybrid systems) Simulation-based test generation with the assistance of predicate abstraction reachability analysis.

  • (For Discrete system)

    • (Option A) Using the proof structures of evidence-ready model checkers.

    • (Option B) Reducing the test generation for LTL formula to safety check

  • (For temporal specification only) Functional test.

    • Generating non-trivial test traces for temporal specification (feature specification)

    • Detecting conflicting in temporal specification.

      Step III: Realizing test harness.

Model-checking based

Test generator

Temporal (feature) Spec.

+

Model (optional)

Test suite

(Finite set of finite traces)


From property and model to test suite property based test generation

I. From infinite length to finite: Synthesizing test suites for 9LTL property

From Property and Model to Test suite: Property-based test generation


Property based test generation li tan oleg sokolsky and insup lee university of pennsylvania

A infinite Lasso-shaped test suite can be checked adequately by finite steps if the implementation is bounded.

Turn=1, :c1, : c2

Turn=1, c1, : c2

+

Turn=2, c1, : c2

Turn=2, c1, c2

Turn=1, c1, c2

Estimating the number of relevant implementation states using slicing

Turn=1, c1, c2

A quasi-linear proof skeleton

A finite test suite


Property based test generation li tan oleg sokolsky and insup lee university of pennsylvania

Test Generation using Model Checkers

Option A: Modifying model checkers and retaining proofs.

Option B: Using the idea of reducing LTL model checking to reachability analysis [A. Biere etc], but enhancing the observer to retain proof

SMV model

SMV model

+

Extended

Observer Model

+

Repetition information

Linear Temporal

Logic Specification

Extracted Proof

Generated test trace


Property based test generation li tan oleg sokolsky and insup lee university of pennsylvania

II. From infinite numbers of traces to finite: selecting interesting traces

System properties are required to be held on all the paths, we will select only nontrivial paths, whose characteristics are caught by ELTL formula systematically deriving from the properties.

LTL f => ELTL formulae a2e(f)={f(f’ !ð(f’))|f’ Áf}

F= G(req -> F(cancelÇresponse))

FÆ (: G(req ! F(cancelÇ false)))

= FÆ F(req Æ G(: cancel))

Test the case that no cancel follows a request

(hence a reponse must be placed)

FÆ (: G(req ! F(false Ç response)))

= FÆ F(req Æ G(: response))

Test the case that no response follows a request

(hence a cancel must be placed)

FÆ (: G(true ! F(cancel Ç response)))

= FÆ FG(: (cancel Ç response))

Test the case that no cancel or reponses occurs after time t, (hence should not a request occur).

FÆ (: G(req ! false))

= FÆ F(req)

Test the case that there is request


From only property to test suite functional test generation

So, what if only behavior (feature) specification is available ……

From only Property to Test suite: Functional test generation

LTL formulae

F

Nontrivial ELTL formulae

Derived from F

Y=a2e(F)

f02FÆ 2 Y

f12FÆ 2 Y

fn2FÆ 2 Y

…….

Buchi automaton B0

Buchi automaton B1

Buchi automaton Bn

Check nonemptiness

Check nonemptiness

A trace satisfies f0

A trace satisfies f1

A trace satisfies fn


Property based test generation li tan oleg sokolsky and insup lee university of pennsylvania

Testing Hybrid system: simulation-based test generator with predicate-abstract reachability analysis

System

Modeling

Coverage

Criteria

No

Bad set

Reachability

Checker

CHARON

(Model)

Flatten hybrid model

Predicate set

NO w/ more predicates

Simulation

/refinement

Yes w/ Trace

YES

Test Suite

Concretize

Implementation


Property based test generation li tan oleg sokolsky and insup lee university of pennsylvania

An implementation of simulation-based test generator

a. CHARON simulator with test generator

b. Progress report of test generator

c. Visual display of generated test traces.


Property based test generation li tan oleg sokolsky and insup lee university of pennsylvania

Charon model

for CARA

Charon model

For patient

Closed Charon Model for CARA

Test harness as

I/O Interface

CARA simulator

/model-generated code

Standalone executable program

Realizing Test Harness

Test trace

Variable back_EMF

Value Time

60.0 0.001

70.0 0.002

……

Simulation-based

test generator

Coverage criteriae

Test Result


Property based test generation li tan oleg sokolsky and insup lee university of pennsylvania

Conclusion

  • Applying model-checking technique to traditional domain of test generation is appealing.

    • Test generation is centralized on system-specific properties

    • State-of-art model checkers may be adapted as general purpose test generator (and think properties as programs ).

    • Techniques in model checking may help find interesting test traces and provide new angle to view and think test generation.

  • Property-based test generation requires integrated efforts.

    • Test generation ¹ witness generation.

      • Proof is necessary to generate partial test suite and perform optimization.

      • Proof is also needed to extend the notion of “testable” properties.

    • Model-based code generation may help build test harness.


  • Login