Property based test generation li tan oleg sokolsky and insup lee university of pennsylvania
This presentation is the property of its rightful owner.
Sponsored Links
1 / 12

Property-Based Test Generation Li Tan, Oleg Sokolsky, and Insup Lee University of Pennsylvania PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

Property-Based Test Generation Li Tan, Oleg Sokolsky, and Insup Lee University of Pennsylvania. The Overview of Our Approach. LTL formulae F 1 , …, F n. Quasi linear (Lasso-shape) proof structure. Feature conflict detection. Model Checker. Test trace generator. Behavior specification.

Download Presentation

Property-Based Test Generation Li Tan, Oleg Sokolsky, and Insup Lee University of Pennsylvania

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Property-Based Test GenerationLi Tan, Oleg Sokolsky, and Insup LeeUniversity of Pennsylvania

The Overview of Our Approach

LTL formulae

F1, …, Fn

Quasi linear (Lasso-shape) proof structure

Feature conflict


Model Checker

Test trace


Behavior specification

LTL formulae

F1, …, Fn

Temporal Property


Traces {r1,…,rn}

Criteria not being covered

Specification Model




test generator




Testing result

System Modeling

Environ. Modeling

Interface Definition


System Specification



Test Harness


Test result

Goal: Using model-checking technique to make test generation more efficient, flexible, and centered on the system-specific properties (features).

Step I. Preparing specifications

  • Properties (feature specification) as linear temporal logic formula

  • (optional) System specification system as CHARON (for hybrid systems) and EFSM (for discrete systems)

    Step II. Test generation using model checkers.

  • (For hybrid systems) Simulation-based test generation with the assistance of predicate abstraction reachability analysis.

  • (For Discrete system)

    • (Option A) Using the proof structures of evidence-ready model checkers.

    • (Option B) Reducing the test generation for LTL formula to safety check

  • (For temporal specification only) Functional test.

    • Generating non-trivial test traces for temporal specification (feature specification)

    • Detecting conflicting in temporal specification.

      Step III: Realizing test harness.

Model-checking based

Test generator

Temporal (feature) Spec.


Model (optional)

Test suite

(Finite set of finite traces)

I. From infinite length to finite: Synthesizing test suites for 9LTL property

From Property and Model to Test suite: Property-based test generation

A infinite Lasso-shaped test suite can be checked adequately by finite steps if the implementation is bounded.

Turn=1, :c1, : c2

Turn=1, c1, : c2


Turn=2, c1, : c2

Turn=2, c1, c2

Turn=1, c1, c2

Estimating the number of relevant implementation states using slicing

Turn=1, c1, c2

A quasi-linear proof skeleton

A finite test suite

Test Generation using Model Checkers

Option A: Modifying model checkers and retaining proofs.

Option B: Using the idea of reducing LTL model checking to reachability analysis [A. Biere etc], but enhancing the observer to retain proof

SMV model

SMV model



Observer Model


Repetition information

Linear Temporal

Logic Specification

Extracted Proof

Generated test trace

II. From infinite numbers of traces to finite: selecting interesting traces

System properties are required to be held on all the paths, we will select only nontrivial paths, whose characteristics are caught by ELTL formula systematically deriving from the properties.

LTL f => ELTL formulae a2e(f)={f(f’ !ð(f’))|f’ Áf}

F= G(req -> F(cancelÇresponse))

FÆ (: G(req ! F(cancelÇ false)))

= FÆ F(req Æ G(: cancel))

Test the case that no cancel follows a request

(hence a reponse must be placed)

FÆ (: G(req ! F(false Ç response)))

= FÆ F(req Æ G(: response))

Test the case that no response follows a request

(hence a cancel must be placed)

FÆ (: G(true ! F(cancel Ç response)))

= FÆ FG(: (cancel Ç response))

Test the case that no cancel or reponses occurs after time t, (hence should not a request occur).

FÆ (: G(req ! false))

= FÆ F(req)

Test the case that there is request

So, what if only behavior (feature) specification is available ……

From only Property to Test suite: Functional test generation

LTL formulae


Nontrivial ELTL formulae

Derived from F


f02FÆ 2 Y

f12FÆ 2 Y

fn2FÆ 2 Y


Buchi automaton B0

Buchi automaton B1

Buchi automaton Bn

Check nonemptiness

Check nonemptiness

A trace satisfies f0

A trace satisfies f1

A trace satisfies fn

Testing Hybrid system: simulation-based test generator with predicate-abstract reachability analysis






Bad set





Flatten hybrid model

Predicate set

NO w/ more predicates



Yes w/ Trace


Test Suite



An implementation of simulation-based test generator

a. CHARON simulator with test generator

b. Progress report of test generator

c. Visual display of generated test traces.

Charon model

for CARA

Charon model

For patient

Closed Charon Model for CARA

Test harness as

I/O Interface

CARA simulator

/model-generated code

Standalone executable program

Realizing Test Harness

Test trace

Variable back_EMF

Value Time

60.0 0.001

70.0 0.002



test generator

Coverage criteriae

Test Result


  • Applying model-checking technique to traditional domain of test generation is appealing.

    • Test generation is centralized on system-specific properties

    • State-of-art model checkers may be adapted as general purpose test generator (and think properties as programs ).

    • Techniques in model checking may help find interesting test traces and provide new angle to view and think test generation.

  • Property-based test generation requires integrated efforts.

    • Test generation ¹ witness generation.

      • Proof is necessary to generate partial test suite and perform optimization.

      • Proof is also needed to extend the notion of “testable” properties.

    • Model-based code generation may help build test harness.

  • Login