Towards automatic generation of parameterized test cases from abstractions
This presentation is the property of its rightful owner.
Sponsored Links
1 / 16

Towards Automatic Generation of Parameterized Test Cases from Abstractions PowerPoint PPT Presentation


  • 44 Views
  • Uploaded on
  • Presentation posted in: General

Towards Automatic Generation of Parameterized Test Cases from Abstractions. Jens R. Calamé Natalia Ioustinova Jaco van de Pol Centrum voor Wiskunde en Informatica, Amsterdam. Agenda. Testing Theory Data Abstraction Test Case Parameterization Some Lemmas Conclusion. Testing Theory.

Download Presentation

Towards Automatic Generation of Parameterized Test Cases from Abstractions

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Towards automatic generation of parameterized test cases from abstractions

Towards Automatic Generation of Parameterized Test Casesfrom Abstractions

Jens R. Calamé

Natalia Ioustinova

Jaco van de Pol

Centrum voor Wiskunde en Informatica, Amsterdam


Agenda

Agenda

  • Testing Theory

  • Data Abstraction

  • Test Case Parameterization

  • Some Lemmas

  • Conclusion

IPA Lentedagen 2006


Testing theory

Testing Theory

  • Conformance Testing:

    • Implementation Imp conforms specification Spec, ifffor all traces t in Spec holds: all input should be accepted after t, but at most the specified output should be generated by Imp.

    • Based on Tretmans‘ ioco theory

  • Test Generation by the tool TGV(Test Generation with Verification Techniques)

    • Based on state enumeration

    • Limitation: Data often leads to state-space explosion

    • Solution: Data abstraction mitigates this limitation

IPA Lentedagen 2006


Our test generation process

Our Test Generation Process

System Specification (e.g. UML)

System Specification (formal)

Data Abstraction

Abstract System Specification

Test Case

Generation

TGV

Abstract Test Case

Test Purpose

TTCN-3

Generation

Constraint

Generation

Parameterizable

Test Case (TTCN-3)

Rule System for

Data Selection

IPA Lentedagen 2006


Data abstraction

Data Abstraction

  • Motivation: Input and output data from large (infinite) domains leads to state explosionApplication of state-based test generationtools impossible

  • Solution (from model-checking): data abstraction

    • Introduction of a chaotic value ╥ for each datatype D

    • Lifting functions for original values from D

    • Replacement of input variables by ╥and propagation through the system

    • Introduction of may functions for guards over three-value-logic (semantics: may(╥) = true)

    • Consequence: Introduction of extra traces bynon-determinism

IPA Lentedagen 2006


Data abstraction example

?

init

Pin

(

p

)

?

initBalance

(

b

)

0

1

2

?

getPin

(

x

)

¹

(

x

p

)

!

pinIncorrect

>

3

==

(

x

p

)

>

!

pinCorrect

4

>

(

y

b

)

>

?

getBalance

!

LowBalance

(

b

)

?

getAmount

(

y

)

8

7

5

?

getAmount

(

y

)

£

(

y

b

)

!

Money

(

y

)

;

!

Balance

(

b

)

>

6

b

:=

b

-

y

Data Abstraction – Example

IPA Lentedagen 2006


Data abstraction test case by tgv

Data Abstraction – Test Case by TGV

IPA Lentedagen 2006


Test case parameterization

Test Case Parameterization

  • To be solved after data abstraction:

    • Pruning of traces introduced by overapproximation

    • Finding possible value ranges for test data

  • Rule System (Prolog)

    • Represents SUT specification

    • Defines rules on data (addition, substraction; and, or etc.)

    • Defines rules for process behavior of Spec

  • Query (Prolog)

    • Represents the abstract test case(one rule per trace)

    • All transitions in trace taken as query body

    • Alternative: one query for whole CTG

IPA Lentedagen 2006


Testing with abstraction rule system

?

init

Pin

(

p

)

?

initBalance

(

b

)

0

1

2

?

getPin

(

x

)

¹

(

x

p

)

!

pinIncorrect

>

3

==

(

x

p

)

>

!

pinCorrect

4

>

(

y

b

)

>

?

getBalance

!

LowBalance

(

b

)

?

getAmount

(

y

)

8

7

5

?

getAmount

(

y

)

£

(

y

b

)

!

Money

(

y

)

;

!

Balance

(

b

)

>

6

b

:=

b

-

y

Testing with Abstraction – Rule System

getPin(state(2,P,B,X,Y), state(3,P,B,X1,Y), param(X1)).

pinIncorrect(state(3,P,B,X,Y), state(8,P,B,X,Y),param(_)) :- X ≠ P.

pinCorrect(state(3,P,B,X,Y), state(4,P,B,X,Y), param(_)) :- X = P.

IPA Lentedagen 2006


Testing with abstraction query

Testing with Abstraction – Query

oracle(P,B,X,Yin,Yout) :-

initPin(state(0,0,0,0,0), G1, param(P)),

initBalance(G1, G2, param(B)),

getPin(G2, G3, param(X)),

pinCorrect(G3, G4, _),

getAmount(G4, G5, param(Yin)),

money(G5, _, param(Yout)).

P=P{-1.0inf..1.0inf}

B=B{-1.0inf..1.0inf}

X=P{-1.0inf..1.0inf}

Yin=Yout{-1.0inf..1.0inf}

Yout=Yout{-1.0inf..1.0inf}

Yout{-1.0inf..1.0inf}-B{-1.0inf..1.0inf}=<0

IPA Lentedagen 2006


Test execution

Test Execution

  • Pre-solve one trace to pass statically(skip internal steps)

  • Execute this trace until the SUT leaves it

  • Try to find a trace to pass solve and execute it

  • If no trace to pass:Try to find a trace to inconc  solve it

  • If no trace to inconc:set test verdict fail

!initPin(P)

!initBalance(B)

0

1

2

!getPin(X)

?pinInCorrect

INCONC

3

?pinCorrect

?lowBalance

4

!getAmount(Yin)

!getBalance

PASS

7

5

?Money(Yout)

?Balance(x50)

!getAmount(x60)

?eatCard

6

IPA Lentedagen 2006


Some lemmas

Some Lemmas

  • The original system Spec is simulated by Spec╥ in all details (and some more).

  • The control flow of the synchronous productSpec x TP is simulated by Spec╥ x TP.

  • The set of accepting/refusing traces of Spec x TP is a subset of those of Spec╥ x TP.

  • If the test oracle holds for a trace in the CTG under a certain valuation, this trace is valid in the original system under the given valuation.

  • The test algorithm terminates with a sound verdict.

    For further details and proofs see the technical report.

IPA Lentedagen 2006


Case study ceps

Case Study CEPS

  • Common Electronic Purse Specifications: protocol for electronic payment using a multi-currency smart-card

  • Input and output parameters of card actions mainly natural numbers  (in)finite domain

  • Variables partially arrays(up to 16 elements in simplified µCRL realization)

  • As µCRL specification:

    • 54 summands

    • 44 process variables (netto)

    • 207process variables (brutto),due to arrays of structures

IPA Lentedagen 2006


Case study ceps cont d

Case Study CEPS (cont'd)

  • Instantiation and reduction (abstracted specification): ca. 16 min. on five 2.2GHz Athlon 64bit single CPU computers (1 GB RAM each)

  • Generation of two test cases: 594 and 109 states, resp. in less than one second (one 2.2GHz AMD Athlon XP 32 bit CPU and 1 GB RAM)

  • Constraint solving produces results in negligable time

IPA Lentedagen 2006


Conclusion

Conclusion

  • Data abstraction makes state-based test generation applicable to systems with large data domains

  • Approach successfully evaluated on CEPS case study

  • Ongoing and future work:

    • Redesign of tools (add "real" constraint-solving) 

    • On-the-fly constraint-solving

    • Treatment of -steps

    • (Generation and) execution of TTCN-3 test cases

    • Integration of UML as specification language

IPA Lentedagen 2006


Related links

TT-Medal Project: www.tt-medal.org

Test generator TGV:

www-verimag.imag.fr/~async/TGV

www.inrialpes.fr/vasy/cadp

Data abstraction tools: www.cwi.nl/~calame/dataabstr.html

Technical report and papers:www.cwi.nl/~calame/works.html

Jens R. Calamé

[email protected]

Natalia Ioustinova

[email protected]

Jaco van de Pol

[email protected]

Related Links

IPA Lentedagen 2006


  • Login