Passive testing
This presentation is the property of its rightful owner.
Sponsored Links
1 / 45

Passive Testing PowerPoint PPT Presentation


  • 45 Views
  • Uploaded on
  • Presentation posted in: General

Passive Testing. Wissam Mallouli INT-LOR [email protected] Institut National des Télécommunications - Evry 11/30/2007. Table of contents. Introduction Active/Passive Testing Forward checking technique The backward checking technique Signature based technique Conclusion.

Download Presentation

Passive Testing

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Passive testing

Passive Testing

Wissam Mallouli

INT-LOR

[email protected]

Institut National des Télécommunications - Evry

11/30/2007


Table of contents

Table of contents

  • Introduction

  • Active/Passive Testing

  • Forward checking technique

  • The backward checking technique

  • Signature based technique

  • Conclusion


Table of contents1

Table of contents

  • Introduction

  • Active/Passive Testing

  • Forward checking technique

  • The backward checking technique

  • Signature based technique

  • Conclusion


Introduction

Introduction

  • Fault detection : essential domain but fastidious. Need of automation.

  • Testing of a system.

  • Active/Passive testing techniques


Table of contents2

Table of contents

  • Introduction

  • Active/Passive Testing

  • Forward checking technique

  • The backward checking technique

  • Signature based technique

  • Conclusion


Active testing

Active Testing

IUT

Active Tester

Verdict:

PASS,FAIL,

INCONC.

Functional &

Security

Formal Specification

Test Suites

Automatic test generation based on formal descriptions

Generation of a : - reasonable test scenarios number (Execution)

- Complete (to cover all the system transitions)


Conformance testing 1 2

Conformance Testing(1/2)

  • Check if the implementation of a system conforms to its specification

I

O

System

(S)

I

O1=O???

System

(I)


Errors

Errors

IMPLEMENTATION

i1 / o2

1

2

SPECIFICATION

output error

i1 / o1

1

2

i1 / o1

1

3

transfert error

i1 / o2

1

3

mixte error


Passive testing1

Passive Testing

IUT

Trace

Collection

Passive Tester

PO

Verdict:

PASS,FAIL,

INCONC.

System User

Security Properties Specification


The two methods of conformity test 1 2

The two methods of conformity test (1/2)

  • Active testing :

    • principe : send messages through the implementation, wait for answer, compare answer with the desired one

    • need two phases :

      • test sequences generation

      • tests application

    • pros and cons :

      • + : possibility to focus on a particular area of the specification

      • - : automatic test generation is difficult

      • - : can disturb (or crash) the IUT


The two methods of conformity test 2 2

The two methods of conformity test (2/2)

  • Passive testing :

    • principe : observation input/output sequences (trace), application of the trace on the spécification

    • pros and cons :

      • + : no interference with the IUT

      • + : no need of tests generation

      • - : algorithmsefficiency


Table of contents3

Table of contents

  • Introduction

  • Active/Passive Testing

  • Forward/backward checking technique

  • Signature based technique

  • Conclusion


Principles

Principles

  • Trace + Formal specification of the system

  • To map the trace in the specification

  • 2 differents algorithms : forward and backward checking


Example exercice

a/y

b/y

S1

S2

a/y

b/y

b/y

a/x

b/y

S0

S3

a/x

Example/Exercice

a/x b/y a/x b/y b/y a/y b/y a/y b/y (trace 1)

a/x b/y a/x b/y b/y a/x b/x a/y b/y (trace 2)


Passive testing

EFSM

  • EFSM (Extended Finite State Machine) :

    • I/O events with or without parameters

    • a predicate (to be satisfied)

    • actions (to be done)


Efsm definitions

EFSM : definitions

  • EFSM = heptuplet (S, s0, Sf, I, O, x, T)

    • S : finite non empty set of states

    • s0 : initial state

    • Sf : finite set of final states

    • I : finite set of input symbols (  parameters)

    • O : finite set of output symbols (  parameters)

    • x : finite set of variables

    • T : finite set of transitions


Passive testing

EFSM

  • Transition = sextuplet (st, ft, it, ot, Pt, At)

    • st : initial state

    • ft : final state

    • it : input event

    • ot : output event

    • Pt : predicate

    • At : set of actions


Efsm formalism

EFSM Formalism

b/y

a/y

S0

S2

a/y

P(X0) true

b/y

b/y

A(X0)

a/x

S1

S3

b/y

a/x

S=(S0,S1,S2,S3) I=(a,b) O=(x,y)

18


Variables values checking

Variables values checking

  • Test by value determination

  • Test by determination of interval of variables


Test by value determination

Principle

value determination according to an event trace (naive approach)

Test by value determination


Test by value determination1

Test by value determination

  • Algorithm in two phases :

    • phase 1 : homing. Trace exploration until only one candidat state stays

    • phase 2 : error detection. Trace exploration until the end, or an error


Test by value determination2

Test by value determination

  • Variables determination :

    • few possibles transitions giving different values to one variable  UNDEFINED variable

    • predicate with at least one UNDEFINED variable  accepted predicate (doubt)


State of art of passive testing

State of art of passive testing

  • Test by value determination

  • Test by determination of interval of variables


Test by determination of interval of variables

Test by determination of interval of variables

  • Principle

    • Intervals to record values

    • Assertions = boolean formula of predicates

    • Uses Candidate Configuration Set (CCS)


Le test par d termination d intervalles de valeurs

Le test par détermination d'intervalles de valeurs

  • Intervalles

    • objectif : résoudre le problème de la perte d'information (plusieurs valeurs possibles pour une variable)

    • notation :

      • a v  b  R(v) = [a; b]

      • v = a  R(v) = [a; a]


Test by determination of interval of variables1

Test by determination of interval of variables

  • Intervals : operations

    • sum of two intervals :

      • [a; b] + [c; d] = [a+c; b+d]

    • Subtraction of two intervals :

      • [a; b] - [c; d] = [a-c; d-b]

    • multiplication of an interval by a scalar :

      • w  [a; b] = [wa; wb] si w  0

      • w  [a; b] = [wb; wa] si w  0


Test by determination of interval of variables2

Test by determination of interval of variables

  • Consistency and refinement test

    • refinement of intervals according to constraints (predicats and actions)

    • if an interval becomes empty then the configuration is false


Table of contents4

Table of contents

  • Introduction

  • Error types and testing methods

  • State of art of passive testing

  • The backward checking technique

  • Conclusion


The backward checking technique

The backward checking technique

  • Backward checking : principle

  • Backtracking a transition

  • Backtracking a trace

  • Backtracking the past of a trace


Backward checking principle

Backward checking : principle

  • Idea : « To understand present we must study the past »


Backward checking principle1

Backward checking : principle

  • Backward checking :

    • backtracking the event trace

    • backtracking the past of the trace

past of the trace

FAIL

FAIL

trace

FAIL

OK

sens of the trace

FAIL

sens of the algorithm


Backward checking principle2

Backward checking : principle

  • Advantages of this approach :

    • start with correct information

    • build artificially a longer trace


Backward checking principle3

Backward checking : principle

  • To validate a trace in the past we must validate special variables : determinants

  • Definition : a variable v is a determinant for the trace t if v must necessarily be validated before we can validate t


The backward checking technique1

The backward checking technique

  • Backward checking : principle

  • Backtracking a transition

  • Backtracking a trace

  • Backtracking the past of a trace


Backtracking a transition

4

f

i

Backtracking a transition

P : u>=1

A : x=1

y=y+1

z=v+w

R = < f ;

u [0;3], x [1;1],

y [2;8], z [1;2], a [7;9] ;

Asrt = - ;

D = {u, y, z, a} >

R = < i ;

u [1;3],

a [7;9],

cste [1;2],

y [1;7] ;

Asrt = v+w=cste ;

D = {u, a, y, v, w} >


The backward checking technique2

The backward checking technique

  • Backward checking : principle

  • Backtracking a transition

  • Backtracking a trace

  • Backtracking the past of a trace


Backtracking a trace

Backtracking a trace

  • similar to intervals determination algorithm but backward

  • process the transition backtracking for each I/O couple


The backward checking technique3

The backward checking technique

  • Backward checking : principle

  • Backtracking a transition

  • Backtracking a trace

  • Backtracking the past of a trace


Table of contents5

Table of contents

  • Introduction

  • Active/Passive Testing

  • Forward checking technique

  • The backward checking technique

  • Invariant based technique

  • Conclusion


Test by invariants principle

Test by invariants : principle

  • Definition : invariant = property that is always true

  • Two step test :

    • extraction of invariants from the specification

    • application of invariants on event traces from implementation

  • Problem : temporal logic not in the model

  • Solution : I/O invariants


Test by invariants i o invariants

Test by invariants : I/O invariants

  • An invariant is composed of two parts :

    • the test (an input or output)

    • the preambule (I/O sequence)

  • 3 kind of invariants :

    • output invariants

    • input invariants

    • succession invariants


Test by invariants output invariant

Test by invariants : Output invariant

  • Definition : invariant in which the test is an output

  • Meaning : « immediatly after the sequence préambule there is always the output test

  • Example :

    (i1 / o1) (i2 / o2)

    (preambule in blue, test in red)


Test by invariants input invariant

Test by invariants : Input invariant

  • Definition : invariant in which the test is an input

  • Meaning : « immediatly before the sequence préambule there is always the input test

  • Example :

    (i1 / o1) (i2 / o2) (i3/

    (preambule in blue, test in red)


Test by invariants succession invariant

Test by invariants : succession invariant

  • Definition : I/O invariant for complex properties

    (loops …)

  • Example :

    • the 3 invariants below build the property : « only the third i2 we meet is followed by o3 »

      (i1 / o1) (i2 / o2)

      (i1 / o1) (i2 / o2) (i2 / o2)

      (i1 / o1) (i2 / o2) (i2 / o2) (i2 / o3)


Test by invariants

Test by invariants

  • + : powerful method (complex properties)

  • + : fast when invariants are known

  • - : difficulties to extract interesting invariants (not automated)

  • - : sensible to non-determinism of specification

  • - : doesn't detect every errors


  • Login