Passive testing
Sponsored Links
This presentation is the property of its rightful owner.
1 / 45

Passive Testing PowerPoint PPT Presentation


  • 52 Views
  • Uploaded on
  • Presentation posted in: General

Passive Testing. Wissam Mallouli INT-LOR [email protected] Institut National des Télécommunications - Evry 11/30/2007. Table of contents. Introduction Active/Passive Testing Forward checking technique The backward checking technique Signature based technique Conclusion.

Download Presentation

Passive Testing

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Passive Testing

Wissam Mallouli

INT-LOR

[email protected]

Institut National des Télécommunications - Evry

11/30/2007


Table of contents

  • Introduction

  • Active/Passive Testing

  • Forward checking technique

  • The backward checking technique

  • Signature based technique

  • Conclusion


Table of contents

  • Introduction

  • Active/Passive Testing

  • Forward checking technique

  • The backward checking technique

  • Signature based technique

  • Conclusion


Introduction

  • Fault detection : essential domain but fastidious. Need of automation.

  • Testing of a system.

  • Active/Passive testing techniques


Table of contents

  • Introduction

  • Active/Passive Testing

  • Forward checking technique

  • The backward checking technique

  • Signature based technique

  • Conclusion


Active Testing

IUT

Active Tester

Verdict:

PASS,FAIL,

INCONC.

Functional &

Security

Formal Specification

Test Suites

Automatic test generation based on formal descriptions

Generation of a : - reasonable test scenarios number (Execution)

- Complete (to cover all the system transitions)


Conformance Testing(1/2)

  • Check if the implementation of a system conforms to its specification

I

O

System

(S)

I

O1=O???

System

(I)


Errors

IMPLEMENTATION

i1 / o2

1

2

SPECIFICATION

output error

i1 / o1

1

2

i1 / o1

1

3

transfert error

i1 / o2

1

3

mixte error


Passive Testing

IUT

Trace

Collection

Passive Tester

PO

Verdict:

PASS,FAIL,

INCONC.

System User

Security Properties Specification


The two methods of conformity test (1/2)

  • Active testing :

    • principe : send messages through the implementation, wait for answer, compare answer with the desired one

    • need two phases :

      • test sequences generation

      • tests application

    • pros and cons :

      • + : possibility to focus on a particular area of the specification

      • - : automatic test generation is difficult

      • - : can disturb (or crash) the IUT


The two methods of conformity test (2/2)

  • Passive testing :

    • principe : observation input/output sequences (trace), application of the trace on the spécification

    • pros and cons :

      • + : no interference with the IUT

      • + : no need of tests generation

      • - : algorithmsefficiency


Table of contents

  • Introduction

  • Active/Passive Testing

  • Forward/backward checking technique

  • Signature based technique

  • Conclusion


Principles

  • Trace + Formal specification of the system

  • To map the trace in the specification

  • 2 differents algorithms : forward and backward checking


a/y

b/y

S1

S2

a/y

b/y

b/y

a/x

b/y

S0

S3

a/x

Example/Exercice

a/x b/y a/x b/y b/y a/y b/y a/y b/y (trace 1)

a/x b/y a/x b/y b/y a/x b/x a/y b/y (trace 2)


EFSM

  • EFSM (Extended Finite State Machine) :

    • I/O events with or without parameters

    • a predicate (to be satisfied)

    • actions (to be done)


EFSM : definitions

  • EFSM = heptuplet (S, s0, Sf, I, O, x, T)

    • S : finite non empty set of states

    • s0 : initial state

    • Sf : finite set of final states

    • I : finite set of input symbols (  parameters)

    • O : finite set of output symbols (  parameters)

    • x : finite set of variables

    • T : finite set of transitions


EFSM

  • Transition = sextuplet (st, ft, it, ot, Pt, At)

    • st : initial state

    • ft : final state

    • it : input event

    • ot : output event

    • Pt : predicate

    • At : set of actions


EFSM Formalism

b/y

a/y

S0

S2

a/y

P(X0) true

b/y

b/y

A(X0)

a/x

S1

S3

b/y

a/x

S=(S0,S1,S2,S3) I=(a,b) O=(x,y)

18


Variables values checking

  • Test by value determination

  • Test by determination of interval of variables


Principle

value determination according to an event trace (naive approach)

Test by value determination


Test by value determination

  • Algorithm in two phases :

    • phase 1 : homing. Trace exploration until only one candidat state stays

    • phase 2 : error detection. Trace exploration until the end, or an error


Test by value determination

  • Variables determination :

    • few possibles transitions giving different values to one variable  UNDEFINED variable

    • predicate with at least one UNDEFINED variable  accepted predicate (doubt)


State of art of passive testing

  • Test by value determination

  • Test by determination of interval of variables


Test by determination of interval of variables

  • Principle

    • Intervals to record values

    • Assertions = boolean formula of predicates

    • Uses Candidate Configuration Set (CCS)


Le test par détermination d'intervalles de valeurs

  • Intervalles

    • objectif : résoudre le problème de la perte d'information (plusieurs valeurs possibles pour une variable)

    • notation :

      • a v  b  R(v) = [a; b]

      • v = a  R(v) = [a; a]


Test by determination of interval of variables

  • Intervals : operations

    • sum of two intervals :

      • [a; b] + [c; d] = [a+c; b+d]

    • Subtraction of two intervals :

      • [a; b] - [c; d] = [a-c; d-b]

    • multiplication of an interval by a scalar :

      • w  [a; b] = [wa; wb] si w  0

      • w  [a; b] = [wb; wa] si w  0


Test by determination of interval of variables

  • Consistency and refinement test

    • refinement of intervals according to constraints (predicats and actions)

    • if an interval becomes empty then the configuration is false


Table of contents

  • Introduction

  • Error types and testing methods

  • State of art of passive testing

  • The backward checking technique

  • Conclusion


The backward checking technique

  • Backward checking : principle

  • Backtracking a transition

  • Backtracking a trace

  • Backtracking the past of a trace


Backward checking : principle

  • Idea : « To understand present we must study the past »


Backward checking : principle

  • Backward checking :

    • backtracking the event trace

    • backtracking the past of the trace

past of the trace

FAIL

FAIL

trace

FAIL

OK

sens of the trace

FAIL

sens of the algorithm


Backward checking : principle

  • Advantages of this approach :

    • start with correct information

    • build artificially a longer trace


Backward checking : principle

  • To validate a trace in the past we must validate special variables : determinants

  • Definition : a variable v is a determinant for the trace t if v must necessarily be validated before we can validate t


The backward checking technique

  • Backward checking : principle

  • Backtracking a transition

  • Backtracking a trace

  • Backtracking the past of a trace


4

f

i

Backtracking a transition

P : u>=1

A : x=1

y=y+1

z=v+w

R = < f ;

u [0;3], x [1;1],

y [2;8], z [1;2], a [7;9] ;

Asrt = - ;

D = {u, y, z, a} >

R = < i ;

u [1;3],

a [7;9],

cste [1;2],

y [1;7] ;

Asrt = v+w=cste ;

D = {u, a, y, v, w} >


The backward checking technique

  • Backward checking : principle

  • Backtracking a transition

  • Backtracking a trace

  • Backtracking the past of a trace


Backtracking a trace

  • similar to intervals determination algorithm but backward

  • process the transition backtracking for each I/O couple


The backward checking technique

  • Backward checking : principle

  • Backtracking a transition

  • Backtracking a trace

  • Backtracking the past of a trace


Table of contents

  • Introduction

  • Active/Passive Testing

  • Forward checking technique

  • The backward checking technique

  • Invariant based technique

  • Conclusion


Test by invariants : principle

  • Definition : invariant = property that is always true

  • Two step test :

    • extraction of invariants from the specification

    • application of invariants on event traces from implementation

  • Problem : temporal logic not in the model

  • Solution : I/O invariants


Test by invariants : I/O invariants

  • An invariant is composed of two parts :

    • the test (an input or output)

    • the preambule (I/O sequence)

  • 3 kind of invariants :

    • output invariants

    • input invariants

    • succession invariants


Test by invariants : Output invariant

  • Definition : invariant in which the test is an output

  • Meaning : « immediatly after the sequence préambule there is always the output test

  • Example :

    (i1 / o1) (i2 / o2)

    (preambule in blue, test in red)


Test by invariants : Input invariant

  • Definition : invariant in which the test is an input

  • Meaning : « immediatly before the sequence préambule there is always the input test

  • Example :

    (i1 / o1) (i2 / o2) (i3/

    (preambule in blue, test in red)


Test by invariants : succession invariant

  • Definition : I/O invariant for complex properties

    (loops …)

  • Example :

    • the 3 invariants below build the property : « only the third i2 we meet is followed by o3 »

      (i1 / o1) (i2 / o2)

      (i1 / o1) (i2 / o2) (i2 / o2)

      (i1 / o1) (i2 / o2) (i2 / o2) (i2 / o3)


Test by invariants

  • + : powerful method (complex properties)

  • + : fast when invariants are known

  • - : difficulties to extract interesting invariants (not automated)

  • - : sensible to non-determinism of specification

  • - : doesn't detect every errors


  • Login