Passive testing
Download
1 / 45

Passive Testing - PowerPoint PPT Presentation


  • 62 Views
  • Uploaded on

Passive Testing. Wissam Mallouli INT-LOR [email protected] Institut National des Télécommunications - Evry 11/30/2007. Table of contents. Introduction Active/Passive Testing Forward checking technique The backward checking technique Signature based technique Conclusion.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Passive Testing' - sarila


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Passive testing

Passive Testing

Wissam Mallouli

INT-LOR

[email protected]

Institut National des Télécommunications - Evry

11/30/2007


Table of contents
Table of contents

  • Introduction

  • Active/Passive Testing

  • Forward checking technique

  • The backward checking technique

  • Signature based technique

  • Conclusion


Table of contents1
Table of contents

  • Introduction

  • Active/Passive Testing

  • Forward checking technique

  • The backward checking technique

  • Signature based technique

  • Conclusion


Introduction
Introduction

  • Fault detection : essential domain but fastidious. Need of automation.

  • Testing of a system.

  • Active/Passive testing techniques


Table of contents2
Table of contents

  • Introduction

  • Active/Passive Testing

  • Forward checking technique

  • The backward checking technique

  • Signature based technique

  • Conclusion


Active testing
Active Testing

IUT

Active Tester

Verdict:

PASS,FAIL,

INCONC.

Functional &

Security

Formal Specification

Test Suites

Automatic test generation based on formal descriptions

Generation of a : - reasonable test scenarios number (Execution)

- Complete (to cover all the system transitions)


Conformance testing 1 2
Conformance Testing(1/2)

  • Check if the implementation of a system conforms to its specification

I

O

System

(S)

I

O1=O???

System

(I)


Errors
Errors

IMPLEMENTATION

i1 / o2

1

2

SPECIFICATION

output error

i1 / o1

1

2

i1 / o1

1

3

transfert error

i1 / o2

1

3

mixte error


Passive testing1
Passive Testing

IUT

Trace

Collection

Passive Tester

PO

Verdict:

PASS,FAIL,

INCONC.

System User

Security Properties Specification


The two methods of conformity test 1 2
The two methods of conformity test (1/2)

  • Active testing :

    • principe : send messages through the implementation, wait for answer, compare answer with the desired one

    • need two phases :

      • test sequences generation

      • tests application

    • pros and cons :

      • + : possibility to focus on a particular area of the specification

      • - : automatic test generation is difficult

      • - : can disturb (or crash) the IUT


The two methods of conformity test 2 2
The two methods of conformity test (2/2)

  • Passive testing :

    • principe : observation input/output sequences (trace), application of the trace on the spécification

    • pros and cons :

      • + : no interference with the IUT

      • + : no need of tests generation

      • - : algorithmsefficiency


Table of contents3
Table of contents

  • Introduction

  • Active/Passive Testing

  • Forward/backward checking technique

  • Signature based technique

  • Conclusion


Principles
Principles

  • Trace + Formal specification of the system

  • To map the trace in the specification

  • 2 differents algorithms : forward and backward checking


Example exercice

a/y

b/y

S1

S2

a/y

b/y

b/y

a/x

b/y

S0

S3

a/x

Example/Exercice

a/x b/y a/x b/y b/y a/y b/y a/y b/y (trace 1)

a/x b/y a/x b/y b/y a/x b/x a/y b/y (trace 2)


EFSM

  • EFSM (Extended Finite State Machine) :

    • I/O events with or without parameters

    • a predicate (to be satisfied)

    • actions (to be done)


Efsm definitions
EFSM : definitions

  • EFSM = heptuplet (S, s0, Sf, I, O, x, T)

    • S : finite non empty set of states

    • s0 : initial state

    • Sf : finite set of final states

    • I : finite set of input symbols (  parameters)

    • O : finite set of output symbols (  parameters)

    • x : finite set of variables

    • T : finite set of transitions


EFSM

  • Transition = sextuplet (st, ft, it, ot, Pt, At)

    • st : initial state

    • ft : final state

    • it : input event

    • ot : output event

    • Pt : predicate

    • At : set of actions


Efsm formalism
EFSM Formalism

b/y

a/y

S0

S2

a/y

P(X0) true

b/y

b/y

A(X0)

a/x

S1

S3

b/y

a/x

S=(S0,S1,S2,S3) I=(a,b) O=(x,y)

18


Variables values checking
Variables values checking

  • Test by value determination

  • Test by determination of interval of variables


Test by value determination

Principle

value determination according to an event trace (naive approach)

Test by value determination


Test by value determination1
Test by value determination

  • Algorithm in two phases :

    • phase 1 : homing. Trace exploration until only one candidat state stays

    • phase 2 : error detection. Trace exploration until the end, or an error


Test by value determination2
Test by value determination

  • Variables determination :

    • few possibles transitions giving different values to one variable  UNDEFINED variable

    • predicate with at least one UNDEFINED variable  accepted predicate (doubt)


State of art of passive testing
State of art of passive testing

  • Test by value determination

  • Test by determination of interval of variables


Test by determination of interval of variables
Test by determination of interval of variables

  • Principle

    • Intervals to record values

    • Assertions = boolean formula of predicates

    • Uses Candidate Configuration Set (CCS)


Le test par d termination d intervalles de valeurs
Le test par détermination d'intervalles de valeurs

  • Intervalles

    • objectif : résoudre le problème de la perte d'information (plusieurs valeurs possibles pour une variable)

    • notation :

      • a v  b  R(v) = [a; b]

      • v = a  R(v) = [a; a]


Test by determination of interval of variables1
Test by determination of interval of variables

  • Intervals : operations

    • sum of two intervals :

      • [a; b] + [c; d] = [a+c; b+d]

    • Subtraction of two intervals :

      • [a; b] - [c; d] = [a-c; d-b]

    • multiplication of an interval by a scalar :

      • w  [a; b] = [wa; wb] si w  0

      • w  [a; b] = [wb; wa] si w  0


Test by determination of interval of variables2
Test by determination of interval of variables

  • Consistency and refinement test

    • refinement of intervals according to constraints (predicats and actions)

    • if an interval becomes empty then the configuration is false


Table of contents4
Table of contents

  • Introduction

  • Error types and testing methods

  • State of art of passive testing

  • The backward checking technique

  • Conclusion


The backward checking technique
The backward checking technique

  • Backward checking : principle

  • Backtracking a transition

  • Backtracking a trace

  • Backtracking the past of a trace


Backward checking principle
Backward checking : principle

  • Idea : « To understand present we must study the past »


Backward checking principle1
Backward checking : principle

  • Backward checking :

    • backtracking the event trace

    • backtracking the past of the trace

past of the trace

FAIL

FAIL

trace

FAIL

OK

sens of the trace

FAIL

sens of the algorithm


Backward checking principle2
Backward checking : principle

  • Advantages of this approach :

    • start with correct information

    • build artificially a longer trace


Backward checking principle3
Backward checking : principle

  • To validate a trace in the past we must validate special variables : determinants

  • Definition : a variable v is a determinant for the trace t if v must necessarily be validated before we can validate t


The backward checking technique1
The backward checking technique

  • Backward checking : principle

  • Backtracking a transition

  • Backtracking a trace

  • Backtracking the past of a trace


Backtracking a transition

4

f

i

Backtracking a transition

P : u>=1

A : x=1

y=y+1

z=v+w

R = < f ;

u [0;3], x [1;1],

y [2;8], z [1;2], a [7;9] ;

Asrt = - ;

D = {u, y, z, a} >

R = < i ;

u [1;3],

a [7;9],

cste [1;2],

y [1;7] ;

Asrt = v+w=cste ;

D = {u, a, y, v, w} >


The backward checking technique2
The backward checking technique

  • Backward checking : principle

  • Backtracking a transition

  • Backtracking a trace

  • Backtracking the past of a trace


Backtracking a trace
Backtracking a trace

  • similar to intervals determination algorithm but backward

  • process the transition backtracking for each I/O couple


The backward checking technique3
The backward checking technique

  • Backward checking : principle

  • Backtracking a transition

  • Backtracking a trace

  • Backtracking the past of a trace


Table of contents5
Table of contents

  • Introduction

  • Active/Passive Testing

  • Forward checking technique

  • The backward checking technique

  • Invariant based technique

  • Conclusion


Test by invariants principle
Test by invariants : principle

  • Definition : invariant = property that is always true

  • Two step test :

    • extraction of invariants from the specification

    • application of invariants on event traces from implementation

  • Problem : temporal logic not in the model

  • Solution : I/O invariants


Test by invariants i o invariants
Test by invariants : I/O invariants

  • An invariant is composed of two parts :

    • the test (an input or output)

    • the preambule (I/O sequence)

  • 3 kind of invariants :

    • output invariants

    • input invariants

    • succession invariants


Test by invariants output invariant
Test by invariants : Output invariant

  • Definition : invariant in which the test is an output

  • Meaning : « immediatly after the sequence préambule there is always the output test

  • Example :

    (i1 / o1) (i2 / o2)

    (preambule in blue, test in red)


Test by invariants input invariant
Test by invariants : Input invariant

  • Definition : invariant in which the test is an input

  • Meaning : « immediatly before the sequence préambule there is always the input test

  • Example :

    (i1 / o1) (i2 / o2) (i3/

    (preambule in blue, test in red)


Test by invariants succession invariant
Test by invariants : succession invariant

  • Definition : I/O invariant for complex properties

    (loops …)

  • Example :

    • the 3 invariants below build the property : « only the third i2 we meet is followed by o3 »

      (i1 / o1) (i2 / o2)

      (i1 / o1) (i2 / o2) (i2 / o2)

      (i1 / o1) (i2 / o2) (i2 / o2) (i2 / o3)


Test by invariants
Test by invariants

  • + : powerful method (complex properties)

  • + : fast when invariants are known

  • - : difficulties to extract interesting invariants (not automated)

  • - : sensible to non-determinism of specification

  • - : doesn't detect every errors


ad