Specification based protocol testing l.jpg
This presentation is the property of its rightful owner.
Sponsored Links
1 / 38

Specification-based Protocol Testing PowerPoint PPT Presentation


  • 315 Views
  • Updated On :
  • Presentation posted in: General

Specification-based Protocol Testing. Hyoung Seok Hong Oleg Sokolsky CSE 642. Outline. Introduction Specifications, implementations Conformance relations Tests, testing architectures Assumptions, etc. Formal protocol specifications Finite-state machine (FSM) testing

Related searches for Specification-based Protocol Testing

Download Presentation

Specification-based Protocol Testing

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Specification based protocol testing l.jpg

Specification-based Protocol Testing

Hyoung Seok Hong

Oleg Sokolsky

CSE 642


Outline l.jpg

Outline

  • Introduction

    • Specifications, implementations

    • Conformance relations

    • Tests, testing architectures

    • Assumptions, etc.

  • Formal protocol specifications

  • Finite-state machine (FSM) testing

  • Extended FSM (EFSM) testing

  • Test generation for StateCharts


Testing problem l.jpg

Testing problem

  • A specification gives a description of the system behavior.

  • An implementation should behave according to the specification.

  • Is this really true?

a

a

b

c

b

b

a


Black box vs white box testing l.jpg

Black-box vs. White-box testing

  • Black-box: unknown system structure

    • Testing against reference specification

    • Use the specification interface for

      • test selection, coverage criteria, result analysis

  • White-box: known system structure

    • Testing against reference specification

    • Use system structure for

      • test selection, etc.


Protocol testing architectures l.jpg

  • IUT – implementation under test

  • PCO – point of control and observation

Tester

PCO

IUT

Protocol testing architectures

Local

architecture


Protocol testing architectures6 l.jpg

Coordinated

architecture

Remote

architecture

PCO1

Upper

Tester

Lower

Tester

IUT

PCO2

underlying service layer

Protocol testing architectures

Distributed

architecture


General testing architecture l.jpg

General testing architecture

IUT

Tester

Tester

PCO

PCO

PCO

Tester


Tests and test suites l.jpg

Tests and test suites

  • A test case defines:

    • A finite sequence of inputs

    • A finite sequence of expected outputs

  • A test suite is a collection of tests to achieve a certain test coverage

  • Testing process

    • test generation: construct a test suite

    • test application: execute each test

    • result evaluation: interpret outputs


Test structure l.jpg

Test structure

  • Test purpose

    • e.g., identify a state of the implementation

  • Test preamble

    • lead the IUT into known state

  • Test body

    • invoke behavior corresponding to the test purpose

  • Checking of test results


Modeling approach l.jpg

  • Abstract models make reasoning easier

conformance

relation

abstract model

of specification

abstract model

of implementation

assumptions and

test hypothesis

assumptions and

test hypothesis

Modeling approach

  • Testing criteria on a concrete system are hard to formulate

detailed

specification

implementation

?


Outline11 l.jpg

Outline

  • Introduction

  • Formal protocol specifications

    • Formal specification languages

  • Finite-state machine (FSM) testing

  • Extended FSM (EFSM) testing

  • Test generation for StateCharts


Formal specification languages l.jpg

Formal specification languages

  • Extended FSM

    • SDL

    • Estelle

    • StateCharts

  • Process Algebras

    • LOTOS

  • Algebraic Specification

    • Z, VDM


Outline13 l.jpg

Outline

  • Introduction

  • Formal protocol specifications

  • Finite-state machine (FSM) testing

    • FSM model

    • FSM fault models

    • FSM test selection

  • Extended FSM (EFSM) testing

  • Test generation for StateCharts


Fsm model l.jpg

  • A set of states

  • An initial state

  • Transfer function

  • Output function

x1

/y1

x3

/y3

x2

/y2

FSM model

  • Finite sets of input (X) and output (Y) events


Fsm based fault model l.jpg

FSM-based fault model

  • Output faults

  • Transfer faults

  • Transfer faults with additional states

  • Additional or missing transitions

  • Control and data flow faults?


Testing based on fsm models l.jpg

Testing based on FSM models

  • Testing methods based on control flow

    • Transition-tour (TT) method

      • a single sequence of inputs to traverse all transitions

      • simple but weak

    • Unique-input-output (UIO) method

      • a test to identify each state in the specification

    • Distinguishing sequence (DS) method

      • identify each state by the same test body

      • provides full coverage, but may not have a DS


Uio testing method l.jpg

UIO testing method

  • Assumption: for each state, there is an input sequence that produces a unique output

  • Each test identifies one state

  • s1b/1

  • s2a/0a/0 b/1

  • s3b/1a/1

  • preamble body

b/1

s1

s2

b/0

b/0

a/0

a/0

a/1

s3


Outline18 l.jpg

Outline

  • Introduction

  • Formal protocol specifications

  • Finite-state machine (FSM) testing

  • Extended FSM (EFSM) testing

    • EFSM model

    • EFSM fault models

    • EFSM test selection

  • Test generation for StateCharts


Efsm model l.jpg

  • A set of states

  • An initial state

  • Transition relation

b1

x1

{output y1}

  • Guards

  • Update blocks

{output y3}

b2

x2

b3

x3

{update v2}

EFSM model

  • Finite sets of input (X) and output (Y) events

  • A finite set of variables (V)


Tests for efsm models l.jpg

Tests for EFSM models

  • EFSM executions depend on input signals and data

  • A test consists of:

    • test sequence

    • test data

  • Executability problem:

    • Find data to execute the test sequence

      • Undecidable, in general 


Test validation l.jpg

  • test validation: keep only executable tests

  • test application: execute each test

  • result evaluation: interpret outputs

Test validation

  • Executability problem introduces an additional step in the testing process:

    • test generation: construct a test suite

    • test application: execute each test

    • result evaluation: interpret outputs


Control and data flow testing l.jpg

Control and data flow testing

  • EFSM executions are data-dependent

  • Control flow FSM testing methods are not adequate for EFSM models

  • Data flow testing methods account for data dependencies


Data flow definitions and uses l.jpg

Data flow: definitions and uses

  • Data variables are

    • defined by inputs and updates (def)

    • used in

      • updates (c-use)

      • guards (p-use)

  • Data-flow graph captures data dependencies


Data flow graph l.jpg

Data flow graph

  • Directed graph with nodes labeled with definitions and uses of variables

p-use v

def u

v=0{input u}

u<0{v:=u+1}

p-use u

c-use u

def v


Data flow coverage criteria l.jpg

Data-flow coverage criteria

  • all-def

    • test suite covers each definition at least once

  • all-use

    • cover each def-use association at least once

  • all-du-paths

    • exercise all paths from each definition of a variable to every its use.


All use coverage l.jpg

all-use coverage

  • Find at least one definition-free path for every def-use association

no definitons of v

def v

use v


Outline27 l.jpg

Outline

  • Introduction

  • Formal protocol specifications

  • Finite-state machine (FSM) testing

  • Extended FSM (EFSM) testing

  • Test generation for Statecharts


Statecharts l.jpg

EFSMs (FSMs + variables) + concurrency + hierarchy + communication + real-time

Widely used for specifying real-time embedded HW/SW controllers

Also used in most of object-oriented methodologies, e.g., UML

Statecharts


A coffee vending machine l.jpg

off

EFSMs +

power-on/

light-on; m:=0

power-off/

light-off

Concurrency +

on

Hierarchy +

coffee

Communication +

coffee[m>0]/start;dec

coffee[m>0]/start

idle

busy

Real-time

tm(coffee,[3,5])/stop

done/stop

money

inc/m:=m+1

inc/m:=0

not

empty

empty

dec[m=1]/m:=0

dec[m>1]/m:=m-1

A coffee vending machine

Statecharts =


Test sequences for statecharts l.jpg

Test sequences for Statecharts

  • The main purpose

    • What should implementations do?

      • {power-on}/{light-on}, {inc}/{}, {coffee}/{start}

    • What should not implementations do?

      • {power-on}/{light-on}, {coffee}/{}

  • The main issue

    • How can we generate a finite and reasonable number of test sequences?


State coverage l.jpg

off

power-on/

light-on; m:=0

power-off/

light-off

on

coffee

coffee[m>0]/start;dec

coffee[m>0]/start

idle

busy

done/stop

tm(coffee,[3,5])/stop

money

inc/m:=m+1

inc/m:=0

not

empty

empty

dec[m=1]/m:=0

dec[m>1]/m:=m-1

State coverage


Transition coverage l.jpg

off

power-on/

light-on; m:=0

power-off/

light-off

on

coffee

coffee[m>0]/start;dec

coffee[m>0]/start

idle

busy

done/stop

tm(coffee,[3,5])/stop

money

inc/m:=m+1

inc/m:=0

not

empty

empty

dec[m=1]/m:=0

dec[m>1]/m:=m-1

Transition coverage


Data flow oriented coverage l.jpg

off

power-on/

light-on; m:=0

power-off/

light-off

on

coffee

coffee[m>0]/start;dec

coffee[m>0]/start

idle

busy

done/stop

tm(coffee,[3,5])/stop

money

inc/m:=m+1

inc/m:=0

not

empty

empty

dec[m=1]/m:=0

dec[m>1]/m:=m-1

Data flow oriented coverage


Test generation from statecharts i l.jpg

Test generation from Statecharts (I)

  • Basic idea

    • Transforms Statecharts into EFSMs

  • Advantage

    • Can reuse the existing methods and tools for EFSMs

    • Can handle infinite state space

  • Disadvantage

    • Cannot determine the executability of test sequences


Test generation from statecharts ii l.jpg

Test generation from Statecharts (II)

  • Basic Idea

    • Test generation can be reduced into model checking

    • TG: for each state s, is there a path leading to s

    • MC: a temporal logic formula EFs express this property

  • Advantage

    • Fully automatic

    • Generates only executable test sequences

  • Disadvantages

    • Can handle only finite state space


Another perspective l.jpg

Another perspective

  • Testing classes in object-oriented programs

    • Can we show an CLASS implementation conforms to a Statechart specification?

    • Variables of Statecharts: data members of classes

    • Events of Statecharts: member functions of classes

  • Testing inheritance in object-oriented programs

    • Can we reuse the test sequences of a super-class for testing its subclasses ?


A coffee cocoa vending machine l.jpg

Vending

Machine

off

power-on/

light-on; m:=0

power-off/

light-off

on

coffee

coffee[m>0]/start;dec

Coffee

Vending

Machine

Cocoa

Vending

Machine

idle

busy

tm(coffee,[3,5])/stop

cocoa

cocoa[m>0]/start;dec

idle

busy

tm(coffee,[3,5])/stop

Coffee-

Cocoa

Vending

Machine

money

inc/m:=m+1

inc/m:=0

not

empty

empty

dec[m=1]/m:=0

dec[m>1]/m:=m-1

A coffee-cocoa vending machine


  • Login