Specification based protocol testing
Download
1 / 38

Specification-based Protocol Testing - PowerPoint PPT Presentation


  • 341 Views
  • Updated On :
  • Presentation posted in: General

Specification-based Protocol Testing. Hyoung Seok Hong Oleg Sokolsky CSE 642. Outline. Introduction Specifications, implementations Conformance relations Tests, testing architectures Assumptions, etc. Formal protocol specifications Finite-state machine (FSM) testing

Related searches for Specification-based Protocol Testing

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha

Download Presentation

Specification-based Protocol Testing

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Specification-based Protocol Testing

Hyoung Seok Hong

Oleg Sokolsky

CSE 642


Outline

  • Introduction

    • Specifications, implementations

    • Conformance relations

    • Tests, testing architectures

    • Assumptions, etc.

  • Formal protocol specifications

  • Finite-state machine (FSM) testing

  • Extended FSM (EFSM) testing

  • Test generation for StateCharts


Testing problem

  • A specification gives a description of the system behavior.

  • An implementation should behave according to the specification.

  • Is this really true?

a

a

b

c

b

b

a


Black-box vs. White-box testing

  • Black-box: unknown system structure

    • Testing against reference specification

    • Use the specification interface for

      • test selection, coverage criteria, result analysis

  • White-box: known system structure

    • Testing against reference specification

    • Use system structure for

      • test selection, etc.


  • IUT – implementation under test

  • PCO – point of control and observation

Tester

PCO

IUT

Protocol testing architectures

Local

architecture


Coordinated

architecture

Remote

architecture

PCO1

Upper

Tester

Lower

Tester

IUT

PCO2

underlying service layer

Protocol testing architectures

Distributed

architecture


General testing architecture

IUT

Tester

Tester

PCO

PCO

PCO

Tester


Tests and test suites

  • A test case defines:

    • A finite sequence of inputs

    • A finite sequence of expected outputs

  • A test suite is a collection of tests to achieve a certain test coverage

  • Testing process

    • test generation: construct a test suite

    • test application: execute each test

    • result evaluation: interpret outputs


Test structure

  • Test purpose

    • e.g., identify a state of the implementation

  • Test preamble

    • lead the IUT into known state

  • Test body

    • invoke behavior corresponding to the test purpose

  • Checking of test results


  • Abstract models make reasoning easier

conformance

relation

abstract model

of specification

abstract model

of implementation

assumptions and

test hypothesis

assumptions and

test hypothesis

Modeling approach

  • Testing criteria on a concrete system are hard to formulate

detailed

specification

implementation

?


Outline

  • Introduction

  • Formal protocol specifications

    • Formal specification languages

  • Finite-state machine (FSM) testing

  • Extended FSM (EFSM) testing

  • Test generation for StateCharts


Formal specification languages

  • Extended FSM

    • SDL

    • Estelle

    • StateCharts

  • Process Algebras

    • LOTOS

  • Algebraic Specification

    • Z, VDM


Outline

  • Introduction

  • Formal protocol specifications

  • Finite-state machine (FSM) testing

    • FSM model

    • FSM fault models

    • FSM test selection

  • Extended FSM (EFSM) testing

  • Test generation for StateCharts


  • A set of states

  • An initial state

  • Transfer function

  • Output function

x1

/y1

x3

/y3

x2

/y2

FSM model

  • Finite sets of input (X) and output (Y) events


FSM-based fault model

  • Output faults

  • Transfer faults

  • Transfer faults with additional states

  • Additional or missing transitions

  • Control and data flow faults?


Testing based on FSM models

  • Testing methods based on control flow

    • Transition-tour (TT) method

      • a single sequence of inputs to traverse all transitions

      • simple but weak

    • Unique-input-output (UIO) method

      • a test to identify each state in the specification

    • Distinguishing sequence (DS) method

      • identify each state by the same test body

      • provides full coverage, but may not have a DS


UIO testing method

  • Assumption: for each state, there is an input sequence that produces a unique output

  • Each test identifies one state

  • s1b/1

  • s2a/0a/0 b/1

  • s3b/1a/1

  • preamble body

b/1

s1

s2

b/0

b/0

a/0

a/0

a/1

s3


Outline

  • Introduction

  • Formal protocol specifications

  • Finite-state machine (FSM) testing

  • Extended FSM (EFSM) testing

    • EFSM model

    • EFSM fault models

    • EFSM test selection

  • Test generation for StateCharts


  • A set of states

  • An initial state

  • Transition relation

b1

x1

{output y1}

  • Guards

  • Update blocks

{output y3}

b2

x2

b3

x3

{update v2}

EFSM model

  • Finite sets of input (X) and output (Y) events

  • A finite set of variables (V)


Tests for EFSM models

  • EFSM executions depend on input signals and data

  • A test consists of:

    • test sequence

    • test data

  • Executability problem:

    • Find data to execute the test sequence

      • Undecidable, in general 


  • test validation: keep only executable tests

  • test application: execute each test

  • result evaluation: interpret outputs

Test validation

  • Executability problem introduces an additional step in the testing process:

    • test generation: construct a test suite

    • test application: execute each test

    • result evaluation: interpret outputs


Control and data flow testing

  • EFSM executions are data-dependent

  • Control flow FSM testing methods are not adequate for EFSM models

  • Data flow testing methods account for data dependencies


Data flow: definitions and uses

  • Data variables are

    • defined by inputs and updates (def)

    • used in

      • updates (c-use)

      • guards (p-use)

  • Data-flow graph captures data dependencies


Data flow graph

  • Directed graph with nodes labeled with definitions and uses of variables

p-use v

def u

v=0{input u}

u<0{v:=u+1}

p-use u

c-use u

def v


Data-flow coverage criteria

  • all-def

    • test suite covers each definition at least once

  • all-use

    • cover each def-use association at least once

  • all-du-paths

    • exercise all paths from each definition of a variable to every its use.


all-use coverage

  • Find at least one definition-free path for every def-use association

no definitons of v

def v

use v


Outline

  • Introduction

  • Formal protocol specifications

  • Finite-state machine (FSM) testing

  • Extended FSM (EFSM) testing

  • Test generation for Statecharts


EFSMs (FSMs + variables) + concurrency + hierarchy + communication + real-time

Widely used for specifying real-time embedded HW/SW controllers

Also used in most of object-oriented methodologies, e.g., UML

Statecharts


off

EFSMs +

power-on/

light-on; m:=0

power-off/

light-off

Concurrency +

on

Hierarchy +

coffee

Communication +

coffee[m>0]/start;dec

coffee[m>0]/start

idle

busy

Real-time

tm(coffee,[3,5])/stop

done/stop

money

inc/m:=m+1

inc/m:=0

not

empty

empty

dec[m=1]/m:=0

dec[m>1]/m:=m-1

A coffee vending machine

Statecharts =


Test sequences for Statecharts

  • The main purpose

    • What should implementations do?

      • {power-on}/{light-on}, {inc}/{}, {coffee}/{start}

    • What should not implementations do?

      • {power-on}/{light-on}, {coffee}/{}

  • The main issue

    • How can we generate a finite and reasonable number of test sequences?


off

power-on/

light-on; m:=0

power-off/

light-off

on

coffee

coffee[m>0]/start;dec

coffee[m>0]/start

idle

busy

done/stop

tm(coffee,[3,5])/stop

money

inc/m:=m+1

inc/m:=0

not

empty

empty

dec[m=1]/m:=0

dec[m>1]/m:=m-1

State coverage


off

power-on/

light-on; m:=0

power-off/

light-off

on

coffee

coffee[m>0]/start;dec

coffee[m>0]/start

idle

busy

done/stop

tm(coffee,[3,5])/stop

money

inc/m:=m+1

inc/m:=0

not

empty

empty

dec[m=1]/m:=0

dec[m>1]/m:=m-1

Transition coverage


off

power-on/

light-on; m:=0

power-off/

light-off

on

coffee

coffee[m>0]/start;dec

coffee[m>0]/start

idle

busy

done/stop

tm(coffee,[3,5])/stop

money

inc/m:=m+1

inc/m:=0

not

empty

empty

dec[m=1]/m:=0

dec[m>1]/m:=m-1

Data flow oriented coverage


Test generation from Statecharts (I)

  • Basic idea

    • Transforms Statecharts into EFSMs

  • Advantage

    • Can reuse the existing methods and tools for EFSMs

    • Can handle infinite state space

  • Disadvantage

    • Cannot determine the executability of test sequences


Test generation from Statecharts (II)

  • Basic Idea

    • Test generation can be reduced into model checking

    • TG: for each state s, is there a path leading to s

    • MC: a temporal logic formula EFs express this property

  • Advantage

    • Fully automatic

    • Generates only executable test sequences

  • Disadvantages

    • Can handle only finite state space


Another perspective

  • Testing classes in object-oriented programs

    • Can we show an CLASS implementation conforms to a Statechart specification?

    • Variables of Statecharts: data members of classes

    • Events of Statecharts: member functions of classes

  • Testing inheritance in object-oriented programs

    • Can we reuse the test sequences of a super-class for testing its subclasses ?


Vending

Machine

off

power-on/

light-on; m:=0

power-off/

light-off

on

coffee

coffee[m>0]/start;dec

Coffee

Vending

Machine

Cocoa

Vending

Machine

idle

busy

tm(coffee,[3,5])/stop

cocoa

cocoa[m>0]/start;dec

idle

busy

tm(coffee,[3,5])/stop

Coffee-

Cocoa

Vending

Machine

money

inc/m:=m+1

inc/m:=0

not

empty

empty

dec[m=1]/m:=0

dec[m>1]/m:=m-1

A coffee-cocoa vending machine


ad
  • Login