- 303 Views
- Uploaded on
- Presentation posted in: General

Specification-based Protocol Testing

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Specification-based Protocol Testing

Hyoung Seok Hong

Oleg Sokolsky

CSE 642

- Introduction
- Specifications, implementations
- Conformance relations
- Tests, testing architectures
- Assumptions, etc.

- Formal protocol specifications
- Finite-state machine (FSM) testing
- Extended FSM (EFSM) testing
- Test generation for StateCharts

- A specification gives a description of the system behavior.
- An implementation should behave according to the specification.
- Is this really true?

a

a

b

c

b

b

a

- Black-box: unknown system structure
- Testing against reference specification
- Use the specification interface for
- test selection, coverage criteria, result analysis

- White-box: known system structure
- Testing against reference specification
- Use system structure for
- test selection, etc.

- IUT – implementation under test

- PCO – point of control and observation

Tester

PCO

IUT

Local

architecture

Coordinated

architecture

Remote

architecture

PCO1

Upper

Tester

Lower

Tester

IUT

PCO2

underlying service layer

Distributed

architecture

IUT

Tester

Tester

PCO

PCO

PCO

Tester

- A test case defines:
- A finite sequence of inputs
- A finite sequence of expected outputs

- A test suite is a collection of tests to achieve a certain test coverage
- Testing process
- test generation: construct a test suite
- test application: execute each test
- result evaluation: interpret outputs

- Test purpose
- e.g., identify a state of the implementation

- Test preamble
- lead the IUT into known state

- Test body
- invoke behavior corresponding to the test purpose

- Checking of test results

- Abstract models make reasoning easier

conformance

relation

abstract model

of specification

abstract model

of implementation

assumptions and

test hypothesis

assumptions and

test hypothesis

- Testing criteria on a concrete system are hard to formulate

detailed

specification

implementation

?

- Introduction
- Formal protocol specifications
- Formal specification languages

- Finite-state machine (FSM) testing
- Extended FSM (EFSM) testing
- Test generation for StateCharts

- Extended FSM
- SDL
- Estelle
- StateCharts

- Process Algebras
- LOTOS

- Algebraic Specification
- Z, VDM

- Introduction
- Formal protocol specifications
- Finite-state machine (FSM) testing
- FSM model
- FSM fault models
- FSM test selection

- Extended FSM (EFSM) testing
- Test generation for StateCharts

- A set of states

- An initial state

- Transfer function

- Output function

x1

/y1

x3

/y3

x2

/y2

- Finite sets of input (X) and output (Y) events

- Output faults
- Transfer faults
- Transfer faults with additional states
- Additional or missing transitions
- Control and data flow faults?

- Testing methods based on control flow
- Transition-tour (TT) method
- a single sequence of inputs to traverse all transitions
- simple but weak

- Unique-input-output (UIO) method
- a test to identify each state in the specification

- Distinguishing sequence (DS) method
- identify each state by the same test body
- provides full coverage, but may not have a DS

- Transition-tour (TT) method

- Assumption: for each state, there is an input sequence that produces a unique output
- Each test identifies one state
- s1b/1
- s2a/0a/0 b/1
- s3b/1a/1
- preamble body

b/1

s1

s2

b/0

b/0

a/0

a/0

a/1

s3

- Introduction
- Formal protocol specifications
- Finite-state machine (FSM) testing
- Extended FSM (EFSM) testing
- EFSM model
- EFSM fault models
- EFSM test selection

- Test generation for StateCharts

- A set of states

- An initial state

- Transition relation

b1

x1

{output y1}

- Guards

- Update blocks

{output y3}

b2

x2

b3

x3

{update v2}

- Finite sets of input (X) and output (Y) events

- A finite set of variables (V)

- EFSM executions depend on input signals and data
- A test consists of:
- test sequence
- test data

- Executability problem:
- Find data to execute the test sequence
- Undecidable, in general

- Find data to execute the test sequence

- test validation: keep only executable tests

- test application: execute each test
- result evaluation: interpret outputs

- Executability problem introduces an additional step in the testing process:
- test generation: construct a test suite
- test application: execute each test
- result evaluation: interpret outputs

- EFSM executions are data-dependent
- Control flow FSM testing methods are not adequate for EFSM models
- Data flow testing methods account for data dependencies

- Data variables are
- defined by inputs and updates (def)
- used in
- updates (c-use)
- guards (p-use)

- Data-flow graph captures data dependencies

- Directed graph with nodes labeled with definitions and uses of variables

p-use v

def u

v=0{input u}

u<0{v:=u+1}

p-use u

c-use u

def v

- all-def
- test suite covers each definition at least once

- all-use
- cover each def-use association at least once

- all-du-paths
- exercise all paths from each definition of a variable to every its use.

- Find at least one definition-free path for every def-use association

no definitons of v

def v

use v

- Introduction
- Formal protocol specifications
- Finite-state machine (FSM) testing
- Extended FSM (EFSM) testing
- Test generation for Statecharts

EFSMs (FSMs + variables) + concurrency + hierarchy + communication + real-time

Widely used for specifying real-time embedded HW/SW controllers

Also used in most of object-oriented methodologies, e.g., UML

off

EFSMs +

power-on/

light-on; m:=0

power-off/

light-off

Concurrency +

on

Hierarchy +

coffee

Communication +

coffee[m>0]/start;dec

coffee[m>0]/start

idle

busy

Real-time

tm(coffee,[3,5])/stop

done/stop

money

inc/m:=m+1

inc/m:=0

not

empty

empty

dec[m=1]/m:=0

dec[m>1]/m:=m-1

Statecharts =

- The main purpose
- What should implementations do?
- {power-on}/{light-on}, {inc}/{}, {coffee}/{start}

- What should not implementations do?
- {power-on}/{light-on}, {coffee}/{}

- What should implementations do?
- The main issue
- How can we generate a finite and reasonable number of test sequences?

off

power-on/

light-on; m:=0

power-off/

light-off

on

coffee

coffee[m>0]/start;dec

coffee[m>0]/start

idle

busy

done/stop

tm(coffee,[3,5])/stop

money

inc/m:=m+1

inc/m:=0

not

empty

empty

dec[m=1]/m:=0

dec[m>1]/m:=m-1

off

power-on/

light-on; m:=0

power-off/

light-off

on

coffee

coffee[m>0]/start;dec

coffee[m>0]/start

idle

busy

done/stop

tm(coffee,[3,5])/stop

money

inc/m:=m+1

inc/m:=0

not

empty

empty

dec[m=1]/m:=0

dec[m>1]/m:=m-1

off

power-on/

light-on; m:=0

power-off/

light-off

on

coffee

coffee[m>0]/start;dec

coffee[m>0]/start

idle

busy

done/stop

tm(coffee,[3,5])/stop

money

inc/m:=m+1

inc/m:=0

not

empty

empty

dec[m=1]/m:=0

dec[m>1]/m:=m-1

- Basic idea
- Transforms Statecharts into EFSMs

- Advantage
- Can reuse the existing methods and tools for EFSMs
- Can handle infinite state space

- Disadvantage
- Cannot determine the executability of test sequences

- Basic Idea
- Test generation can be reduced into model checking
- TG: for each state s, is there a path leading to s
- MC: a temporal logic formula EFs express this property

- Advantage
- Fully automatic
- Generates only executable test sequences

- Disadvantages
- Can handle only finite state space

- Testing classes in object-oriented programs
- Can we show an CLASS implementation conforms to a Statechart specification?
- Variables of Statecharts: data members of classes
- Events of Statecharts: member functions of classes

- Testing inheritance in object-oriented programs
- Can we reuse the test sequences of a super-class for testing its subclasses ?

Vending

Machine

off

power-on/

light-on; m:=0

power-off/

light-off

on

coffee

coffee[m>0]/start;dec

Coffee

Vending

Machine

Cocoa

Vending

Machine

idle

busy

tm(coffee,[3,5])/stop

cocoa

cocoa[m>0]/start;dec

idle

busy

tm(coffee,[3,5])/stop

Coffee-

Cocoa

Vending

Machine

money

inc/m:=m+1

inc/m:=0

not

empty

empty

dec[m=1]/m:=0

dec[m>1]/m:=m-1