Overview of ETSI Testing Methodology ICT-OSA/Parlay Workshop Brazil, March 2006 - PowerPoint PPT Presentation

Overview of etsi testing methodology ict osa parlay workshop brazil march 2006 l.jpg
Download
1 / 56

Overview of ETSI Testing Methodology ICT-OSA/Parlay Workshop Brazil, March 2006. Anthony Wiles Manager ETSI Protocol and Testing Competence Centre. Interoperability is …. The ultimate aim of ICT standardisation

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Overview of ETSI Testing Methodology ICT-OSA/Parlay Workshop Brazil, March 2006

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Overview of etsi testing methodology ict osa parlay workshop brazil march 2006 l.jpg

Overview of ETSI Testing Methodology ICT-OSA/Parlay WorkshopBrazil, March 2006

Anthony Wiles

Manager

ETSI Protocol and Testing Competence Centre


Interoperability is l.jpg

Interoperability is …

  • The ultimate aim of ICT standardisation

  • The red thread running through the entire standards development process, it’s not an isolated issue

  • Not something to be somehow fixed at the end

  • The ability of systems and products to interwork is fundamental

  • ETSI approach

    • ETSI produces with the required degree of parallelism

      • Base Standards and

      • Test Specifications

    • Base standards should be designed with interoperability in mind

    • Profiles designed to reduce potential non-interoperability

    • Protocol conformance and Interoperability statements

    • Two complementary forms of testing

      • Conformance testing

      • Interoperability testing (formal IOT)

    • Testing provides vital feedback into the standardization work

Testing Methodology - ICT Workshop, Brazil March 2006


Slide3 l.jpg

ETSI Resources for Interoperability

  • Technical Committee MTS (Methods for Testing and Specification)

    • Development of methodologies, techniques and languages (including TTCN3) http://portal.etsi.org

  • ETSI’s Protocol and Testing Competence Centre (PTCC)

    • Supports ETSI committees on the application of formal techniques in standards on a daily basis

    • Development of test specifications (conformance and interop)

      http://www.etsi.org/ptcc

  • ETSI Plugtests™ Service

    • Validation of standards and prototypes through interoperability events http://www.etsi.org/plugtests

  • These resources are highly complementary

Testing Methodology - ICT Workshop, Brazil March 2006


Typical standards development process in etsi l.jpg

Industry

Interoperability Testing

(Unit) Conformance Testing

Interoperability Test Specifications

Iterative feedback

Conformance Test Specifications

Typical Standards Development Process in ETSI

Certification

Products mature from prototypes to commercial products

time

ETSI: Development of base standards

Testing Methodology - ICT Workshop, Brazil March 2006


Different kinds of etsi test specifications l.jpg

Conformance

Network Integration

Robustness

Performance

Interoperability

RF/EMC

Different Kinds of ETSI Test Specifications

Testing Methodology - ICT Workshop, Brazil March 2006


Progressive testing l.jpg

Progressive Testing

  • All engineering disciplines accept that testing should be applied to progressively complex units

    • From individual components to entire systems

  • This demands a number of different testing solutions

    • There is no silver bullet

  • In the two extremes

    • Just testing the components is not enough

    • Just testing the system is not enough

  • Limitations are economic, not technical

    • What can I afford to test?

    • What can I afford not to test?

  • What should be covered by standardised test specifications

  • What is the right kind of testing for the job

    • What kind of ‘thing’ is being tested? Device? System? Service?

    • What is the most economic method(s)?

    • What resources are (or must be made) available?

      • Tools, test scripts, testbeds etc.

Testing Methodology - ICT Workshop, Brazil March 2006


Conformance and interoperability testing are complementary l.jpg

Interop Testing

Conformance Testing

Interoperability

Conformance and Interoperability Testing are Complementary

  • ETSI experience

    • As you move up a system stack the emphasis should change from conformance to IOT

  • Lower layer protocols, infrastructure

    • Emphasis on conformance

  • Middleware, enablers

    • Combination of Conformance + IOT

  • Services, applications, systems

    • Emphasis on IOT

  • Conformance testing as a pre-requisite to IOT

Testing Methodology - ICT Workshop, Brazil March 2006


Conformance testing l.jpg

Device

Reqs.

Tester

Conformance Testing

  • Is Black-Box testing

    • Stimulation and Response

Point of Control and Observation (PCO)

Testing Methodology - ICT Workshop, Brazil March 2006


Conformance testing tests is usually layer bylayer l.jpg

IUT = Implementation Under Test

Upper Tester

MMI

API

L3

L3

Test

SUT

L2

System

d

i

g

i

t

a

l

PHY

1

2

3

4

5

6

7

8

9

8

#

*

Lower Tester

SUT = System Under Test

Conformance Testing Tests is Usually Layer-byLayer

i

i

t

a

l

Testing Methodology - ICT Workshop, Brazil March 2006


Characteristics of conformance testing 1 l.jpg

Characteristics of Conformance Testing (1)

  • Is unit testing

    • Tests a single ‘part’ of a device (e.g., a protocol layer)

      • Often incrementally – layer-by-layer

  • Tests against well-specified requirements

    • For conformance to the requirements in a base specification or profile (standard)

    • Usually limited to one requirement per test

  • Tests at a 'low' level

    • At the protocol (message/behaviour) level (bits)

      • Or service primitive, API, Interface

  • Tests over standardised interfaces

    • May not be available to ‘normal’ user

  • Requires a test system (and executable test cases)

    • Can be expensive (e.g., radio-based test system)

    • Tests under ideal conditions

Testing Methodology - ICT Workshop, Brazil March 2006


Characteristics of conformance testing 2 l.jpg

Characteristics of Conformance Testing (2)

  • High control and observability

    • Means we can explicitly test error behaviour

    • Can provoke and test non-normal (but legitimate) scenarios

    • Can be extended to include robustness tests

  • It is usually automated and tests are repeatable

  • Conformance Testing is DEEP and NARROW

    • Thorough and accurate but limited in scope

    • Gives a high-level of confidence that key components of a device or system are working as they were specified and designed to do

Testing Methodology - ICT Workshop, Brazil March 2006


Limitations of conformance testing l.jpg

Limitations of Conformance Testing

  • Does not prove end-to-end functionality (interoperability) between communicating systems

    • Conformance tested implementations may still not interoperate

      • This is often a specification problem rather than a testing problem! Need minimum requirements or profiles

  • Does not test a complete system

    • Tests individual system components, not the whole

      • A system is often greater than the sum of its parts!

      • Does not test functionality

        • Does not test the user’s ‘perception’ of the system

  • Standardised conformance tests do not include proprietary ‘aspects’

    • Though this may well be done by a manufacturer with own conformance tests for proprietary requirements

Testing Methodology - ICT Workshop, Brazil March 2006


Conformance testing documentation and process iso 9646 methodology l.jpg

logging and analysis

Test System

Product

Implementation Under Test

testing

Executable Test Suite

(e.g., C++)

Test Report

Industry

validation

implementation

compilation

Base Standard

(or Profile)

Conformance Testing Documentation and Process: ISO 9646 Methodology

Req. checklist

ICS

Test Purposes

TSS & TP

Test Suite (Test Cases)

ATS

Testing Methodology - ICT Workshop, Brazil March 2006


Implementation conformance statement ics l.jpg

Implementation Conformance Statement (ICS)

The ICS is a checklist of the capabilities supported by the Implementation Under Test (IUT)

Provides an overview of the features and options that are implemented in any given product

The ICS can be used to select and parameterise test cases and can be used as an indicator of basic interoperability between different products.

Conditional support e.g., Qn must be supported if Q1 supported then otherwise not.

Testing Methodology - ICT Workshop, Brazil March 2006


Profile ics l.jpg

Profile ICS

A profile ICS reflects how a (protocol) profile restricts the scope of a base standard by making certain options mandatory or excluding certain options.

Testing Methodology - ICT Workshop, Brazil March 2006


Selecting tests with the ics l.jpg

Selecting Tests with the ICS

A filled-in ICS can be used to select tests. For example, a test for an OPTIONAL feature which is not supported in a given product will be deselected from the test suite.

Testing Methodology - ICT Workshop, Brazil March 2006


Parameterizing tests with extra information for testing ixit l.jpg

Parameterizing Tests with eXtra Information for Testing (IXIT)

A filled-in IXIT can be used to parameterize tests. The Value column is used to specify explicit IUT or test system dependent values.

Testing Methodology - ICT Workshop, Brazil March 2006


Test purposes tp l.jpg

Test Purposes (TP)

Test Purposes (TP) are natural language, precise descriptions of the purpose of the test in relation to a particular (base standard) requirement

They are very focussed. They are not code.

A TP is WHAT to test not HOW to test

TPId:SIP_SS _PR_CE _V_012

Selection:Mandatory [Q2]

Precondition:Registration of both simulated UA to the IUT and an initiated session

Ref:5.1.1 [1], 17.3.6 [1], 17.4 [1] , 10.46.6 [1]

Purpose:Ensure that the IUT on receipt of a Server Internal Error (500 Server Internal Error) server failure response, sends an ACK request, with the same field Via and Branch parameter as in the previous INVITE, to the UAS and forwards it to the UAC.

Testing Methodology - ICT Workshop, Brazil March 2006


Test suite structure tss l.jpg

Test Suite Structure (TSS)

Registration

Registrant

Valid

Invalid

Registrar

:

Session

Originating Endpoint

Call Establishment

TP_S_OE_CE_01

TP_S_OE_CE_02

TP_S_OE_CE_03

:

Call Release

:

Terminating Endpoint

Proxy

Redirect Server

:

  • Test Purposes are grouped into a logical Test Suite Structure (TSS) according to suitable criteria

  • e.g., basic interconnection, error handling, functionality etc.).

  • The complete document is usually called the Test Suite Structure and Test Purposes (TSS&TP)

Testing Methodology - ICT Workshop, Brazil March 2006


Abstract test suite l.jpg

Abstract Test Suite

  • A Test Case is the ‘implementation’ of the test purpose

    • Is the HOW to test not WHAT to test

  • The Test Suite (ATS) is the collection of all Test Cases

  • Most ETSI Test Suites are written in TTCN

    • Predominantly in TTCN-2

    • Progression to TTCN-3 for new work

    • Not RF testing (generally not physical layer)

  • TTCN-3

    • Testing and Test Control Notation

    • Allows tests to be specified in detail (test code) that is independent of the (eventual) real test system on which it may be run

      • i.e., TTCN-3 will run on any test system that supports a standardised TTCN-3 execution environment

Testing Methodology - ICT Workshop, Brazil March 2006


Executable test suites l.jpg

Executable Test Suites

  • Executable test suites are compiled from the abstract

    • Either direct to binary, or

    • More commonly to some intermediate programming language (C++, Java, etc.)

  • ETSI does not deliver executables, but

    • We try to ensure all code can be compiled

    • And validated by executing the tests against a real implementation on at least one test system

      • 5 test systems in the case of UMTS, for example

Testing Methodology - ICT Workshop, Brazil March 2006


Interoperability testing l.jpg

Device2

Device1

Devicen

Interoperability Testing

  • End-to-End Interoperability of devices

    • What is being tested?

    • Assignment of verdicts?

  • Need to distinguish between

    • Formal interoperability testing

    • And informal interoperability events used to validate/develop technologies

Testing Methodology - ICT Workshop, Brazil March 2006


Interoperability testing tests functionality between devices l.jpg

1

2

3

1

2

3

4

5

6

4

5

6

7

8

9

7

8

9

8

#

8

#

*

*

MMI

MMI

SUT

API

API

QE = Qualified Equipment

EUT = Equipment Under Test

L3

L3

L2

L2

PHY

PHY

Conformance Monitoring

Interoperability Testing Tests Functionality Between Devices

QE

EUT

Means of Communication (MoC)

Interoperability Testing

(

of terminals)

Testing Methodology - ICT Workshop, Brazil March 2006


Characteristics of interoperability testing l.jpg

Characteristics of Interoperability Testing

  • Is system testing

    • Tests a complete device or a collection of devices

  • Shows that (two) devices interoperate

    • Albeit within a limited scenario

  • Tests at a ‘high’ level (as perceived by users)

    • Tests the ‘whole’, not the parts

      • e.g, protocol stacks + applications

    • Tests functionality

      • Shows function is accomplished (but not how)

  • Does not necessarily require a test system

    • Uses existing interfaces (standard/proprietary)

  • Interoperability Testing is BROAD and SHALLOW

    • Less thorough but wide in scope

    • Gives a high-level of confidence that devices (or components in a system) will interoperate with other devices (components)

Testing Methodology - ICT Workshop, Brazil March 2006


Limitations of interoperability testing l.jpg

Limitations of Interoperability Testing

  • Does not prove interoperability with other implementations with which no testing has been done

    • A may interoperate with B and B may interoperate with C. But it doesn’t necessarily follow that A will interoperate with C.

    • Combinatorial explosion

  • Does not prove that a device is conformant

    • Interoperable devices may still interoperate even though they are non-conformant

  • Cannot explicitly test error behaviour or unusual scenarios

    • Or other conditions that may need to be forced (lack of controllability)

    • Has limited coverage (does not fully exercise the device)

  • Not usually automated and may not be repeatable

Testing Methodology - ICT Workshop, Brazil March 2006


Interoperability test methodology l.jpg

Interoperability Test Methodology

  • ETSI TS 102 237-1

  • Uses best principles of ISO 9646 and adapts them for interoperability testing

  • Definitions

    • EUT – Equipemt Under Test

    • QE – Qualified Equipment

  • Concepts

    • IFS (like the PICS)

      • Interoperable Functions Statement

    • Generic testing architecture

    • Test Descriptions (in prose, tabular format)

    • If suitable interfaces (API/MMI) are exposed then can automate the tests by running TTCN-3 scripts

  • Processes

    • Basic test specification development process

Testing Methodology - ICT Workshop, Brazil March 2006


Interoperability testing documentation and process ts 102 237 methodology l.jpg

Test Report

Product 1

Qualified Equipment

Product 2

Equipment Under Test

logging and analysis

testing

implementation

implementation

Base Standard (or Profile)

Interoperability Testing Documentation and Process: TS 102 237 Methodology

Interop Test Cases

Interop Test Purposes

IFS (functional checklist)

Testing Methodology - ICT Workshop, Brazil March 2006


Test purpose language tplan l.jpg

Test Purpose Language (TPLan)

  • ETSI language, not necessarily restricted to IOP testing

  • Keywords and syntax provide clear and consistent structure

  • Keywords chosen for communications applications (sends,receivesetc.)

  • Text between keywords not part of syntax so free expression possible

  • A TP’s basic structure (corresponding keyword):

    • Header (TP id)

    • Pre-conditions (with)

    • Stimulus (when)

    • Expected response (then)

Testing Methodology - ICT Workshop, Brazil March 2006


Tplan example for interoperability l.jpg

TPLan Example for Interoperability

TP id : TP_COR_1719_02

Summary : 'EUT sends packet to All-RT LL M/cast address'

RQ ref : RQ_COR_1719

Config : CF_021_I

TD ref : TD_COR_1719_02

with { QE1'configured as a router'

andQE2'configured as a router'}

ensurethat {

when { EUT is requested to

'send packet to All-RoutersLink-Local M/cast addr' } then { QE1 indicates'receipt of the packet'

andQE2 indicates'receipt of the packet' }

}

Testing Methodology - ICT Workshop, Brazil March 2006


Interoperability test descriptions l.jpg

Interoperability Test Descriptions

  • Specify detailed steps to be followed to achieve stated test purpose

  • Steps are specified clearly and unambiguously without unreasonable restrictions on actual method:

    • Example:

      • Answer incoming call

        NOT

      • Pick up telephone handset

  • Written in a structured and tabulated natural language so tests can be performed manually

  • Can be automated using TTCN‑3 when EUT has software interfaces

Testing Methodology - ICT Workshop, Brazil March 2006


Example test description l.jpg

Example Test Description

Testing Methodology - ICT Workshop, Brazil March 2006


Automated iop testing using ttcn 3 l.jpg

AT Commands

AT Commands

Automated IOP Testing Using TTCN-3

Test

Controller

(TTCN-3)

Test

Driver

(TTCN-3)

Test

Driver

(TTCN-3)

Testing Methodology - ICT Workshop, Brazil March 2006


Network integration testing nit l.jpg

PoC

RADIUS

SIP

DIAMETER

Test Driver2

Test Driver1

Network Integration Testing (NIT)

  • End-to-end testing of the system

  • Components in the network must interoperate

SUT

Testing Methodology - ICT Workshop, Brazil March 2006


Example nit for ims l.jpg

Example NIT for IMS

Testing Methodology - ICT Workshop, Brazil March 2006


Slide35 l.jpg

Testing Methodology - ICT Workshop, Brazil March 2006


Ttcn 3 can be used for conformance and interoperability testing l.jpg

TTCN-3 can be used for Conformance and Interoperability Testing

Testing Methodology - ICT Workshop, Brazil March 2006


Benefits of ttcn 3 l.jpg

Benefits of TTCN-3

  • Specifically designed for testing

  • Concentrates on the test not the test system

  • Commonly understood syntax and operational semantics

  • Constantly maintained and developed

  • Off-the-shelf tools and TTCN-based test systems are readily available

  • Single language for many (all?) testing activities

    • Education and training costs can be rationalized

    • Maintenance of test suites (and products) is easier

  • Allows the application of a common methodology and style, both on a corporate level and within standardization

Testing Methodology - ICT Workshop, Brazil March 2006


Main capabilities of ttcn 3 l.jpg

Main Capabilities of TTCN-3

  • Dynamic concurrent testing configurations

  • Various communication mechanisms (synch and asynch)

  • Data and signature templates with powerful matching mechanisms (including regular expressions)

  • Specification of encoding information

  • Display and user-defined attributes

  • Test suite parameterization

  • Control of Test Case execution and selection mechanisms

  • Control of complex test configurations

  • Assignment and handling of test verdicts

  • Harmonized with ASN.1 (XML and IDL coming)

  • Different presentation formats

  • Well-defined syntax, static - and operational semantics

Testing Methodology - ICT Workshop, Brazil March 2006


The core language and other presentation formats l.jpg

Text format

Tabular Format

Graphical Format

PresentationFormat3

PresentationFormatn

The Core Languageand Other Presentation Formats

  • Core format is text based (most popular)

  • TTCN-3 can be edited or viewed in other formats

    • Tabular format (for TTCN-2 people)

    • Graphical format (good for visual overview)

    • Other standardized formats in the future?

    • Proprietary formats possible

TTCN-3 Core Language

Testing Methodology - ICT Workshop, Brazil March 2006


Example core text format l.jpg

Example Core (Text) Format

function PO49901(integer FL) runson MyMTC

{

L0.send(A_RL3(FL,CREF1,16))

TAC.start

alt {

[] L0.receive(A_RC1((FL+1) mod 2)) {

TAC.cancel

verdict.set(pass)

}

[] TAC.timeout {

verdict.set(inconc)

}

[] any.receive {

verdict.set(fail)

}

}

}

Testing Methodology - ICT Workshop, Brazil March 2006


Example graphical format l.jpg

Example Graphical Format

Testing Methodology - ICT Workshop, Brazil March 2006


Example tabular format l.jpg

Example Tabular Format

Testing Methodology - ICT Workshop, Brazil March 2006


Use of ttcn 3 with other languages l.jpg

ASN.1 Types & Values

IDL Types & Values

XML Types & Values

Other types & Valuesn

Use of TTCN-3 With Other Languages

  • TTCN can be integrated with other languages

  • Fully harmonized with ASN.1 (1997)

  • Harmonized with other languages

    • IDL, XML, C/C++

TTCN-3 Core Language

Testing Methodology - ICT Workshop, Brazil March 2006


Major elements of a ttcn 3 test suite l.jpg

Test Suite

Test Data Types

Test System Architecture

Actual Test Data

Test Behaviour

Major Elements of a TTCN-3 Test Suite

  • Test components

  • Test ports

  • Connections

    • between test components

    • to System Under Test

  • Configurations

    • static

    • dynamic

  • Actual test data (values) used during testing

  • Message templates

  • (Remote) signature templates

  • Matching mechanisms

    • values, lists, wildcards,

    • regular expressions etc.

  • Encoding information

  • Definition of data types for

  • Protocol Messages and

  • Information elements (fields, parameters)

  • Internal data (e.g., for computation)

  • Test coordination messages etc.

  • Predefined simple types

  • Integer, boolean, float

  • bitstring, hexstring, octetstring

  • charstring, universalcharstring

  • ... and complex types

  • record, recordof, set, set of

  • union, enumerated

  • ... and special types such as

  • verdict, address

test cases

test steps

default behaviour

management of test components

sending/receiving messages

verdict assignment

computation (e.g., checksums)

timers and timeouts

test execution and control

etc.

Testing Methodology - ICT Workshop, Brazil March 2006


Simple ttcn 3 architecture l.jpg

stimulus

response

Simple TTCN-3 Architecture

Implementation Under Test

Black-Box

TTCN-3 based Test

System

Point of Control and Observation (PCO)

Test System

IUT

Test port

Test port

Testing Methodology - ICT Workshop, Brazil March 2006


Ttcn 3 architecture test components l.jpg

Parallel Test

Component

(PTC)

Parallel Test

Component

(PTC)

IUT

Main Test

Component

(MTC)

TTCN-3 Architecture – Test Components

Test System

Internal Communication

Communication to IUT

Testing Methodology - ICT Workshop, Brazil March 2006


Ttcn 3 module l.jpg

Module Definitions

Module Control

Attributes

TTCN-3 Module

module Example

{

// Definitions part

control

{

// Control part

}

} with {encode "BER"}

Module (…)

Testing Methodology - ICT Workshop, Brazil March 2006


Definitions part l.jpg

Definitions Part

module Example

{

group TestArchitecture

{

// Port and Test Component definitions

}

group MessageTypes

{

// Defintions of message types

}

group ActualMessages

{

// Templates for instances of actual messages

}

group TestCases

{

// Test Case definitions

}

}

module Example

{

// Port and Test Component definitions

// Defintions of message types

// Templates for instances of actual messages

// Test Case definitions

}

Testing Methodology - ICT Workshop, Brazil March 2006


Specifying test messages l.jpg

Specifying Test Messages

type record MsgType

{

integer msgId

charstring msgName,

charstring msgInfo

}

template MsgType INVITE

{

msgId 0

msgName "INVITE",

msgInfo "For a good Brazilian Dinner"

}

template MsgType ACCEPT

{

msgId 1

msgName "ACCEPT",

msgInfo *

}

Testing Methodology - ICT Workshop, Brazil March 2006


Ttcn 3 behaviour l.jpg

P

INVITE

ACCEPT or DECLINE

TTCN-3 Behaviour

testcase TC1() runs on PTC1

{

P.send(INVITE)

T1.start(30E3)

alt

{

[]PCO.receive(ACCEPT)

{verdict.set(pass)}

[]PCO.receive(DECLINE)

{verdict.set(inconc)} []PCO.receive

{verdict.set(fail)}

[]T1.timeout

{verdict.set(fail)}

}

}

Testing Methodology - ICT Workshop, Brazil March 2006


The control part l.jpg

Module Definitions

Module Control

Attributes

The Control Part

module Example

{

// Definitions part

params { boolean Q1;

:

}

control

{

if Q1 then execute (TC1())

:

}

} with {encode "BER"}

Module (…)

Testing Methodology - ICT Workshop, Brazil March 2006


Ttcn 3 test system l.jpg

Parameterisation

Selection

User

TCI

TTCN-3 Control Interface

Test Suite in TTCN-3

(source)

PICS etc

(reqs. catalogue)

ENCODER

DECODER

(N-protocol specific)

TTCN-3 Test Suite

(object)

The Industry

Compilation

TRI

TTCN-3 Runtime Interface

Adaptation Layers

Control / Logging

Underlying Protocol Stack

(N-1)

Connection to the SUT

TTCN-3 Test System

Testing Methodology - ICT Workshop, Brazil March 2006


Ttcn 3 standards http www ttcn 3 org l.jpg

TTCN-3 Standardshttp://www.ttcn-3.org

  • ES 201 873-1 (Z.140)

    • TTCN-3 Core Language

  • ES 201 873-2 (Z.141)

    • TTCN-3 Tabular Presentation Format (TFT)

  • TR 101 873-3 (will eventually be ES 201 873-3) (Z.142)

    • TTCN-3 Graphical Presentation Format (GFT)

  • ES 201 873-4 (Z.143)

    • TTCN-3 Operational Semantics

  • ES 201 873-5

    • TTCN-3 Runtime Interface (TRI)

  • ES 201 873-6

    • TTCN-3 Control Interfaces (TCI)

  • ES 201 873-7 and upwards

    • Using ASN.1, XML, IDL, C/C++, with TTCN-3

Testing Methodology - ICT Workshop, Brazil March 2006


Concluding thoughts 1 l.jpg

Concluding Thoughts (1)

In the past, interoperability failures meant:

  • Bad publicity in trade magazines

  • Embarrassment for the manufacturer

  • Annoyance of the end customer

Today, interoperability failures in the field means:

  • Front page headlines in the Financial Times

  • Fall in manufacturers stock price

  • Loss of investor confidence

  • Unrecoverable damage to brand name

  • Irretrievable loss of customers

We can no longer afford to get it wrong! Testing is part of ETSI’s strategy to get it right!

Testing Methodology - ICT Workshop, Brazil March 2006


Concluding thoughts 2 l.jpg

Concluding Thoughts (2)

  • ETSI membership believes in methodical testing

  • A good example

    • GSM/UMTS is growing by 1 Million users per day!

    • Can you imagine what would happen if users were experiencing interoperability problems ?

    • More than 1 Million Euros invested per year for writing formal test specifications and test suites

    • We have experienced what it means to get it right!

  • A bad example

    • We have not always got it right!

    • For GPRS we did not take a formal approach to testing

    • First market products did not interoperate

  • We have learnt that testing is not cheap, but it pays in the long run

Testing Methodology - ICT Workshop, Brazil March 2006


Thank you l.jpg

Thank You!

Phone +33 (0)4 92 94 43 26

e-mailptcchelp@etsi.org

Websitewww.etsi.org/ptcc

Anthony Wiles

ETSI Protocol and Testing Competence Centre


  • Login