1 / 20

Test Implementation using TTCN-3 Libraries TTCN-3 User Conference, 7 th of June 2005

Test Implementation using TTCN-3 Libraries TTCN-3 User Conference, 7 th of June 2005. Stephan Schulz Validation & Testing Technologies Computing Architectures Lab Nokia Research Center Helsinki. Sebastian M ü ller FSCOM Sophia Antipolis France. Presentation Outline.

homer
Download Presentation

Test Implementation using TTCN-3 Libraries TTCN-3 User Conference, 7 th of June 2005

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Test Implementation using TTCN-3 LibrariesTTCN-3 User Conference, 7th of June 2005 Stephan Schulz Validation & Testing TechnologiesComputing Architectures LabNokia Research Center Helsinki • Sebastian Müller • FSCOMSophia AntipolisFrance

  2. Presentation Outline • About the ETSI IPv6 Library • Motivation • TTCN-3 Libraries • Library-based Test Suite Implementation • An Example TTCN-3 Library: ETSI Common • Conclusions

  3. About IPv6 Library • Implemented at ETSI by STF 276 • Conformance and interoperability test specs for IPv6 • Joint e-Europe/ETSI project • Promote deployment of IPv6 technology within Europe • Establish European player in IPv6 testing • Developed to support IPv6 Logo program by IPv6 Forum • Provide faster access to IPv6 market • Test implementations are publicly (open+free) available • Insure high deployment • Make tests transparent

  4. Motivation • Common problem in TTCN coding is the “copy & paste” mentality of test engineers • Lots of duplicated code + large code size = hard to maintain • Another problem: “The reinvention of the wheel“ • We desire a test suite implementation which allows as much reuse of TTCN-3 code withinas well as between different test suites as possible • Much of the code developed for the IPv6 Logo certification should be reusable also in a IPv6 test suite for, e.g., 3GPP! • A test suite should also be useful for additional test case development by other, external parties

  5. The Idea • Create a test suite using TTCN-3 libraries which can then be reused to compose also other related test suites • So just use TTCN-3 like a regular programming language! • Ability to reuse depends on making important TTCN-3 code structuring decisions early on in test suite development • Strong need for a sound methodology!

  6. Benefits of TTCN-3 libraries • Less test suite specific TTCN-3 code • Less room for errors in test implementations • Use of validated TTCN-3 code segments • “More readable” test implementations • Less time needed for test case implementation and validation • Avoids the “reinvention of the wheel” • Summary: Higher quality test suite implementations!

  7. Keys to Reusable TTCN-3 Code • Follow the TTCN-3 Framework • Methodology for Test Suite Implementation • Modularization • Rules on how to implement TTCN-3 behavior • Naming conventions • Specify well defined interfaces • Use component type compatibility • Document TTCN-3 code • Standard practice in any programming language • Free documentation tools are available

  8. Libraries in TTCN-3 • A TTCN-3 library consists of modules separating • behavior(s) • types and constants • templates • Example: Libraries in the IPv6 test suite • Common IPv6 independent code (e.g, Synchronization) • Common IPv6 code (e.g., Echo Procedure) • Common IPv6 test suite code (e.g., Ipv6 Host Preamble ) But then (not reusable): • Specific IPv6 test suite code (e.g., IPv6 Core Test Cases)

  9. Basic Structuring of IPv6 Test Cases • Decompose test case behavior based on well established methodology • Configuration (CF) function • for test configuration set up • Preamble (PR) function • get IUT into testing state • Test body (TP) function • perform test as specified by test purpose • Postamble (PO) function • return IUT to its initial state • All of the above implemented by invoking test suite, IPv6, or common library functions! • Implement test case or test case (TC) functions by simply invoking PR, TP, PO functions

  10. TC functions in Single and Multiple Interfaces create/start preamble sync testcase TP sync (PR) function (MTC) (TP) function (PR) function (PR) function testcase (TP) function (TP) function (PO) function (TC) function (TC) function (MTC) (PO) function (PO) function (PTC1) (PTCn) • Specify test case (TC) functions for each test component • Use MTC in concurrent test cases only to spawn and synchronize PTCs No Test case (TC) functions needed

  11. Rules for reusable TTCN-3 Behavior • Function should not contain any stop statement since it prevents the execution of postambles! • Test suite independent functions should avoidto use the setverdict operation • Instead use the return value of a function! • Verdict setting harms reuse, e.g., testing of valid versus invalid behavior • A module which defines reusable behavior should contain a test component type definition tailored its needs

  12. About TTCN-3 Code Documentation • Without proper documentation reuse is questionable • Documentation of library code is well established standard practice in other programming languages (e.g., man pages) • It should be possible to use a TTCN-3 library or modules without ever reading source files • Any TTCN-3 code should be documented regardless of its reusability! • Guidelines part of framework document • A free documentation tool t3doc is available: • Follows ideas of similar tools for C and Java, e.g., doxygen • Generates HTML pages from TTCN-3 code comments • WWW tool download: www.farodyne.com\t3doc

  13. A Library Example: ETSI Common • Three modules with common type and value definitions • Common basic, data string and text string constant and type definitions, e.g., c_CRLF, UInt, etc. • Module for verdict control handling • Definition of function return code and verdict setting functions • Module for timing related functions, e.g., f_sleep • Modules which implement a generic test component synchronization as well as sync examples • Our classic “reinvention of the wheel” 

  14. The Synchronization Library Module • Implements an easy-to-use synchronization handling for both single as well as multi component test cases • Key concepts • Named synchronization points • TTCN-3 default used for handling shutdown • Uses extensively TTCN-3 log statement • Results in more readable execution traces! • Current main user API:f_selfSync(syncPointName, retCode) f_serverSyncClients(#, syncPointName) f_serverSyncClientsTimed(#, syncPointName, time) f_serverWaitForAllClientsToStop() f_clientSync(syncPointName, retCode) [a_dummyShutDown()] // shutdown example

  15. Ready! Ready! Ready! Ready. Ready! Ready. Ready. Ready. (PTC1) (PTC2) f_tc1_ptc1() f_tc1_ptc2() Example of Nice Multi-Component Sync (MTC) create/start PTC1 and PTC2 f_serverSyncClients(2,“prDone”) tc1() f_serverSyncClients(2,“tpDone”) f_serverWaitForAllClientsToStop() f_PR_tc1_ptc1() f_PR_tc1_ptc2() f_clientSync(“prDone”,e_success); f_clientSync(“prDone”,e_success); f_TP_tc1_ptc1() f_TP_tc1_ptc2() f_clientSync(“tpDone”,e_success); f_clientSync(“tpDone”,e_success); f_PO_tc1_ptc1() f_PO_tc1_ptc2()

  16. STOP! Ready! STOP! STOP! (PTC1) (PTC2) f_tc1_ptc1() f_tc1_ptc2() Example of Sync Problem Handling (MTC) create/start PTC1 and PTC2 f_serverSyncClients(2,“prDone”) tc1() f_wait4AllClientsToShutDown() f_PR_tc1_ptc1() f_PR_tc1_ptc2() f_clientSync(“prDone”,e_error); f_clientSync(“prDone”,e_success); a_shutDown_if1()(calls stop!) a_shutDown_if2()(calls stop!)

  17. STOP! Example of Self Synchronization (MTC) f_PR_tc2() f_selfSync(“prDone”,e_success); f_TP_tc2() tc2() f_selfSync(“tpDone”,e_error); a_shutDown_if1()(calls stop!) • Note that shutdown defaults are neither test case nor component but only interface specific (e.g, IPv6, SIP, etc.) • Therefore the same default can be used to shutdown any test component (which uses that interface) in any test case.

  18. Conclusions • This really works in practice • The ETSI common TTCN-3 library has already been successfully deployed within STF 276 as well as a number of different protocol test suites within Nokia • First versions of the generic IPv6 and IPv6 Core libraries have been successfully used in the execution of the ETSI IPv6 test suite prototype against a Linux Ipv6 SUT • At the end the real key to implementing truly reusable TTCN-3 code are • TTCN-3 framework • Well-defined interfaces • TTCN-3 code documentation

  19. References • http://www.ipt.etsi.org • More info about this STF and documents • Test suite and executable download • ETSI DTS/MTS-IPT-0001, “Methods for Testing and Specification (MTS); Internet Protocol Testing (IPT); IPv6 Testing: Methodology and Framework”, Sophia-Antipolis, 2005. • S. Moseley et al., “IPv6 Test Specifications from ETSI”, Global Ipv6 Summit, Barcelona, (to appear), June, 2005.

  20. ! function f_PR_EchoProcUp(…)runs on Ipv6Comp { var FncRetCode v_ret; v_ret := f_echoProc(); f_clientSync(“prDone”, v_ret);} function f_clientSync(…)runs on SyncComp {// some code} OK! About Component Type Compatibility • Key to create self contained libraries is the runs on clause with TTCN-3 functions • An IPv6 preamble may invoke, e.g., a synchronization function, if its component type definition is compatible type component Ipv6Comp { IpPort pt_ip; SyncPort pt_sync;timer t_sync;} type component SyncComp { SyncPort pt_sync;timer t_sync;}

More Related