UMTS terminal testing: A practical perspective

UMTS terminal testing: A practical perspective Olaf Bergengruen May, 2003 Agere Systems - Optimay GmbH May, 2003

„We will not survive without a proper regression test system“, an Optimay engineer Agere Systems - Optimay GmbH May, 2003

Overview • Virtual testing • Logical view (what we want to test) • Why virtual testing • Requirements to the test system • 3GPP approach • Test model • Test cases • Optimay‘s implementation • GSM/GPRS virtual test system • GSM/GPRS/UMTS virtual test system Agere Systems - Optimay GmbH May, 2003

Test environment / logical view(conformance testing, FTA) Agere Systems - Optimay GmbH May, 2003

Test environment / logical view (2) Agere Systems - Optimay GmbH May, 2003

Why virtual test environment • We need a SW only environment (no specific HW) • HW is expensive or does not exist • HW debugging is difficult and time consuming • SW developers don‘t want to be bothered with HW issues or don‘t have the specific skills • We need to be much faster than real time • Developers want to use standard tools (compilers, editors, debuggers) Agere Systems - Optimay GmbH May, 2003

Requirements to the virtual test system • Test system shall be much faster than real time • Support the design of test scenarios at a high level of abstraction • All test suites need to run ‚over night‘: GSM/GPRS tests (GSM 11.10), UMTS test (3GPP 34.123), STK tests (GSM 11.10-4), MMI test cases and possibly other suites (TCP/IP, WAP, ...) • Test system shall be available to anybody, at anytime, on any PC Agere Systems - Optimay GmbH May, 2003

GSM/GPRS test architecture Agere Systems - Optimay GmbH May, 2003

Great approach: L1 simulated interface • Basic communication with the Tester via three functions: RxRadioBlock, TxRadioBlock, TimerTick • These functions can be extended as needed to simulate DSP code or other HW related functionality • MS is the master of system ‚ticks‘, about 100 times faster than real time • Tester and MS are tightly coupled which allows debugging using standard tools Agere Systems - Optimay GmbH May, 2003

Debugging GMM code ... static voidGMMDoRoutingAreaUpdate( GM_IE_UPDATE_TYPE RauType, BOOLEAN FirstAttempt ) { MMRAUpdPending = FALSE; GMRauType = RauType; GMStopT3302( ); GMStopT3311( ); if ( ! FirstAttempt ) { GMT3330Expiries = 0; if ( GMActState == GMActDetachInit ) { GMStopT3321( ); GMDetachPending = TRUE; } } if ( GMTriggerAction == GM_GS_ATTACH_COMP ) { /* Reinitialise the trigger action variable. */ GMTriggerAction = GM_NO_TRIGGER_ACTION; /* Confirm the end of the ATTACH procedure to GS */ SEND( GSGMEstCnf ) ( ); } Agere Systems - Optimay GmbH May, 2003

Highlights of our GSM/GPRS virtual test system • Extremely efficient performance (around 1300 tests in 20 minutes) • Includes testing of DSP code • About 30 man-years development effort (including test case development) Agere Systems - Optimay GmbH May, 2003

Drawbacks of our GSM/GPRS simulation • Tester uses proprietary (in house) test script languages and encoders which are difficult to maintain • It is difficult to write test cases • Tester is not nicely layered as shown in the previous slide, so it is difficult to extend and maintain • Summarizing: the system can not be extended to implement an UMTS System Simulator Agere Systems - Optimay GmbH May, 2003

3GPP test model(TS 34.123-3) Agere Systems - Optimay GmbH May, 2003

Why we need a Test Model • Main reason for us: If we comply with the 3GPP Test Model we can use the TTCN test cases developped here at ETSI / STF 160 • In general: • Clear and stable interfaces to the SS enables manufacturers to develop the test equipment • It reduces ambiguities within test case scenarios • It enables outsourcing of testing activities and SW re-use • It provides a common consens or understanding of the complete system to manufacturers and operators Agere Systems - Optimay GmbH May, 2003

A test case (1/2) ... • Configure SS • Configure PHY / L1 • Configure MAC • Configure RLC • Schedule and send System Information Blocks • Test case preamble • Bring UE into initial state (e.g. CS and PS registration) • Perform Location Update procedure • Perform GPRS Attach procedure Agere Systems - Optimay GmbH May, 2003

A test case (2/2) ... • Test case body • Stimulate the UE, e.g. • Send X message to the UE, or • Change cell power levels, or • Trigger the UE via AT command to initiate a call • Verify responses • Match messages from the UE to expected values • Assign a verdict (PASS, FAIL, INCONC) • Test case postamble • Complete signalling to bring UE into stable state Agere Systems - Optimay GmbH May, 2003

So, how to design a Virtual Tester which matches the 3GPP model and our requirements ? Agere Systems - Optimay GmbH May, 2003

Problems designing an overall GSM/GPRS/UMTS Test Env. • The clean 3GPP model is not enough for a complete MS test environment • We need to re-use tools, tracers, and most of all test cases developed for GSM in the new environment • We can not afford to develop the UMTS test suite in house, we need to re-use the 3GPP tests • The L1 and DSP simulation strategies for GSM do not match those of UMTS Agere Systems - Optimay GmbH May, 2003

Current approach (a compromise) • Multi-threaded system: The proper MS, the GSM SS, the UMTS SS and the SIM simulator • All communicating via shared buffers • The tick-master is the MS, when all jobs for current frame are completed, it issues a ‚tick‘ and all System Simulators prepare data (radio blocks) for next frame. • Each SS will send a ‚tick-ack‘ to the MS when it is ready for next frame • When all SSs are done: the MS L1 engine will fetch or store a radio block Agere Systems - Optimay GmbH May, 2003

GSM/GPRS/UMTS Test Environment Agere Systems - Optimay GmbH May, 2003

Summary • A Virtual Test Environment is an essential tool • for development • for quality assurance • 3GPP test model • A rough overview of our implementation Agere Systems - Optimay GmbH May, 2003

Thank you. Agere Systems - Optimay GmbH May, 2003

UMTS terminal testing: A practical perspective