Ocap ri vendor forum
This presentation is the property of its rightful owner.
Sponsored Links
1 / 25

OCAP-RI Vendor Forum PowerPoint PPT Presentation


  • 103 Views
  • Uploaded on
  • Presentation posted in: General

OCAP-RI Vendor Forum. August 10, 2011 Presenters: NDS and CableLabs Dev Teams Location: CableLabs Summer Conference – Keystone, CO. Agenda. Introductions Ground Rules Release Roadmap NDS Topics CableLabs Topics Miscellaneous Topics. Production / Debug Builds.

Download Presentation

OCAP-RI Vendor Forum

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Ocap ri vendor forum

OCAP-RI Vendor Forum

August 10, 2011

Presenters: NDS and CableLabs Dev Teams

Location: CableLabs Summer Conference – Keystone, CO


Agenda

Agenda

Introductions

Ground Rules

Release Roadmap

NDS Topics

CableLabs Topics

Miscellaneous Topics


Production debug builds

Production / Debug Builds

CableLabs occasionally tests releases in both debug and “release” build configurations

In “release”, compiler optimizations are enabled and much logging is disabled

Changes in timing can exponse thread synchronization bugs in the stack

Harder to debug


Extension based builds

Extension-based Builds

Stack can be configured with OCAP base and any combination of OCAP extensions

DVR, HomeNetworking, FrontPanel, etc

Application developers must be careful to structure their apps so as not to trigger accidental class loading

Class file verification can trip you up

JVM spec (http://java.sun.com/docs/books/jvms/second_edition/html/ClassFile.doc.html)


Extension based app example

Extension-based app example

public class App

{

public void doSomething()

{

MyInterface myIf;

if (System.getProperty(“ocap.api.option.dvr”) != null)

{

myIf = new MyIntefaceDVRImpl();

}

else

{

myIf = new MyIntefaceBaseImpl();

}

}

}


Testing overview

Testing Overview

QA - Bug tracking

Trunk and 1.1.4 Branch

Unit Testing

CTP attended/unattended, CTP smoke, atelite, DLNA & UPnP CTT, JUnit

Integration Testing

Integration Xlets, AutoXlet, CATS TDK

MSO Guides – ODN, Trio, Barcelona

QA: TestLink

We want your tests - Appeal for your tests - Please


Bug tracking

Bug Tracking

Private Database - OCORI

A mix of publicly visible issues, and some are not

To protect confidential information, exposed case by case

Mirrors the issues in the private data

Public database - IT

For these issues, all information in the private databases is also in the public database


Trunk and branch testing schedule

Trunk and Branch Testing Schedule

Trunk and 1.1.4 vendor branch

Daily Guide smoke tests (trunk)

Weekly Guide smoke tests (1.1.4 branch)

Weekly CTP test runs

Full regression run prior to releases

Internal Branch Testing

CTP tests are run prior to merging back into the trunk


Ocap ri vendor forum

CTP Unattended Tests

  • Test Setup

  • 5600+ unattended CTP tests.

  • 1 ATE dedicated to testing the RI.

  • Tests run against Windows and Linux platforms.

  • Run against all test suites requires 3 days.

  • Test Schedule

  • Test all suites against the trunk (frequency: once per week).

  • Test all suites against the 1.1.4 branch (frequency: once per week).

  • Test all suites or a subset of tests against developers branches before merge to trunk (frequency: as needed).

  • Test subset of suites against separable builds: Base, DVR only, HN+DVR, HN only (frequency: when ATE available).


Attended ctp tests

Attended CTP Tests

400+ CTP tests require operator interaction

Attended CTP tests can be roughly divided into two groups:

Tests which demonstrate interactivity.

Tests which require visual inspections.

Automated attended CTP testing addresses the tests which require only visual inspection.

Test run is automated.

Screenshot and verification question are captured and saved for each test.

After test run completes, operator analyzes saved screenshots and verification questions.

Goal of automated attended CTP testing

Increase scope of tests run for each release cycle while using QA resources more efficiently.


Ocap ri vendor forum

Automated Attended CTP Testing utilizes programs developed by the RI team to capture test results, incorporate results into a spreadsheet and pull aggregated results in to a GUI for analysis by QA.


Dlna ctt v 1 5 57 03

DLNA CTT v 1.5.57.03

Unit tests which check the conformance of the Ri against the DLNA specification

Run the DMS (server) tests. Player tests are restricted by PC features supported by the RI.

267 unit level tests are run.


Upnp ctt 2 0 18 5079

UPnP CTT 2.0.18.5079

Unit tests which check the conformance of the Ri against the UPnP specifications

110 unit level tests are run.


Junit tests

JUnit Tests

Modified version of the JUnit test runner

Tests are available as part of the distribution

Executed by development engineers as part of their workflow.


Integration tests

Integration Tests

$OCAPROOT/apps/qa/org/cablelabs/xlets

Each test application should have a readme.txt in its directory.

Currently undergoing clean-up and rework

Latest status is found in $OCAPROOT/apps/qa/docs/IntegrationTestInfo.doc

New test applications will be added going forward for integration testing of new features.


Ocap ri vendor forum

List of tests run with every RI release:


Integration test automation

Integration Test Automation

Autoxlet – a framework that automates xlet launching and dispatch of remote control events.

XletDriver – AUTOSTART application that loads xlets and sends remote control events based on the XML script file.

To automate a xlet, the xlet should implement the Driveable interface.

DispatchEvent(…) provides a method for receiving and interpreting key events.

Uses IXC communication for logging and result reporting.

Deterministic xlet results are provided using JUnit-like asserts.

$OCAPROOT/apps/qa/Autoxlet/


Mso guide tests

MSO Guide Tests

Guides

ODN

Barcelona

Trio

Run a subset of the tests used by the MSOs for their own testing


Home networking interop testing

Home Networking InterOp Testing

List scenarios run at the May Interop

Discover and playback RI recordings on your STB

RI discover and playback recordings from your STB

Trick modes

Delete recordings


Tdk and cats

TDK and CATS

Will run the same tests as NDS talked about earlier

At this moment, still working through installation issues

Our goal is to run all the TDK tests prior to each RI release


Testlink

TestLink

Web based test case management tool

Archive of test results

Repository for test procedures


Testlink results entry

TestLink Results Entry


Testing overview summary

Testing Overview - Summary

QA - Bug tracking

Trunk and 1.1.4 Branch

Unit Testing

CTP attended/unattended, CTP smoke, atelite, DLNA & UPnP CTT, JUnit

Integration Testing

Integration Xlets, AutoXlet, CATS TDK

MSO Guides – ODN, Trio, Barcelona

QA: TestLink

We want your tests - Appeal for your tests - Please


Questions and suggestions

Questions and Suggestions

RI QA is a small team with limited resources

Do you have tests to contribute?

Do you have suggestions for new tests?


  • Login