Ocap ri vendor forum
Download
1 / 25

OCAP-RI Vendor Forum - PowerPoint PPT Presentation


  • 137 Views
  • Uploaded on

OCAP-RI Vendor Forum. August 10, 2011 Presenters: NDS and CableLabs Dev Teams Location: CableLabs Summer Conference – Keystone, CO. Agenda. Introductions Ground Rules Release Roadmap NDS Topics CableLabs Topics Miscellaneous Topics. Production / Debug Builds.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' OCAP-RI Vendor Forum' - wyanet


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Ocap ri vendor forum

OCAP-RI Vendor Forum

August 10, 2011

Presenters: NDS and CableLabs Dev Teams

Location: CableLabs Summer Conference – Keystone, CO


Agenda
Agenda

Introductions

Ground Rules

Release Roadmap

NDS Topics

CableLabs Topics

Miscellaneous Topics


Production debug builds
Production / Debug Builds

CableLabs occasionally tests releases in both debug and “release” build configurations

In “release”, compiler optimizations are enabled and much logging is disabled

Changes in timing can exponse thread synchronization bugs in the stack

Harder to debug


Extension based builds
Extension-based Builds

Stack can be configured with OCAP base and any combination of OCAP extensions

DVR, HomeNetworking, FrontPanel, etc

Application developers must be careful to structure their apps so as not to trigger accidental class loading

Class file verification can trip you up

JVM spec (http://java.sun.com/docs/books/jvms/second_edition/html/ClassFile.doc.html)


Extension based app example
Extension-based app example

public class App

{

public void doSomething()

{

MyInterface myIf;

if (System.getProperty(“ocap.api.option.dvr”) != null)

{

myIf = new MyIntefaceDVRImpl();

}

else

{

myIf = new MyIntefaceBaseImpl();

}

}

}


Testing overview
Testing Overview

QA - Bug tracking

Trunk and 1.1.4 Branch

Unit Testing

CTP attended/unattended, CTP smoke, atelite, DLNA & UPnP CTT, JUnit

Integration Testing

Integration Xlets, AutoXlet, CATS TDK

MSO Guides – ODN, Trio, Barcelona

QA: TestLink

We want your tests - Appeal for your tests - Please


Bug tracking
Bug Tracking

Private Database - OCORI

A mix of publicly visible issues, and some are not

To protect confidential information, exposed case by case

Mirrors the issues in the private data

Public database - IT

For these issues, all information in the private databases is also in the public database


Trunk and branch testing schedule
Trunk and Branch Testing Schedule

Trunk and 1.1.4 vendor branch

Daily Guide smoke tests (trunk)

Weekly Guide smoke tests (1.1.4 branch)

Weekly CTP test runs

Full regression run prior to releases

Internal Branch Testing

CTP tests are run prior to merging back into the trunk


CTP Unattended Tests

  • Test Setup

  • 5600+ unattended CTP tests.

  • 1 ATE dedicated to testing the RI.

  • Tests run against Windows and Linux platforms.

  • Run against all test suites requires 3 days.

  • Test Schedule

  • Test all suites against the trunk (frequency: once per week).

  • Test all suites against the 1.1.4 branch (frequency: once per week).

  • Test all suites or a subset of tests against developers branches before merge to trunk (frequency: as needed).

  • Test subset of suites against separable builds: Base, DVR only, HN+DVR, HN only (frequency: when ATE available).


Attended ctp tests

Attended CTP Tests

400+ CTP tests require operator interaction

Attended CTP tests can be roughly divided into two groups:

Tests which demonstrate interactivity.

Tests which require visual inspections.

Automated attended CTP testing addresses the tests which require only visual inspection.

Test run is automated.

Screenshot and verification question are captured and saved for each test.

After test run completes, operator analyzes saved screenshots and verification questions.

Goal of automated attended CTP testing

Increase scope of tests run for each release cycle while using QA resources more efficiently.


Automated Attended CTP Testing utilizes programs developed by the RI team to capture test results, incorporate results into a spreadsheet and pull aggregated results in to a GUI for analysis by QA.


Dlna ctt v 1 5 57 03
DLNA CTT v 1.5.57.03 by the RI team to capture test results, incorporate results into a spreadsheet and pull aggregated results in to a GUI for analysis by QA.

Unit tests which check the conformance of the Ri against the DLNA specification

Run the DMS (server) tests. Player tests are restricted by PC features supported by the RI.

267 unit level tests are run.


Upnp ctt 2 0 18 5079
UPnP CTT 2.0.18.5079 by the RI team to capture test results, incorporate results into a spreadsheet and pull aggregated results in to a GUI for analysis by QA.

Unit tests which check the conformance of the Ri against the UPnP specifications

110 unit level tests are run.


Junit tests
JUnit Tests by the RI team to capture test results, incorporate results into a spreadsheet and pull aggregated results in to a GUI for analysis by QA.

Modified version of the JUnit test runner

Tests are available as part of the distribution

Executed by development engineers as part of their workflow.


Integration tests
Integration Tests by the RI team to capture test results, incorporate results into a spreadsheet and pull aggregated results in to a GUI for analysis by QA.

$OCAPROOT/apps/qa/org/cablelabs/xlets

Each test application should have a readme.txt in its directory.

Currently undergoing clean-up and rework

Latest status is found in $OCAPROOT/apps/qa/docs/IntegrationTestInfo.doc

New test applications will be added going forward for integration testing of new features.


List of tests run with every RI release: by the RI team to capture test results, incorporate results into a spreadsheet and pull aggregated results in to a GUI for analysis by QA.


Integration test automation
Integration Test Automation by the RI team to capture test results, incorporate results into a spreadsheet and pull aggregated results in to a GUI for analysis by QA.

Autoxlet – a framework that automates xlet launching and dispatch of remote control events.

XletDriver – AUTOSTART application that loads xlets and sends remote control events based on the XML script file.

To automate a xlet, the xlet should implement the Driveable interface.

DispatchEvent(…) provides a method for receiving and interpreting key events.

Uses IXC communication for logging and result reporting.

Deterministic xlet results are provided using JUnit-like asserts.

$OCAPROOT/apps/qa/Autoxlet/


Mso guide tests
MSO Guide Tests by the RI team to capture test results, incorporate results into a spreadsheet and pull aggregated results in to a GUI for analysis by QA.

Guides

ODN

Barcelona

Trio

Run a subset of the tests used by the MSOs for their own testing


Home networking interop testing
Home Networking InterOp Testing by the RI team to capture test results, incorporate results into a spreadsheet and pull aggregated results in to a GUI for analysis by QA.

List scenarios run at the May Interop

Discover and playback RI recordings on your STB

RI discover and playback recordings from your STB

Trick modes

Delete recordings


Tdk and cats
TDK and CATS by the RI team to capture test results, incorporate results into a spreadsheet and pull aggregated results in to a GUI for analysis by QA.

Will run the same tests as NDS talked about earlier

At this moment, still working through installation issues

Our goal is to run all the TDK tests prior to each RI release


Testlink
TestLink by the RI team to capture test results, incorporate results into a spreadsheet and pull aggregated results in to a GUI for analysis by QA.

Web based test case management tool

Archive of test results

Repository for test procedures


Testlink results entry
TestLink Results Entry by the RI team to capture test results, incorporate results into a spreadsheet and pull aggregated results in to a GUI for analysis by QA.


Testing overview summary
Testing Overview - Summary by the RI team to capture test results, incorporate results into a spreadsheet and pull aggregated results in to a GUI for analysis by QA.

QA - Bug tracking

Trunk and 1.1.4 Branch

Unit Testing

CTP attended/unattended, CTP smoke, atelite, DLNA & UPnP CTT, JUnit

Integration Testing

Integration Xlets, AutoXlet, CATS TDK

MSO Guides – ODN, Trio, Barcelona

QA: TestLink

We want your tests - Appeal for your tests - Please


Questions and suggestions
Questions and Suggestions by the RI team to capture test results, incorporate results into a spreadsheet and pull aggregated results in to a GUI for analysis by QA.

RI QA is a small team with limited resources

Do you have tests to contribute?

Do you have suggestions for new tests?


ad