time boxed testing n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
TIME BOXED TESTING PowerPoint Presentation
Download Presentation
TIME BOXED TESTING

Loading in 2 Seconds...

play fullscreen
1 / 39

TIME BOXED TESTING - PowerPoint PPT Presentation


  • 101 Views
  • Uploaded on

TIME BOXED TESTING. BCS SIGIST 13 th July 1998 Graham Thomas - OSI Group. Abstract.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'TIME BOXED TESTING' - neviah


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
time boxed testing

TIME BOXED TESTING

BCS SIGIST

13th July 1998

Graham Thomas - OSI Group

abstract
Abstract

There is great pressure upon developers today to improve productivity and effectiveness. To achieve this there is a move away from the traditional structured methodologies towards more dynamic, iterative and RAD approaches.

This is being combined with Object and Component based techniques, and delivered with a new generation of IDE’s, to produce thin client, web based, voice and data products.

So how do we test it?

  • Propose Time Boxed Testing
  • Outline an approach
  • Discuss the problems
  • Give a flavour of the fun ?
agenda
Agenda
  • Development Lifecycle
  • Testing Lifecycle
  • Testing Helix
  • Testing Stage by Stage
  • Traceability
  • Testing Process
  • How to sell time boxed testing
  • Lessons Learnt
the project
The Project
  • Telephony based project
  • 2 types of proprietary hardware
  • Client Server running Windows NT
  • Interfaces to legacy systems
  • Rational Rose case tool (using UML)
  • Objectory process
  • Visual Studio
  • Was considering Java & thin clients
  • Incremental/Iterative development
incompatible models

Rqmts.

UAT

Sys Test

Analysis

Depth

Design

Int Test

Build

Unit Test

Breadth

Classic V Model

Iterative (RAD) Model

Incompatible Models?
difficult to represent test stages

Component

Integration

unit

unit

Component

Integration

Depth

unit

Component

Integration

System

Breadth

Acceptance

Difficult to Represent Test Stages
a dual spiral life cycle
A dual spiral life cycle ?

¬

­

®

R

B

. . .

P

E

A

D

Rqmts

Analysis

Design

Build

D

C

Plan

Design

Construct

Execute

testing lifecycle1
Testing Lifecycle
  • Unit Testing
  • Component Integration Testing
  • System Testing
  • Acceptance Testing
  • Year 2000 Millenium Compliance
  • Testing Processes and Activities
  • Tools & Traceability
  • Resources
  • Planning
scope of test phases
Scope of Test Phases

Conical

Model

Component

Integration

Testing

Unit

Testing

User

Acceptance

Testing

System

Testing

Release

Iterative

tuned test lifecycle
Tuned Test Lifecycle

Testing Activities Brought Forward

System Testing

- Non functional

Requirements

- Performance

Analysis

User Acceptance Testing

- Scenario Testing

- Acceptance

Criteria

Component

Integration

Testing

Unit

Testing

Release

Iterative

unit testing

Sub Unit

Testing

Unit Testing

Unit

Integration

Testing

Unit

Testing

External

Interface

Testing

Static

Analysis

Dynamic

Analysis

Quality

Assurance

Unit Testing
  • Iterative Time-Boxed Activity
  • Specific to delivered components
  • ‘White Box’ testing techniques employed
  • External Interfaces tested
  • Use of tools for Static & Dynamic analysis
unit testing1

Unit

Package

Sub

Unit

Unit

Sub

Unit

Sub

Unit

Package

Sub

Unit

Unit

Unit

Sub

Unit

Sub

Unit

Unit

Sub

Unit

Sub

Unit

Package

Sub

Unit

Unit

Unit Integration Testing

Sub Unit Testing

Unit Testing

Previously tested & checked-in

Stubs, Drivers, Harnesses

Unit Testing
component integration testing

Package

Integration

Component

Integration

Performance

Analysis

Component

Integration

Testing

Non Functional

Requirements

Functional

Requirements

Regression Test

Scenario

Regression

Test

Component Integration Testing
  • Iterative Time-Boxed Activity
  • Testing prioritised by Business
  • End to End process threads where possible
  • Contains elements of following test phases
  • Build regression test packs
component integration testing1

Component

Package

Package

Component

Package

Package

Component

Component Integration Testing

Step 1 - Package Integration

Package Integration must precede Component Integration

component integration testing2

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Drop1

Drop2

. . .

Component Integration Testing

Step 2 - Component Integration

component integration testing3

®

­

¬

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

. . .

Component Integration Testing

Drop by Drop

Automated Regression Test Pack

system testing

Non Functional

Requirements

Functional

Requirements

Regression Test

System

Testing

Performance

Analysis

Design

Verification

System Testing
  • One off activity
  • Structured testing techniques
  • End to End process threads
  • Ties together the development iterations
  • Regression test already constructed
acceptance testing

Scenario

Regression

Testing

Acceptance

Criteria

Testing

Acceptance

Testing

Business

Process

Testing

Data Take On

& Conversion

Testing

Usability

Testing

Acceptance Testing
  • One off activity
  • Test that the computer system integrates with business processes
  • Structured testing techniques
  • End to End process threads
  • Regression test already constructed
y2000 millennium compliance
Y2000 - Millennium Compliance

6 Date Scenarios

  • Separate test runs
  • Carried out in sequence
  • Some tests can be exercised post Y2000

Across

Century

Boundary

Wholly

next

Century

This

Century

Jan 1st

Feb 29th

Dec 31st

2 Business Tests

Financial

Year ?

Fiscal

Periods ?

testing lifecycle traceability

User

Requirements

Development

Repository

TestPlanning

Testing Lifecycle Traceability

High Level Function

Functional

Requirement

Non FR

Use Case

Scenario

Test Script

Test Case

testing processes and activities

Process

Quality

Interface

Problem

Mgmt

Conf. Mgmt.

Interface

Activity

Entry

Testing Processes and Activities

Test Management

Test Stage

Sign off

Test Execution

Test Automation

Regression Test

Test Reporting

Monitor

Measure

Manage

Exit

Test Planning

Test Design

Test Preparation

Analysis of Change

processes
Processes
  • Essential
    • Configuration Management
    • Impact Analysis using a Test Planning repository
    • Quality Management
  • Good practice
    • Fault Reporting - on the Intranet - Live
    • Gathering metric information
tools
Tools
  • Full set of testing tools is required
    • Static analysis
    • Dynamic Analysis
    • Interactive debugging
    • Test Automation/Capture Replay essential
    • Test Planning Repository
    • Performance monitoring (stress/load)
    • Fault Reporting system
test resources
Test Resources

Component

Integration

Testing

User

Acceptance

Testing

Unit

Testing

System

Testing

TEST

TEAM

Business

Users

Business

Users

Developers

Developers

Testers

Secondment

test planning

Test Approach

Decision

Test Strategy

Definition

Testing Process

Test drop ¬

Test drop ­

Development

Test drop ®

Non Functional Testing

System Test

Acceptance Test

Delivery

Y2000 Test

Time-boxed Testing

Test Planning
selling the approach
Selling the Approach
  • Target the following;
    • Test Team
    • Systems Architect
    • Development team
    • Standards & Methods
    • Business Representatives
    • Project Sponsor
  • Via
    • Animated Presentation
    • Test Strategy document
so what went well
So What went well?
  • Systems Architect liked it
  • The presentation was well received
what was difficult
What was difficult?
  • Initial cool response from other members of the test team
  • Development Team were uneasy
    • Dev. Why test anyway, our code is wonderful and our test harnesses will do all the testing?
  • Business Representatives were unhappy about;
    • Anything less than 100% testing
    • Bringing their effort forward in the development lifecycle
    • The difference between the ‘Use Case’ modeling they had done, and the test planning that they would need to carry out
what was difficult 2
What was difficult? (2)
  • Standards & Methods

Saw nothing wrong with their approach, it was either;

      • structured in which case their approach worked fine,

or

      • it was incremental, in which case their approach worked fine.
      • They did not do iterative development, but if they did their approach would still work.

This took several meetings to gain approval.

  • Project Sponsor
    • Cancelled the project. (Time Boxed Testing was not cited as the reason !!!)
what did i learn from this
What did I learn from this?
  • Organisations change their development approach more quickly than their testing approach
  • There is a reluctance to do anything less than 100% testing
  • Testing is a key and integral part of any development lifecycle
  • The more you think about it, the more sense Time BoxedTesting makes
contact information
Contact Information

Graham Thomas

OSI

the management support company

e-mail : gthomas@osi.co.uk

Phone : 0171 323 3353

Copies of slides including animated gifs at

www.osi.co.uk