test plan introduction l.
Skip this Video
Download Presentation
Test Plan: Introduction

Loading in 2 Seconds...

play fullscreen
1 / 10

Test Plan: Introduction - PowerPoint PPT Presentation

  • Uploaded on

Test Plan: Introduction. Primary focus: developer testing Implementation phase Release testing Maintenance and enhancement Secondary focus: formal system verification Addressed within current test plan via examples Assumed to be primarily independent. Tests to be Performed.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Test Plan: Introduction' - tammy

Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
test plan introduction
Test Plan: Introduction
  • Primary focus: developer testing
    • Implementation phase
    • Release testing
    • Maintenance and enhancement
  • Secondary focus: formal system verification
    • Addressed within current test plan via examples
    • Assumed to be primarily independent
tests to be performed
Tests to be Performed
  • Bottom-up integration testing
    • Build a module, build a test to simulate how it is used
  • Black-box
    • Based on specification and ‘educated guess’ stress tests
  • White-box
    • Based on code inspection
  • Platform testing
    • Establish baseline capability of hardware/OS, detect differences
  • Performance
    • Per module, and peer to peer distributed costs
  • Test coverage
    • Coverage level dependent on resources, time, and cost of failure
allocation of testing responsibilities
Allocation of Testing Responsibilities
  • Assumption: development team performs
    • Internal (module-level) performance
    • Sample system performance (limited example federation)
    • Full white-box
    • Limited black-box (basic system functionality)
  • Assumption: external group performs
    • Independent, detailed system verification (Spec compliant)
    • Standardized performance tests
  • Test coverage
    • Coverage will be measured, analyzed for relative effectiveness
    • Coverage levels TBD
testing philosophy
Testing Philosophy
  • Testing is not just pre-release! Continual process within development phase
  • Catch defects as early as possible
  • Make sure defects stay fixed
  • Track cause of defects: repair problem, do not keep re-patching the tire
  • Need support at both design level and implementation level to accomplish these goals
continual testing process
Continual Testing Process
  • Tests created during development
  • Central code repository: modules, test suites tied together
    • Tests are treated as live code to be maintained, not ‘once-offs’
    • Test documentation: how to run, what is tested, and why
  • Revision control on both modules and tests
  • Modular development
    • System broken down into hierarchies of testable components
  • Automated, incremental integration testing
    • As code is developed, it is tested standalone, then incrementally within the confines of the system
  • Continual feedback into development, maintenance cycle
    • Weekly postings of performance results, current defect status
design support
Design Support
  • Standard testing and debugging methods on each module / class (peek, dump, exercise, performance)
  • Self-checking code (pre and post parameter asserts, valid state calls)
  • Debug levels, controlled via runtime flags
  • Centralized logging mechanisms to collect distributed traces
  • Logs used in both developer testing and sample user traces from the field
development tool support
Development Tool Support
  • Shadow development trees for automated and individual testing (ClearCase ‘views’)
  • Common testing tools, testing approach to simplify testing process, and to allow automated testing
  • Examples:
    • Standard test harnesses, method of invocation
    • Standard testing directories, makefile commands per module
    • Standard set of test-record-evaluate tools
    • Central I/O mechanisms to collect distributed test results
    • Standard system-level drivers
    • Sequential test harnesses and emulated distributed harnesses to emulate determinism during development
levels of testing per module
Levels of Testing per Module
  • Basic: used in initial development, later in porting
    • Minimal level of functionality to establish module operation
    • Simplicity is critical during development, porting
  • Detailed: used to verify module correctness before checking into current baseline
    • Does module meet interface contract
    • Tests for common coding errors
  • Regression: replicates previous use which caused failure
    • Used to verify defect has been corrected
    • Used to ensure defect does not re-occur over time
  • Performance: tracks the CPU usage per module and peer to peer performance across distributed components
complex functional areas testing examples
Complex Functional Areas: Testing Examples
  • Memory: continual test for memory leaks and over-writes
    • Automated use of Purify within weekly test suites across all modules, all levels of the system
  • Threads: platform-specific performance and implementation variances must be established per platform
    • Standard set of tests which mimic system use of threads
  • Causality: complex areas of the system (such as zero-lookahead ordering) are difficult to establish correctness across all use cases (dropped packets, simultaneous time stamps with variable arrival times, cancelled events, …)
    • Detailed test scripts, executed in deterministic test harnesses with varying error conditions
periodic testing activities
Periodic Testing Activities
  • Code walkthroughs: encompass both system code and associated test suites. Testing focus:
    • Do the test suites sufficiently stress the module
    • Do the test suites still accurately represent the expected use of the module
    • Has the underlying platform changed, and has performance changed accordingly
  • Change Control Board: formal tracking process and tools to establish, record and monitor status of functional change requests, defect priority and status data