slide1 l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Don’t Just “Test”… Validate!! PowerPoint Presentation
Download Presentation
Don’t Just “Test”… Validate!!

Loading in 2 Seconds...

play fullscreen
1 / 45

Don’t Just “Test”… Validate!! - PowerPoint PPT Presentation


  • 111 Views
  • Uploaded on

Don’t Just “Test”… Validate!!. Agenda. Overview of Testing versus Validation Common Activities and Responsibilities Justification for Validation Discussion. Install Set For Year End. Production Environment. Design and Coding. Testing. Traditional View of Testing.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

Don’t Just “Test”… Validate!!


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
agenda
Agenda
  • Overview of Testing versus Validation
  • Common Activities and Responsibilities
  • Justification for Validation
  • Discussion
traditional view of testing

Install

Set For

Year End

Production

Environment

Design and Coding

Testing

Traditional View of Testing

Project Life Cycle

slide4

Hiring Freeze

Install

Set For

Year End

Design and Coding

Changes to

Requirements

Time & Resource Constraints

Production

Environment

Testing

Project Life Cycle

risk factors examples of project risk factors
Risk FactorsExamples of Project Risk Factors

SEI identifies at least 10 project risk factors:

  • Over-ambitious Schedule
  • Under-ambitious Budget
  • Over-ambitious or unrealistic expectations
  • Undefined or misunderstood obligations
  • Inadequate software sizing estimate
  • Unsuitable or lack of development process model
  • Continuous requirements change
  • Unsuitable organizational structure
  • Inadequate software development plan
  • Lack of political support

Assess these factors in relationship to the testing effort

factors affecting testing
Factors Affecting Testing

Short Discussion:

What have you seen?

slide7

Testing Objectives

Did we build the system right?

Maximize System Quality

Minimize Testing Effort

slide8

What Is Validation?

Providing documented evidence with high degree of assurance the system will consistently produce a product or perform a process which meets the predetermined specifications and quality attributes

slide9

Critical Success Factors

Risk to Business

Validation

Business Needs

Factors Driving Validation

slide10

Relationship of Risk to Validation

Risk tells you

  • What to address first
  • What to address most thoroughly

Degree of Validation

Degree of Risk

slide11

Validation Objectives

Did we build the right system?

Maximize System Quality

Meet Business Needs

slide12

Goals of Validation

  • Control of system development:
    • Requirements meaningful, specified, and approved
    • Effective process capability of developer
    • Existence of and conformance to SOPs
    • Effective, appropriate, documented testing
    • Configuration management & change control
  • Control of system use:
    • Current accuracy and reliability
    • Continued accuracy and reliability
    • Management awareness and control
    • Auditability
    • Data integrity
    • Reviewer independence and credentials
slide13

What Makes Up A System?

Peripherals

Documentation

Procedures

Infrastructure

Software

Hardware

Trained Personnel

The Computer System

The Business Environment

slide14

Testing (traditionally) Encompasses…

Documentation

Procedures

Infrastructure

Software

Hardware

Trained Personnel

The Computer System

slide15

Validation Encompasses…

Peripherals

Documentation

Procedures

Infrastructure

Software

Hardware

Trained Personnel

The Computer System

The Business Environment

slide16

Marketing

Design

Project

Orientation

SDLC

Construction

Test Construction

Test Execution

Elaboration

Test Evaluation

Transition

Test Planning

Inception

Test Strategy

Test Scope

QC

Process

Transition

Executing

Initiation

Planning

Closure

Customer

Requirement

Idea

Generation

General

Release

Support

Retirement

Operations

Implementation

CMM Quality Assurance ProgramProgram Owner AccountabilitiesQA Accountabilities occur at every touch point

Product Process

Project Management Process

slide17

QA & QC Coming Together

Project

Orientation

Initiation

Transition

Executing

Planning

Closure

Quality Assurance

Process Audits

Process Consultation

Defect Analysis

Measurement Analysis

Continuous Improvement

Standards Development

Risk Management

Defect Management

Test Planning

Static Testing

Dynamic Testing

Test Automation

Test & Quality Control

Defect

Resolution

Metrics

Test Execution

Metrics

Project Planning

Support

Defect Cost

Analysis

Benchmarking

Forecasting

Measurement & Analysis

slide18

Strategy

Evaluate

Design

Execute

Common Activities

Each Level is Composed of These Activities

slide19

Strategy

Evaluate

Design

Execute

Strategy Development

  • Determine Validation Strategy
  • Perform Risk Assessment
  • Identify Critical Success Factors
  • Set Validation Objectives
  • Define Validation Activities
  • Estimate Time & Resources
  • Document Validation Strategy
slide20

Factor

Extent to which ...

Correctness

… system must satisfy stated requirements

Authorization

… processing requires management authorization

Data Integrity

… data stored by the system must be accurate

Service Levels

… schedules must be met

Access Control

… access must be restricted

Methodology

… development plan must be followed

Reliability

… system cannot fail during operation

Ease of use

… effort is required to learn and use system

Maintainability

… effort is required to fix errors in the system

Portability

… system can operate in multiple environments

Performance

… functions must perform within a specified time

Identify Critical Success Factors

slide21

Strategy

Evaluate

Design

Execute

Design

  • Develop/obtain tools for conducting Validation Activities
    • Deliverables & Phase Checklists
    • Procedures describing reviews & inspections
    • Validation Matrices
    • Test Cases
    • Automated Testing Tools
    • Audit plans
slide22

Examples of Validation Objectives

#

Validation Objective

Priority

Completion Criteria

1

Customer orders processed correctly

High

Enter customer order information as described in the validation test cases and verify the results are correct

2

System is ready to use

High

No open defects of type SEVERE or HIGH

Performance criteria are met

Users are trained, documentation is ready

3

Hold is placed on inventory when QC test fails

Med

Enter FAIL test results for product

Verify MRP II indicates status as HOLD

Verify product is segregated in warehouse

4

Increased through-put

Low

Measure increase by time study

5

Labels printed as req’d

Low

Print labels, verify against approved label copy specifications

slide23

Examples of Validation Activities

Activity

Performed by

Description

Deliverable

Requirements Reviews

Business Analysts,

Developers,

Users

Study & discuss system requirements to ensure they meet business needs

Reviewed statement of requirements, ready to be translated into design

Design Reviews

Developers

Study & discuss system design to ensure it will support requirements

Design, ready to be translated into software, hardware, documents, etc.

Code Inspections

Developers

Informal analysis of code to find defects

Software ready to be inspected or tested

Code Walk-throughs

Developers

Formal analysis of code to find defects, problem areas, coding techniques, design flaws

Software ready to be tested

slide24

Examples of Validation Activities

Activity

Performed by

Description

Deliverable

Unit Testing

Developers

Test single units of code. Validates that the software performs as designed.

Software unit ready for testing with other components

Integration Testing

Independent Test Team

Test related units, programs, modules. Validates that multiple parts of the system interact according to design.

Portions of the system ready for testing with other portions

System Testing

Conducted at end of Integration Testing. Test entire system. Validates system requirements.

Tested system, based on specification

Acceptance Testing

Users

Test system to make sure it works in the environment. Validates business needs.

Tested system, based on user needs

slide25

Detailed Validation Matrices

Requirement

Test Case

1

2

3

4

5

6

Validate sign-on

x

Invalid term ident

x

Validate password

x

Password/ Signon error

x

Validate Main Menu

x

Re-enter O, X, M, R

x

Validate Customer #

x

Customer # Required

x

Field entry too long

x

Customer ___ not found

x

slide26

Strategy

Evaluate

Design

Execute

Execution

  • Organize Validation Tools
  • Train Team Members
  • Execute Validation Plan
  • Track Validation Progress
  • Perform Regression Testing
  • Document Results
slide27

Strategy

Evaluate

Design

Execute

Evaluation

  • Evaluate Compliance of System Development
  • Evaluate Defects
  • Develop Findings and Recommendations
  • Formalize Report
slide28

Who Is Responsible for Validation?

Developer

Business Unit

(User)

Validation

Process

Quality Assurance

Independent Tester

slide29

Business Unit Responsibility

  • Define system requirements
  • Define data used in the system
  • Enforce Change Control
  • Plan & Perform Acceptance Testing
  • Ensure adequate backup & recovery of data is available
  • Work with developers to coordinate schedules
  • Investigate Defects
  • Develop User Documentation
  • Develop User Procedures
  • Enforce security
  • Provide audit trail
  • Train personnel
slide30

Developer Responsibility

  • Follow development process
  • Work with QA to improve the process
  • Manage changes to system components
  • Provide adequate backup & recovery
  • Maintain configuration management
  • Plan & Perform Unit Testing
  • Provide Technical Documentation
  • Maintain communication with QA , Testers & Business
slide31

QA Responsibility

  • Define Development & Validation Process
  • Verify personnel have adequate training & resources for their job
  • Review documentation - adherence to standards
  • Periodically review process activities
  • Evaluate vendor capabilities & processes
  • Verify enhancements & maintenance are documented
  • Conduct Risk Assessments
  • Verify business unit has established procedures
  • Ensure system is adequately secure
slide32

Independent Tester Responsibility

  • Follow Validation Process
  • Working with Users &/or BA’s, perform Requirements Validation
  • Work with QA to improve the process
  • Plan & Perform Integration & System Testing
  • Conduct Risk Assessments
  • If required, Plan & Perform Stress & Performance Testing
slide33

Testing Levels & Validation

Business Needs

Acceptance Testing

Requirements

System Testing

Design

Integration Testing

Code

Unit Testing

Production

slide34

Plan for Maintenance

  • Prepare for Maintenance during Development
  • Keep Deliverables and Documentation up-to-date
    • Requirements
    • Design
    • Test Cases, Test Data, Test Results
  • Maintain all related deliverables during error correction
    • Requirement specifications
    • Program specifications
    • Design specifications
    • Defect reports
slide35

Maintenance Deliverables

  • Risk Assessment
  • Software updates
  • Hardware upgrades
  • System documentation
  • Manuals, procedures
  • Revised/New Test Cases
  • Updated Validation Plan
slide36

Validation Issues

  • Responsibility lies with the user
    • If outsourced, the responsibility is shared between the outsourcing organization and the user
  • Develop deliverables as system is maintained
  • Group maintenance changes together
  • Schedule maintenance in releases
  • Follow defined methodology
slide37

Purchased System Responsibilities

Validation Activity

Vendor

Customer

System Staff

Users

Feasibility Review

x

x

Requirements Review

x

x

x

Design Review

x

x

Code Review

x

Unit Testing

x

Integration Testing

x

System Testing

x

x

Acceptance Testing

x

Conversion Testing

x

x

Maintenance Testing

x

x

slide39

Savings

Failure

Failure

Appraisal

Appraisal

Prevention

Prevention

Without

Quality Process

With

Quality Process

Cost of Quality Real Benefits

slide40

Validation Costs and Benefits

Incremental Benefits

Benefits

Net Benefits

Net Costs

Time

Incremental Costs

Costs

Development

Maintenance

self assessment
Self-Assessment

Y

N

?

Are risk assessments being performed on projects?

Is there a methodology in place for system development and validation?

Are verification techniques, such as walk-throughs, being used?

Have all personnel been trained in validation techniques?

Are standards in place for system development and validation?

Are the users active participants in the validation process?

Are the results of the validation used to improve the development and validation processes?

Is the culture ready to accept new processes for validation?

Is there someone responsible for validation of systems?

slide42

Validation:

Did we build the right system?

Testing:

Did we build the system right?

system validation
System Validation
  • Validates BOTH Business & Technical Requirements
  • Reduces Development Costs by identifying Errors early in Development Cycle
  • Reduces Risk of Operating Failures after Implementation
  • Reduces Post Implementation Development Costs by Fixing Defects during Development
  • Improves Overall Quality of System
  • Greatly improves User Satisfaction
slide44

Summary

Projects Delivered

With Validation

Projects Delivered

Without Validation

The Road to Success!!