Cs4311 spring 2011 verification validation dr guoqiang hu department of computer science utep
This presentation is the property of its rightful owner.
Sponsored Links
1 / 26

Announcement What: CS4390 (Cross-listed with CS5390): “ Software Testing ” PowerPoint PPT Presentation


  • 97 Views
  • Uploaded on
  • Presentation posted in: General

CS4311 Spring 2011 Verification & Validation Dr. Guoqiang Hu Department of Computer Science UTEP. Announcement What: CS4390 (Cross-listed with CS5390): “ Software Testing ” Testing Only, Nothing Else! When: This Summer Prerequisite: CS4311

Download Presentation

Announcement What: CS4390 (Cross-listed with CS5390): “ Software Testing ”

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Cs4311 spring 2011 verification validation dr guoqiang hu department of computer science utep

CS4311 Spring 2011 Verification & Validation Dr. GuoqiangHuDepartment of Computer ScienceUTEP


Announcement what cs4390 cross listed with cs5390 software testing

Announcement

  • What: CS4390 (Cross-listed with CS5390): “Software Testing”

    Testing Only, Nothing Else!

  • When: This Summer

  • Prerequisite: CS4311

    * Note: The V&V content covered in this course (CS4311) can be considered as an introduction to the above course


Announcement what cs4390 cross listed with cs5390 software testing

What Are Verification and Validation?

  • Groups of 2

  • Why? Who? What? Against what? When? How?

  • 5 minutes

    Verification:

  • Evaluating the product (a system or component) of a development phase to determine whether the product satisfies the specification at the start of the phase.

  • Did we build the system or component right?

    Validation:

  • Evaluating the product (mostly the system or subsystem) during or at the end of the development process to determine whether it satisfies the specified requirements.

  • Did we build the right system?


Announcement what cs4390 cross listed with cs5390 software testing

V&V Activities and the Software Lifecycle

Requirements engineering:

  • Determine general test strategy/plan(techniques, criteria, team)

  • Test requirements specification

  • Completeness

  • Consistency

  • Feasibility

    (Functional, performance requirements)

  • Testability

    (Specific; Unambiguous; Quantitative; Traceable)

  • Generate acceptance/validation testing data

    Design:

  • Determine integration test strategy

  • Assess/Test the design: Completeness; Consistency; Handling scenarios; Traceability (to and fro). (design walkthrough, inspection)


Announcement what cs4390 cross listed with cs5390 software testing

V&V Activities and the Software Lifecycle

Implementation:

  • Determine unit test strategy

  • Techniques (Static v. Dynamic):

  • Read/Have it read

  • Code walkthrough/Formal code inspection

  • Formal verification/Proof

  • Manual testing

  • Tools, and whistles and bells (driver/harness, stub)

    Maintenance:

  • Determine regression test strategy

  • Documentation maintenance (vital)


Announcement what cs4390 cross listed with cs5390 software testing

V&V Activities and the Software Lifecycle

The “V” model:


Announcement what cs4390 cross listed with cs5390 software testing

Where Do the Errors Come From?

  • Groups of 2

  • What kinds of errors? Who?

  • 3 minutes

    Kinds of errors:

  • Missing information

  • Wrong information/design/implementation

  • Extra information

    Facts about errors:

  • To err is human (but different person has different error rate).

  • Different studies indicate 30 to 85 errors per 1000 lines. After extensive testing, 0.5 to 3 errors per 1000 lines remain.

  • The longer an error goes undetected, the more costly to correct.


Announcement what cs4390 cross listed with cs5390 software testing

Basic Concepts

Correct specification:

  • Specification matches the client’s intent.

    Correct program:

  • Program matches its specification.

    Error:

  • A human activity that leads to the creation of a fault.

    Fault/Bug:

  • The physical manifestation of an error. It can cause a failure.

    Failure:

  • The state when a fault is encountered during execution.


Announcement what cs4390 cross listed with cs5390 software testing

Basic Concepts

Fault identification:

  • Process of determining what fault caused a failure

    Fault correction:

  • Process of changing a system to remove a fault

    Debugging:

  • Process of finding and fixing program faults

    Testing:

  • Designing and executing tests. And (if bugs present) debugging.

    Test case:

  • A particular set of input and the expected output


Announcement what cs4390 cross listed with cs5390 software testing

Basic Concepts

Test set:

  • A finite set of test cases working together with the same purpose

    Test objective:

  • The main goal for a particular test. Ex., finding faults/fault detection, or, demonstrating reliability/confidence building (no/low failure rate in normal use).

    Test objective affects test strategy, test criteria, and test selection.

    Test criteria:

  • Specifies testing requirements/stopping rule/measurement. It is closely linked to test techniques. Ex., for coverage-based test techniques, 100% statements, or branches, coverage, or both.


Announcement what cs4390 cross listed with cs5390 software testing

The General Approaches of Verification & Validation

  • Groups of 2

  • How?

  • 2 minutes


Announcement what cs4390 cross listed with cs5390 software testing

The Exhaustive Test Examples

  • Groups of 2

  • 3 minutes

  • How many test cases are required?

  • How long will it take?


Announcement what cs4390 cross listed with cs5390 software testing

The Exhaustive Test Examples

  • Groups of 2

  • 3 minutes

A

B

C


Announcement what cs4390 cross listed with cs5390 software testing

The Exhaustive Test Examples

  • Groups of 2

  • 3 minutes


Announcement what cs4390 cross listed with cs5390 software testing

The Purpose of Testing

  • The purpose of testing is NOT to prove the program is correct

  • Instead, it is to find problems in the program so that the found problems can be fixed

    Interesting phenomena of testing results:

  • Successful test:

    The more faults are found, the more successful is a test case/ set.

  • Quality of the code:

    The more faults are found in a unit of code, it normally means the worse is the quality of the code.

    Studies found that more faults may still go undetected in the same piece of code.

    Although you can say the quality of the code has been improved after the found faults are fixed.


Announcement what cs4390 cross listed with cs5390 software testing

Hierarchy of V&V techniques

V&V

Static Techniques

Dynamic Techniques

Formal

Analysis

Informal

Analysis

Symbolic

Execution

Testing

Walkthrough

Inspection

Review


Announcement what cs4390 cross listed with cs5390 software testing

Hierarchy of Testing

Testing

Ad hoc

Program Testing

System Testing

Acceptance

Testing

Unit Testing

Integration Testing

Function

Pilot

Properties

Black Box

Top Down

Alpha

Performance

Equivalence

Bottom Up

Reliability

Boundary

Beta

Big Bang

Availability

Decision Table

Sandwich

Security

State Transition

Usability

Use Case

Documentation

Domain Analysis

Portability

White Box

Control Flow

Data Flow

Capacity


Announcement what cs4390 cross listed with cs5390 software testing

Hierarchy of Testing

Testing

Ad hoc

Program Testing

System Testing

Acceptance

Testing

Unit Testing

Integration Testing

Function

Pilot

Properties

Black Box

Top Down

Alpha

Performance

Equivalence

Bottom Up

Reliability

Boundary

Beta

Big Bang

Availability

Decision Table

Sandwich

Security

State Transition

Usability

Use Case

Documentation

Domain Analysis

Portability

White Box

Control Flow

Data Flow

Capacity


Announcement what cs4390 cross listed with cs5390 software testing

Types of System Testing

  • Function Testing: Integrated system performs specified function

  • Scenarios

  • Black-box techniques

  • Properties Testing: Integrated system tests against non-functional requirements

  • Performance, Reliability, Security, Usability, etc.

  • Performance:

  • Stress test: maximum throughput

  • Overload test: exceed specification

  • Volume test: sustained large throughput

  • Response time

  • Robustness: test things not specified, but quite possible

  • Recovery: crash, or, recoverable?

  • Acceptance Testing: Customers test system

  • Pilot (initial, customer), Alpha test (in-house), Beta test (on-site)


Announcement what cs4390 cross listed with cs5390 software testing

Types of Faults (not exhaustive)

  • Algorithmic: algorithm or logic does not produce the proper output for the given input

  • Syntax: improper use of language constructs

  • Computation (precision): formula’s implementation wrong or result not to correct degree of accuracy

  • Documentation: documentation does not match what program does

  • Stress (overload): data structures filled past capacity

  • Capacity: system performance unacceptable as activity reaches its specified limit

  • Timing: coordinating of events not correct

  • Throughput: system does not perform at speed required

  • Recovery: failure encountered and does not recover (crashed)


Announcement what cs4390 cross listed with cs5390 software testing

Who Are Involved?

  • Professional/Third party testers: organize and run tests

  • Analysts: involved in system requirements definition and specification

  • Designers: involved in the design and understand proposed solution and solution’s constraints

  • Implementers: involved in the implementation and understand the constraints associated with implementation

  • Configuration management representative: arranges for changes to be made consistently across all artifacts

The Advice

  • View testing as part of the development process

  • Testing is the last line of defense: Errors indicate that there is a problem with the development process


Announcement what cs4390 cross listed with cs5390 software testing

Test Plan

Objectives of Test Plan:

  • Facilitate task of testing (strategy):

  • The scope, approach, resources, and schedule

  • Test techniques

  • Test criteria

  • Test documentation requirements

  • Avoid repetition

  • Improve test coverage

  • Improve test efficiency

  • Provide structure for final tests

  • Improve communication about testing

  • Provide structure for:

  • Organizing

  • Scheduling

  • Managing


Announcement what cs4390 cross listed with cs5390 software testing

Test Plan Report

  • Table of Contents

  • DOCUMENT CONTROL

  • Approval

  • Document Change Control

  • Distribution List

  • Change Summary

  • 1. INTRODUCTION

    1.1. Purpose

    1.2. Scope

    1.3. System Overview

    1.4. Test Approach Overview

  • 2. APPLICABLE REFERENCES

  • 3. TESTING APPROACH

  • 4. TEST SCHEDULE

  • 5. TEST XX


Announcement what cs4390 cross listed with cs5390 software testing

3. TESTING APPROACH

  • Specify types of tests to be performed

  • List specific tests

  • Test descriptions are in section 5

  • May include

  • Test Management Requirements: how testing is to be managed

  • Personnel Requirements

  • Hardware Requirements

  • Software Requirements

  • Cost


Announcement what cs4390 cross listed with cs5390 software testing

Section 5 and later

  • Test No.

  • Current Status (Passed / Failed / Pending)

  • Test title

  • Testing approach

  • Concluding Remarks

  • Testing Team

  • Date Completed:


Announcement what cs4390 cross listed with cs5390 software testing

Assignments

Lead: V&V

Due Date: Tuesday, April 19, 2011

Due Time: Midnight MDT


  • Login