slide1 l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Object-Oriented and Classical Software Engineering Fifth Edition, WCB/McGraw-Hill, 2002 Stephen R. Schach srs@vuse.vand PowerPoint Presentation
Download Presentation
Object-Oriented and Classical Software Engineering Fifth Edition, WCB/McGraw-Hill, 2002 Stephen R. Schach srs@vuse.vand

Loading in 2 Seconds...

play fullscreen
1 / 37

Object-Oriented and Classical Software Engineering Fifth Edition, WCB/McGraw-Hill, 2002 Stephen R. Schach srs@vuse.vand - PowerPoint PPT Presentation


  • 151 Views
  • Uploaded on

Object-Oriented and Classical Software Engineering Fifth Edition, WCB/McGraw-Hill, 2002 Stephen R. Schach srs@vuse.vanderbilt.edu. CHAPTER 6. TESTING. Overview. Quality issues Nonexecution-based testing Execution-based testing What should be tested? Testing versus correctness proofs

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Object-Oriented and Classical Software Engineering Fifth Edition, WCB/McGraw-Hill, 2002 Stephen R. Schach srs@vuse.vand' - zavad


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Object-Oriented and Classical Software EngineeringFifth Edition, WCB/McGraw-Hill, 2002Stephen R. Schachsrs@vuse.vanderbilt.edu

chapter 6
CHAPTER 6

TESTING

overview
Overview
  • Quality issues
  • Nonexecution-based testing
  • Execution-based testing
  • What should be tested?
  • Testing versus correctness proofs
  • Who should perform execution-based testing?
  • When testing stops
testing
Testing
  • Two types of testing
    • Execution-based testing
    • Nonexecution-based testing
testing contd
Testing (contd)
  • “V & V”
    • Verification
      • Determine if the phase was completed correctly
    • Validation
      • Determine if the product as a whole satisfies its requirements
testing contd6
Testing (contd)
  • Warning
    • “Verify” also used for all nonexecution-based testing
software quality
Software Quality
  • Not “excellence”
  • Extent to which software satisfies its specifications
  • Software Quality Assurance (SQA)
    • Goes far beyond V & V
    • Managerial independence
      • development group
      • SQA group
nonexecution based testing
Nonexecution-Based Testing
  • Underlying principles
    • We should not review our own work
    • Group synergy
walkthroughs
Walkthroughs
  • 4–6 members, chaired by SQA
  • Preparation—lists of items
  • Inspection
    • Up to 2 hours
    • Detect, don’t correct
    • Document-driven, not participant-driven
    • Verbalization leads to fault finding
    • Performance appraisal
inspections
Inspections
  • Five-stage process
    • Overview
    • Preparation, aided by statistics of fault types
    • Inspection
    • Rework
    • Follow-up
fault statistics
Fault statistics
  • Recorded by severity and fault type
  • Compare with previous products
  • What if there are a disproportionate number of faults in a module?
  • Carry forward fault statistics to the next phase
statistics on inspections
Statistics on Inspections
  • 82% of all detected faults (IBM, 1976)
  • 70% of all detected faults (IBM, 1978)
  • 93% of all detected faults (IBM, 1986)
  • 90% decrease in cost of detecting fault (Switching system, 1986)
  • 4 major faults, 14 minor faults per 2 hours (JPL, 1990). Savings of $25,000 per inspection
  • Number of faults decreased exponentially by phase (JPL, 1992)
  • Warning
    • Fault statistics and performance appraisal
metrics for inspections
Metrics for Inspections
  • Fault density (e.g., faults per KLOC)
  • Fault detection rate (e.g., faults detected per hour)
  • By severity (major/minor), by phase
  • What does a 50% increase in the fault detection rate mean?
execution based testing
Execution-Based Testing
  • Definitions
    • Failure (incorrect behavior)
    • Fault (NOT “bug”)
    • Error (mistake made by programmer)
  • Nonsensical statement
    • “Testing is demonstration that faults are not present”
what is execution based tested
What is execution-based tested?
  • “The process of inferring certain behavioral properties of product based, in part, on results of executing product in known environment with selected inputs.”
    • Inference
    • Known environment
    • Selected inputs
  • But what should be tested?
utility
Utility
  • Does it meet user’s needs?
    • Ease of use
    • Useful functions
    • Cost-effectiveness
reliability
Reliability
  • Frequency and criticality of failure
    • Mean time between failures
    • Mean time to repair
    • Mean time, cost to repair results of failure
robustness
Robustness
  • Range of operating conditions
  • Possibility of unacceptable results with valid input
  • Effect of invalid input
performance
Performance
  • Extent to which space and time constraints are met
  • Real-time software
correctness of specifications
Correctness of specifications
  • Incorrect specification for a sort
  • Function trickSort which satisfies this specification:
correctness of specifications coptd
Correctness of specifications (coptd)
  • Incorrect specification for a sort:
  • Corrected specification for the sort:
correctness
Correctness
  • NOT necessary
  • NOT sufficient
correctness proofs
Correctness Proofs
  • Alternative to execution-based testing
example of correctness proof
Example of Correctness Proof
  • Code segment to be proven correct
example of correctness proof contd
Example of Correctness Proof (contd)
  • Flowchart of code segment
correctness proof case study
Correctness Proof Case Study
  • Never prove a program correct without testing it as well
episode 1
Episode 1
  • 1969 — Naur Paper
  • “Naur text-processing problem”

Given a text consisting of words separated byblankor bynl (new line) characters, convert it to line-by-line form in accordance with following rules:

(1) line breaks must be made only where given text has blankor nl ;

(2) each line is filled as far as possible, as long as

(3) no line will contain more than maxpos characters

  • Naur constructed a procedure (25 lines of Algol 60), and informally proved its correctness
episode 2
Episode 2
  • 1970 — Reviewer in Computing Reviews
    • The first word of the first line is preceded by blank unless the first word is exactly maxpos characters long
episode 3
Episode 3
  • 1971 — London finds 3 more faults
  • Including:
    • The procedure does not terminate unless a word longer than maxpos characters is encountered
episode 4
Episode 4
  • 1975 — Goodenough and Gerhart find three further faults
  • Including:
    • The last word will not be output unless it is followed by blank or nl
proofs and software engineering
Proofs and Software Engineering
  • Lesson:
  • Even if product is proved correct, it must STILL be tested.
three myths
Three Myths
  • Software engineers do not have enough math for proofs
  • Proving is too expensive to be practical
  • Proving is too hard
proofs and software engineering contd
Proofs and Software Engineering (contd)
  • Can we trust a theorem prover ?
  • How to find input–output specifications, loop invariants
  • What if the specifications are wrong?
  • Can never be sure that specifications or a verification system are correct
proofs and software engineering contd35
Proofs and Software Engineering (contd)
  • Correctness proofs are a vital software engineering tool, WHERE APPROPRIATE.
  • If
    • Human lives are at stake
    • Indicated by cost/benefit analysis
    • Risk of not proving is too great
  • Also, informal proofs can improve the quality of the product
    • Assert statement
who performs execution based testing
Who Performs Execution-Based Testing?
  • Testing is destructive
    • A successful test finds a fault
  • Solution
    • 1. The programmer does informal testing
    • 2. SQA does systematic testing
    • 3. The programmer debugs the module
  • All test cases must be
    • Planned beforehand, including expected output
    • Retained afterwards
when testing can stop
When Testing Can Stop
  • Only when the product has been irrevocably retired