Cs115 class 15 testing
This presentation is the property of its rightful owner.
Sponsored Links
1 / 39

CS115 Class 15: Testing PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

CS115 Class 15: Testing. Due today User Manual Software Inspections Review: Testing Next time Review No Silver Bullet. French Guyana, June 4, 1996. $800 million software failure. Mars Climate Orbiter.

Download Presentation

CS115 Class 15: Testing

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Cs115 class 15 testing

CS115 Class 15: Testing

  • Due today

    • User Manual

    • Software Inspections

    • Review: Testing

  • Next time

    • Review No Silver Bullet

Cs115 class 15 testing

French Guyana, June 4, 1996

$800 million software failure

Mars climate orbiter

Mars Climate Orbiter

  • The 125 million dollar Mars Climate Orbiter is assumed lost by officials at NASA. The failure responsible for loss of the orbiter is attributed to a failure of NASA’s system engineer process. The process did not specify the system of measurement to be used on the project. As a result, one of the development teams used Imperial measurement while the other used the metric system of measurement. When parameters from one module were passed to another during orbit navigation correct, no conversion was performed, resulting in the loss of the craft.

More bsod embarrassments

More BSOD Embarrassments

Economic impact

Economic Impact

  • NIST study

    • On CNN.com - April 27, 2003


Basic definitions

Basic Definitions

  • Error-mistake or deviation from correct value

  • Failure-unacceptable system behavior

  • Defect/fault-flaw in any part of system that may cause a failure

Testing objective

Testing Objective

  • Find defects/faults

    • Find them effectively

      • as many as possible

    • Find them efficiently

      • as many as possible given testing time

Testing scope

Testing scope

  • Unit

  • Integration

  • System

  • Acceptance

  • Alpha/Beta

Test approach

Test approach

  • White box

  • Black box

Software development today

Decision Maker



Software Development Today

Why do we have this structure?

Typical scenario 1

Decision Maker



Typical Scenario (1)

“OK, calm down. We’ll slip the schedule. Try again.”

“It doesn’t #$%& compile!”

“I’m done.”

Typical scenario 2

Decision Maker



Typical Scenario (2)

“Now remember, we’re all in this together. Try again.”

“It doesn’t install!”

“I’m done.”

Typical scenario 3

Decision Maker



Typical Scenario (3)

“Let’s have a meeting to straighten out the spec.”

“It does the wrong thing in half the tests.”

“I’m done.”

“No, half of your tests are wrong!”

Typical scenario 4

Decision Maker



Typical Scenario (4)

“Try again, but please hurry up!”

“It still fails some tests we agreed on.”

“I’m done.”

Typical scenario 5

Decision Maker



Typical Scenario (5)

“Oops, the world has changed. Here’s the new spec.”

“Yes, it’s done!”

“I’m done.”

Software development today1

Decision Maker



Software Development Today

Why do we have this structure?

Standard testing questions

Standard Testing Questions

  • How shall we generate/select test cases?

  • Did this test execution succeed or fail?

  • How do we know when to stop testing?



  • Testing is hard

    • If done manually, also very expensive and boring

  • Use inspections!

    • Will save more time in testing and debugging

  • A number of techniques can make testing effective

    • Randomized testing

    • Exhaustive testing on small examples

    • Regression testing with nightly build

Back to design

Back to Design

  • Testing has a profound impact on design

    • Because some designs are easier to test

  • Design software so it can be tested!

Principles of testability

Principles of Testability

  • Avoid unpredictable results

    • No unnecessary non-deterministic behavior

  • Design in self-checking

    • At appropriate places have system check its own work

      • Asserts

    • May require adding some redundancy to the code

  • Have a test interface

  • Minimize interactions between features

    • Number of interactions can easily grow huge

    • Rich breeding ground for bugs



Test automation

Benefits of junit

Benefits of JUnit

  • Free - http://www.junit.org

  • Automatic check of expected vs. actual

  • Simple to use – quick to write and run

  • Tests are written in Java

Design of junit

Design of JUnit

  • Built around Command pattern

    • each method under test is represented by a separate test method

Junit sample unit test write tests for all methods in a class

JUnit Sample Unit Test write tests for all methods in a class

import junit.framework.TestCase

public class ShoppingCartTest extends TestCase


protected void setUp() {…}

protected void tearDown() {…}

public void testEmpty() {…}

public void testAddItem() {…}


How to create a test suite

How to create a test suite

import junit.framework.TestSuite;

public class EcommerceTestSuite {

public static Test suite() {

TestSuite suite = new TestSuite();




/** * Runs the test suite using the textual runner. */

public static void main(String[] args) { junit.textui.TestRunner.run(suite());



A gui for running the test suite

A GUI for running the test suite

java junit.swingui.TestRunner EcommerceTestSuite

Initialization called before every test case method

Initialization called before every test case method

protected void setUp() {

cart = new ShoppingCart();

book1 = new Product("Pragmatic Unit Testing", 29.95); cart.addItem(book1);


Clean up called after every test case method

Clean up called after every test case method

protected void tearDown() {

// set to null any references to big objects

Tests the empty method

Tests the empty method

public void testEmpty() {


assertEquals(0, cart.getItemCount());


Tests additem method

Tests addItem method

public void testAddItem() {

Product book2 = new Product("Pragmatic Project Automation",



double expectedBalance = book1.getPrice() + book2.getPrice();

assertEquals(expectedBalance, cart.getBalance());

assertEquals(2, cart.getItemCount());


Output of junit

Output of JUnit

>java junit.textui.TestRunner TestServer

1) testAddItem

AssertionFailedError: expected:<59.90> but was:<59.00 >

Junit assertxxx

JUnit assertXXX

  • assertEquals(x,y) // .equals, or values

  • assertFalse(boolean expr)

  • assertNotNull(obj ref)

  • assertNotSame(x, y) // using ==

  • assertNull()

  • assertSame()

  • assertTrue()

  • fail()

public void testPassNullsToConstructor(){

try{ Server s = new Server(null, null);

fail(“Expected IllegalArgumentException”);

} catch (IllegalArgumentException expected){}


Junit sequence

JUnit Sequence

  • New test case for every test<foo> method

  • For each test<foo> method

    • create new object

      • no sharing of instance variables between tests

    • setUp()

    • test<foo>

    • tearDown()

Junit practices

JUnit Practices

  • One function check per test method

    • usually one assert per test

      • sometimes more

    • failure aborts entire method

  • Many test<foo> per TestCase

  • Can group multiple TestCase into a TestSuite

Junit recommendations

JUnit Recommendations

  • Add JUnit tests in your code

    • run regularly (eg before SVN commit)

  • Benefits:

    • catch errors early

      • easier to debug, easier to fix

    • reduce cost of integration testing

    • reduce risk

      • of large, untested code base

      • of late slips in schedule (no time to recover)

      • increase confidence, reduce stress

More details

More details ...

  • JUnit home: www.junit.org

  • Tutorial http://clarkware.com/articles/JUnitPrimer.html

  • Also (thanks Stan)

    • http://www.soe.ucsc.edu/classes/cmps115/Spring06/SECURE/9904c.htm

Best practices session

“Best Practices” Session

  • By now, you have now developed some expertise in your particular specialization

    • (tester, coder, documenter, facilitator)

  • Group by specialization to discuss

    • what knowledge you’ve gained

    • what works, what doesn’t

    • tips to help other teams

  • Short (5min) presentation using doc camera

  • Login