Concjunit unit testing for concurrent programs
This presentation is the property of its rightful owner.
Sponsored Links
1 / 40

ConcJUnit: Unit Testing for Concurrent Programs PowerPoint PPT Presentation


  • 114 Views
  • Uploaded on
  • Presentation posted in: General

ConcJUnit: Unit Testing for Concurrent Programs. PPPJ 2009 Mathias Ricken and Robert Cartwright Rice University August 28, 2009. Concurrency in Practice. Brian Goetz, Java Concurrency in Practice , Addison-Wesley, 2006. Concurrency Practiced Badly. *1. Unit Tests…. Occur early

Download Presentation

ConcJUnit: Unit Testing for Concurrent Programs

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Concjunit unit testing for concurrent programs

ConcJUnit:Unit Testing for Concurrent Programs

PPPJ 2009

Mathias Ricken and Robert Cartwright

Rice University

August 28, 2009


Concurrency in practice

Concurrency in Practice

Brian Goetz, Java Concurrency in Practice, Addison-Wesley, 2006


Concurrency practiced badly

Concurrency Practiced Badly

*1


Unit tests

Unit Tests…

  • Occur early

  • Automate testing

  • Keep the shared repository clean

  • Serve as documentation

  • Prevent bugs from reoccurring

  • Allow safe refactoring


Existing unit testing frameworks

Existing Unit Testing Frameworks

  • JUnit, TestNG

  • Don’t detect test failures in child threads

  • Don’t ensure that child threads terminate

  • Tests that should fail may succeed


Concjunit

ConcJUnit

  • Replacement for JUnit

    • Backward compatible, just replace junit.jar file

    • Available at www.concutest.org

  • Detects failures in all threads

  • Warns if child threads outlive main thread

  • Warns if child threads are not joined


Sample junit tests

Sample JUnit Tests

publicclass Test extends TestCase {

public void testException() {

thrownew RuntimeException("booh!");

}

public void testAssertion() {

assertEquals(0, 1);

}

}

Both tests fail.

Both tests fail.

}

if (0!=1)

throw new AssertionFailedError();


Junit test with child thread

JUnit Test with Child Thread

Main thread

publicclass Test extends TestCase {

public void testException() {

new Thread() {

public void run() {

thrownew RuntimeException("booh!");

}

}.start();

}

}

new Thread() {

public void run() {

thrownew RuntimeException("booh!");

}

}.start();

thrownew RuntimeException("booh!");

Child thread

end of test

spawns

Main thread

success!

uncaught!

Child thread


Junit test with child thread1

JUnit Test with Child Thread

publicclass Test extends TestCase {

public void testException() {

new Thread() {

public void run() {

thrownew RuntimeException("booh!");

}

}.start();

}

}

new Thread() {

public void run() {

thrownew RuntimeException("booh!");

}

}.start();

thrownew RuntimeException("booh!");

Uncaught exception, test should fail but does not!

  • By default, no uncaught exception handler installed for child threads


Changes to junit 1 of 3

Changes to JUnit (1 of 3)

  • Thread group with exception handler

    • JUnit test runs in a separate thread, not main thread

    • Child threads are created in same thread group

    • When test ends, check if handler was invoked

      Reasoning:

  • Uncaught exceptions in all threads must cause failure


Junit test with child thread2

JUnit Test with Child Thread

publicclass Test extends TestCase {

public void testException() {

new Thread() {

public void run() {

thrownew RuntimeException("booh!");

}

}.start();

}

}

new Thread() {

public void run() {

thrownew RuntimeException("booh!");

}

}.start();

thrownew RuntimeException("booh!");

spawns and joins

resumes

Main thread

failure!

check group’s handler

end of test

Test thread

uncaught!

invokes group’s handler

Child thread


Child thread outlives parent

Child Thread Outlives Parent

publicclass Test extends TestCase {

public void testException() {

new Thread() {

public void run() {

thrownew RuntimeException("booh!");

}

}.start();

}

}

new Thread() {

public void run() {

thrownew RuntimeException("booh!");

}

}.start();

thrownew RuntimeException("booh!");

check group’s handler

Main thread

success!

Test thread

Too late!

end of test

uncaught!

invokes group’s handler

Child thread


Changes to junit 2 of 3

Changes to JUnit (2 of 3)

  • Check for living child threads after test ends

    Reasoning:

  • Uncaught exceptions in all threads must cause failure

  • If the test is declared a success before all child threads have ended, failures may go unnoticed

  • Therefore, all child threads must terminate before test ends


Check for living child threads

Check for Living Child Threads

publicclass Test extends TestCase {

public void testException() {

new Thread() {

public void run() {

thrownew RuntimeException("booh!");

}

}.start();

}

}

new Thread() {

public void run() {

thrownew RuntimeException("booh!");

}

}.start();

thrownew RuntimeException("booh!");

check for living

child threads

check group’s handler

Main thread

failure!

Test thread

end of test

uncaught!

invokes group’s handler

Child thread


Correctly written test

Correctly Written Test

publicclass Test extends TestCase {

public void testException() {

Thread t = new Thread() {

public void run() { /* child thread */ }

};

t.start();

t.join();

}

}

Thread t = new Thread() {

public void run() { /* child thread */ }

};

t.start();

t.join(); // wait until child thread has ended

/* child thread */

check for living

child threads

check group’s handler

Main thread

success!

Test thread

end of test

Child thread

*4


Changes to junit 3 of 3

Changes to JUnit (3 of 3)

  • Check if any child threads were not joined

    Reasoning:

  • All child threads must terminate before test ends

  • Without join() operation, a test may get “lucky”

  • Require all child threads to be joined


Fork join model

Fork/Join Model

  • Parent thread joins with each of its child threads

  • May be too limited for a general-purpose programming language

Main thread

Child thread 1

Child thread 2


Example of other join models

Example of Other Join Models

  • Chain of child threads guaranteed to outlive parent

  • Main thread joins with last thread of chain

Main thread

Child thread 1

Child thread 2

Child thread 3


Generalize to join graph

Generalize to Join Graph

  • Threads as nodes; edges to joined thread

  • Test is well-formed as long as all threads are reachable from main thread

Main thread

MT

Child thread 1

CT1

Child thread 2

CT2

Child thread 3

CT3


Unreachable nodes

Unreachable Nodes

  • An unreachable node has not been joined

    • Child thread may outlive the test

Main thread

MT

Child thread 1

CT1

Child thread 2

CT2


Constructing the graph start

Constructing the Graph: start()

// in mainThreadchildThread.start();

  • Add node for childThread

main Thread

MT

childThread

CT


Constructing the graph join

Constructing the Graph : join()

// in mainThreadchildThread.join();

  • When leaving join(), add edge from mainThread to childThread

main Thread

MT

child

Thread

CT

*3


Modifying the java runtime

Modifying the Java Runtime

  • Changing Thread.start()and join()

    • Need to modify Java Runtime Library

    • Utility to process user’s rt.jar file

    • Put new jar file on boot classpath:-Xbootclasspath/p:newrt.jar

  • Still works without modified Thread class

    • Just does not emit “lucky” warnings


Evaluation

Evaluation

  • JFreeChart

    • All tests passed; tests are not concurrent

  • DrJava: 900 unit tests

    • Passed:880

    • No join: 1

    • Lucky: 18

    • Timeout: 1

    • Runtime overhead: ~1 percent


Limitations

Limitations

  • Only checks chosen schedule

    • A different schedule may still fail

  • Example:

    Thread t = new Thread(…);if (nondeterministic()) t.join();

*2


Future work

Future Work

  • Insert sleeps or yields in critical code locations

    • Example:If a notify() is delayed, a wait() may time out.

    • Can detect a number of sample problems

  • Run tests several times on build server

  • Record schedule, replay if test fails

    • Makes failures reproducible if found

*5


Concjunit demo

ConcJUnit Demo


Thank you

Thank you!

www.concutest.org


Notes

Notes


Notes1

Notes

  • Probably not caused by concurrency problems; just a metaphor. ←

  • Also cannot detect uncaught exceptions in a program’s uncaught exception handler (JLS limitation) ←

  • Only add edge if joined thread is really dead; do not add if join ended spuriously. ←


Spurious wakeup

Spurious Wakeup

publicclass Test extends TestCase {

public void testException() {

Thread t = new Thread(new Runnable() {

public void run() {

thrownew RuntimeException("booh!");

}

});

t.start();

while(t.isAlive()) {

try { t.join(); }

catch(InterruptedException ie) { }

}

}

}

Thread t = new Thread(new Runnable() {

public void run() {

thrownew RuntimeException("booh!");

}

});

t.start();

while(t.isAlive()) {

try { t.join(); }

catch(InterruptedException ie) { }

}

thrownew RuntimeException("booh!");

Loop since join() may end spuriously


Notes2

Notes

  • Have not studied probabilities or durations for sleeps/yields:One inserted delay may negatively impact a second inserted delayExample: If both notify() and wait() are delayed. ←


Extra slides

Extra Slides


Junit test with child thread3

JUnit Test with Child Thread

publicclass Test extends TestCase {

public void testException() {

new Thread(new Runnable() {

public void run() {

thrownew RuntimeException("booh!");

}

}).start();

}

}

new Thread(new Runnable() {

public void run() {

thrownew RuntimeException("booh!");

}

}).start();

thrownew RuntimeException("booh!");

invokes

checks

TestGroup’s Uncaught Exception Handler


Join with all offspring threads

Join with All Offspring Threads

  • Main thread joins with all offspring threads, regardless of what thread spawned them

Main thread

Child thread 1

Child thread 2


Join graph examples

Join Graph Examples

Main thread

MT

Child thread 1

CT1

Child thread 2

CT2

Main thread

MT

Child thread 1

CT1

Child thread 2

CT2


Thread creation context

Thread Creation Context

  • In Thread.start()– also record stack trace of Thread.currentThread(

    • Easy to find source code of a child thread that is not joined

    • Also available for uncaught exception stack traces


Creation context example

Creation Context Example

class Helper extends Thread {

void m() { Assert.fail(); }

public void run() { m(); }

}

class Main {

void foo() {

// which one?

new Helper().start();

new Helper().start();

// ...

}

}

AssertionError:

at Helper.m(Helper.java:2)

at Helper.run(Helper.java:3)

Started at:

at Main.foo(Main.java:4)

at Main.bar(Main.java:15)

at Main.main(Main.java:25)


Many thanks to

Many Thanks To…

  • My advisor

    • Corky Cartwright

  • My committee members

    • Walid Taha

    • David Scott

    • Bill Scherer

  • NSF and Texas ATP

    • For providing partial funding


Concurrency problems not found during testing

Concurrency Problems NotFound During Testing

*1


  • Login