Object-Oriented Software Testing

1. Object-Oriented Software Testing ICS 224A Joachim Feise jfeise@ics.uci.edu

2. A Brief Meditation on the Essential Nature of Software Testing OO Software Testing

3. Object-Oriented Testing Papers • Object-Oriented Integration Testing • Jorgensen and Erickson • CACM Sept. 1994 • Object-Oriented Software Testing - A Hierarchical Approach • Shel Siegel • J. Wiley and Sons, 1996 OO Software Testing

4. Structure vs. Behavior • most popular notations in software development portray software structure • examples: E/R models, data flow diagrams, structure charts, etc. • used to show relationships between components, interfaces, control and dataflow, etc. • not very useful for testing • SW testing is fundamentally concerned with behavior, i.e. what it does OO Software Testing

5. Traditional Testing Notions • Traditional software is • written in an imperative language • described by functional decomposition • developed in a waterfall life cycle • separated into three levels of testing • unit level, integration level, system level • these points do often not apply directly to object-oriented software OO Software Testing

6. Imperative vs. Declarative • Imperative languages lend themselves to a description as a directed graph • declarative languages suppress sequentiality • the event-driven nature of OO systems forces a “declarative spirit” on testing • not evident at unit level since most object-oriented languages are imperative • but pronounced at integration and system levels OO Software Testing

7. Unit Testing Overview • Units can be defined in several ways • a single, cohesive function • a function which, when coded fits on one page • the smallest separately compilable segment of code • the amount of code that can be written in 4 to 40 hours • a task in a work breakdown structure • code that is assigned to one person • code that one person designs, codes and test in a three-month period OO Software Testing

8. Unit Testing Overview (cont.) • Many organizations have not chosen their definition of a software unit • Units are always tested “by itself” • stubs and drivers are used to emulate inputs and outputs • goal: verification that, taken by itself, the unit functions correctly OO Software Testing

9. Integration Testing Overview • Once the units are separately tested, they are integrated together • gradual replacement of stubs and drivers by separately tested units • pairwise integration, i.e. each unit is integrated with its adjacent units • bottom-up integration guided by functional decomposition tree • “big bang” integration where all units are thrown together at once OO Software Testing

10. System Testing Overview • Conducted exclusively in terms of inputs and outputs that are visible at the port boundary of a system • Tester only has access to those port events that are visible to the customer/user • Can occur on the target platform only OO Software Testing

11. System Testing Overview (cont.) • Examples of system testing activities • sequences of machine instructions • sequences of source instructions • scenarios of normal usage • system-level test cases • stimulus/response pairs • behavior that results from sequences of system-level inputs • sequences of transitions in a state machine description of the system OO Software Testing

12. Object-Oriented Software • within an object, individual methods are imperative • declarative aspect because of event-driven nature • dynamic binding creates indefiniteness that resembles declarative programs • it is impossible to ever know the full set of “adjacent” objects with which a given object may be composed • two object may be correct, but errors might result when they are composed OO Software Testing

13. Composing Errors Illustrated OO Software Testing

14. Levels of OO Testing • Unit testing • method testing • Integration testing • Message quiescence • Event quiescence • System testing • Thread testing • Thread interaction testing OO Software Testing

15. Integration Testing • Method executions are linked by messages • Quiescence provides natural endpoints • Message quiescence • Method/Message path (MM-Path) • sequence of message executions linked by messages • starts with a method • ends when method is reached that does not issue any messages of its own • endpoint is called message quiescence OO Software Testing

16. Integration Testing (cont.) • Atomic System Function (ASF) • input port event, • followed by a set of MM-Paths, • terminated by output port event • ASFs constitute the point at which integration and system testing meet • elemental function visible at the system level OO Software Testing

17. MM-Paths and ASFs Illustrated OO Software Testing

18. ATM System Class Hierarchy OO Software Testing

19. Example: ATM System • Examples of • MM-Path: Digit entry • ASF: PIN Entry • Thread (sequence of ASFs): • simple transaction: PIN entry, transaction type selection, account detail presentation, operation execution, result reporting • Thread sequence: • a session consisting of multiple simple transactions OO Software Testing

20. Digit Entry MM-Path Illustrated OO Software Testing

21. PIN Entry ASF Illustrated OO Software Testing

22. Observations • Many-to-many relationships dominate (see E/R diagram) • an object may be involved in many threads • threads entail many objects • an object may be involved with many ASFs • ASFs may entail many objects OO Software Testing

23. Observations (cont.) • The five levels of OO testing result in distinct, useful testing goals • Bottom-up testing order • Seamless juncture between OO integration testing and system testing • Constructs are directly usable as basis for test coverage metrics • Works well with composition OO Software Testing

24. The Hierarchical Approach • Everything is a system • Defines and applies testing standards for several levels of software components • objects • classes • foundation components • systems OO Software Testing

25. Hierarchical Approach Illustrated OO Software Testing

26. Safe Components • Components that meet the testing standards for that kind of components • only safe components can be integrated with other safe components to produce the next-level component • Safe is a relative state • depends on risk management practices, attitude toward risk, what is enforced, the application, etc. OO Software Testing

27. Foundation Components • Can be one complete class hierarchy • or some cluster of classes • that performs a core function, • that represents a logical or physical architectural component, etc. • After testing to a safe level, can be integrated with other foundation components OO Software Testing

28. Integration Testing of Foundation Components • Needs to address the interconnections between the foundation components and any new composite functionality ONLY • Eliminates the need to test all of the combinations of states during integration testing • Improves productivity OO Software Testing

29. Integration Testing Problem Illustrated OO Software Testing

30. Test Suites • Conditional Test Suite • Hierarchical Incremental Test Suite • Integration Test Suite • System Test Suite • Regression Test Suite OO Software Testing

31. Conditional Test Suite • Four types of test scripts • Assertions: abort if condition fails • Exceptions: raise exception if condition succeeds • Concurrent test operations: monitor condition • Message polling: evaluate condition on demand • Conditions • class invariant, precondition, postcondition, loop invariant OO Software Testing

32. Pre/Postconditions Illustrated OO Software Testing

33. Hierarchical Incremental Test Suite • Structural models • control-flow model • models class as series of control-flow graphs deriving from the method implementations • data-flow model • models class as series of control-flow graphs with data annotations showing the flow of data values through the method implementations OO Software Testing

34. Hierarchical Incremental Test Suite (cont.) • Functional models • state-transition model, aka OMT dynamic model • models states and transitions for a class in the context of its position in the inheritance hierarchy • transaction-flow model • models object life cycle as transaction from construction to destruction • exception model • specifies what methods raise what exceptions under what conditions OO Software Testing

35. Integration Test Suite • Functional Test Suite • tests vertical integration of a cluster of components, i.e. the way the cluster implements specific functions from the requirements • Abstraction Test Suite • tests horizontal integration of a cluster of components, i.e. the way the cluster abstracts a service contract into an API OO Software Testing

36. Integration Test Suite (cont.) • Possible test scripts • class-to-class test script • interaction between classes based on message relationship • protocol-interface, synchronization, exception handling tests • UI test script • tests UI components of cluster and the components on which they depend • true-time, character-recognition, widget-playback tests OO Software Testing

37. System Test Suite • System Acceptance Test Suite • derived from the set of use cases covering the system requirements • User Options Test Suite • based on probable usage patterns of the features and functions delivered with the system • designed to simulate the way the end users will combine the features and functions OO Software Testing

38. Regression Test Suite • Series of tests run against a previous version of the CUT • Almost always automated • performing manual tests is usually not cost effective OO Software Testing

39. Epilog • “Testing proves the presence, not the absence, of bugs.” • E.W. Dijkstra • “Absence of evidence is not evidence of absence.” • Source unknown • From: E.V. Berard, Issues in the Testing of OO Software • Slides available at http://www.ics.uci.edu/~jfeise/ICS224A OO Software Testing