Chapter 14 Testing Tactics

1. Chapter 14Testing Tactics

2. 14.1 Software Testing Fundamentals • What is a “Good” test? • A good test has a high probability of finding an error • A good test is not redundant • A good test should be “best of breed” • A good test should be neither too simple nor too complex • Test Case Design • OBJECTIVE: to uncover errors • CRITERIA: in a complete manner • CONSTRAINT: with a minimum of effort and time 1/21

3. 14.1 Software Testing Fundamentals • Testability • Operability - the better it works the more efficiently it can be tested • Observability - what you see is what you test • Controllability - the better software can be controlled the more testing can be automated and optimized • Decomposability - by controlling the scope of testing, the more quickly problems can be isolated and retested intelligently • Simplicity - the less there is to test, the more quickly we can test • Stability - the fewer the changes, the fewer the disruptions to testing • Understandability - the more information known, the smarter the testing 2/21

4. loop = 20 X White-Box Testing Black-Box Testing 14.2 Black-box an White-box Testing • Why not try exhaustive testing? There are 5201014 possible paths! If we execute one test per millisecond, it would take 3,170 years to test this program!! Note: Testing cannot show the absence of errors and defects. 3/21

5. 14.3 White-box Testing • Definition: White-box or Glass-box Testing Knowing the internal workings of a product, tests are performed to check the workings of all possible logic paths --testing in the small. • Questions • Can you guarantee that all independent paths within a module will be executed at least once? • Can you exercise all logical decisions on their true and false branches? • Will all loops execute at their boundaries and within their operational bounds? • Can you exercise internal data structures to ensure their validity? 4/21

6. Begin a（begin） K=0 L=0 TOTAL=0 b（in） a flow graph c b Input A Predicate node d c Do While TOTAL<1000 and A0 F d e d’ # of regions R4 Cyclomatic Complexity = E N + 2 f P + 1 e F A>0 R2 i R1 f T T g i TOTAL=TOTAL+A g R3 L++ h h K++ j j（out） Input A k k（stop） Output K, L, TOTAL Stop 14.4 Basis Path Testing • Goal: execute every statement at least once Defines the number of independent paths in the basis set of a program and provides us with an upper bound for the number of tests that must be conducted. Compound conditions 5/21

7. in A > 1 AND B=0 T X = X / A F A=2 OR X > 1 X = X + 1 T F out 14.4 Basis Path Testing • Prepare test cases that will force execution of each path in the basis set – only to execute every statement at least once isNOTenough Example: CC = ? 5 Test case : A=3, B=0, X=4. Note: If AND is written as OR, the error will not be detected by the above test case. 6/21

8. 14.5 Control Structure Testing • Condition Testing: test every condition at least once Examples: Test( E1 > E2 ) AND ( E3 = E4 ) B1 ANDB2 { ( t, t ), ( f, t ), ( t, f ) } B1 AND ( E3 = E4 ) { ( t, = ), ( f, = ), ( t, < ), ( t, > ) } ( E1 > E2 ) AND ( E3 = E4 ) { ( >, = ), ( =, = ), ( <, = ), ( >, < ), ( >, > ) } 7/21

9. 14.5 Control Structure Testing • Data Flow Testing: selects test paths according to the locations of variabledefinitionsandusesin the program DEF(S) = { X| S contains a definition of X }; USE(S) = { X| S contains a use of X }. DU chain of X = [X, S, S’] where XDEF(S) and XUSE(S’), and the definition of X in S is live at S’ ---- the flow of X. Every DU chain is covered once. 8/21

10. Simple loop Nested Loops Concatenated Loops Unstructured Loops 14.5 Control Structure Testing • Loop Testing 0,1 0 1 2 2<m<n n-1 n n+1 9/21

11. 14.6 Black-box Testing • Definition: Black-box or Behavioral Testing Knowing the specified function a product is to perform and demonstrating correct operation based solely on its specification without regard for its internal logic – testing in the large. • Questions • How is functional validity tested? • How is system behavior and performance tested? • What classes of input will make good test cases? • Is the system particularly sensitive to certain input values? • How are the boundaries of a data class isolated? • What data rates and data volume can the system tolerate? • What effect will specific combinations of data have on system operation? 10/21

12. 14.6 Black-box Testing • Techniques • Graph-Based Testing • Equivalence Partitioning • Boundary Value Analysis • Orthogonal Array Testing 11/21

13. 14.6 Black-box Testing • Equivalence Partitioning An equivalence class represents a set of valid or invalid states for input conditions, so that there is no particular reason to choose one element over another as a class representative. • Guidelines • If an input condition specifies a range, one valid and two invalid equivalence classes are defined. • If an input condition requires a specific value, one valid and two invalid equivalence classes are defined. • If an input condition specifies a member of a set, one valid and one invalid equivalence classes are defined. • If an input condition is Boolean, one valid and one invalid equivalence classes are defined. 12/21

14. "Bugs lurk in corners and congregate at boundaries ..." Boris Beizer 14.6 Black-box Testing • Boundary Value Analysis (BVA) • Guidelines • If input condition specifies a range bounded by values a and b, test cases should include a and b, values just above and just below a and b. • If an input condition specifies and number of values, test cases should be exercise the minimum and maximum numbers, as well as values just above and just below the minimum and maximum values. • Apply guidelines 1 and 2 to output conditions, test cases should be designed to produce the minimum and maxim output reports. • If internal program data structures have boundaries (e.g., size limitations), be certain to test the boundaries. 13/21

15. 14.7 Object-Oriented Testing Methods • Test Case Design • 1. Each test case should beuniquely identifiedand should beexplicitly associatedwith the class to be tested, • 2. Thepurposeof the test should be stated, • 3. A list of testingsteps should be developed for each test and should contain [BER94]: • a. a list of specifiedstatesfor the object that is to be tested • b. a list ofmessages and operationsthat will be exercised as a consequence of the test • c. a list ofexceptionsthat may occur as the object is tested • d. a list ofexternalconditions • e. supplementary information that will aid in understanding or implementing the test. Unlike conventional test case design, which is driven by an I-P-O view of software or the algorithmic detail of individual modules, OO testing focuses on designing appropriate sequences of operations to exercise the states of a class. 14/21

16. 14.7 Object-Oriented Testing Methods • Fault-Based Testing • The tester looks for plausible faults (i.e., aspects of the implementation of the system that may result in defects). To determine whether these faults exist, test cases are designed to exercise the design or code. • Class Testing and the Class Hierarchy • Inheritance does not obviate the need for thorough testing of all derived classes. In fact, it can actually complicate the testing process. • Scenario-Based Test Design • Scenario-based testing concentrates on what the user does, not what the product does. This means capturing the tasks (via use-cases) that the user has to perform, then applying them and their variants as tests. 15/21

17. 14.8 Class Level Testing Methods • Random testing • identify operations applicable to a class • define constraints on their use • identify a minimum test sequence • an operation sequence that defines the minimum life history of the class (object) • generate a variety of random (but valid) test sequences • exercise other (more complex) class instance life histories Requires large number of data permutations and combinations and can be inefficient. 16/21

18. 14.8 Class Level Testing Methods • Partition Testing • reduces the number of test cases required to test a class in much the same way as equivalence partitioning for conventional software • state-based partitioning • categorize and test operations based on their ability to change the state of a class • attribute-based partitioning • categorize and test operations based on the attributes that they use • category-based partitioning • categorize and test operations based on the generic function each performs 17/21

19. 14.9 Inter-class Test Case Design • Multiple Class Testing (random) • For each client class, use the list of class operators to generate a series of random test sequences. The operators will send messages to other server classes. • For each message that is generated, determine the collaborator class and the corresponding operatorin the server object. • For each operatorin the server object (that has been invoked by messages sent from the client object), determine the messagesthat it transmits. • For each of the messages, determine the next level of operators that are invoked and incorporate these into the test sequence. 18/21

20. e m p t y s e t u p a c c t a c c t o p e n s e t u p A c c n t d e p o s i t ( i n i t i a l ) d e p o s i t w o r k i n g b a l a n c e a c c t w i t h d r a w c r e d i t a c c n t I n f o w i t h d r a w a l ( f i n a l ) d e a d n o n w o r k i n g a c c t a c c t c l o s e F i g u r e 1 4.12 S t a t e d i a g r a m f o r A c c o u n t c l a s s ( a d a p t e d f r o m [ K I R 9 4 ] ) 14.9 Inter-class Test Case Design • Behavior Testing – The tests to be designed should achieve all state coverage [KIR94]. 19/21

21. 14.10 Specialized Testing • Graphical User Interfaces • Client/Server Architectures • Documentation and Help Facilities • Real-Time Systems 14.11 Testing Patterns • Define the patterns and REUSE 20/21

22. Final Words About Testing Testing never ends, it just gets transferred from you [ the software engineer ] to your customer. Every time your customer uses the program, a test is being conducted. 21/21