SE 4367 Functional Testing

753 Views

Download Presentation
## SE 4367 Functional Testing

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Learning Objectives**• What is functional testing? • How to perform functional testing? • How to generate test inputs? • What are equivalence partitioning, boundary value testing, domain testing, state testing, and decision table testing?**What is Functional Testing?**• When test inputs are generated using functional specifications, we say that we are doing functional testing. • Functional testing tests how well a program meets the functionality requirements.**The Methodology**• The derivation of test inputs is based on functional specifications. • Clues are obtained from the specifications. • Clues lead to test requirements. • Test requirements lead to test specifications. • Test specifications are then used to actually execute the program under test.**Clues**Expected behavior Test requirements Oracle or Test specifications Program has failed; make a note and proceed with testing or get into the debug mode. Test driver Until specs. Exhausted. Actual behavior Program Test Methodology Specifications Program output is correct**Equivalence Partitioning**• The input domain is usually too large for exhaustive testing. • It is therefore partitioned into a finite number of sub-domains for the selection of test inputs. • Each sub-domain is known as an equivalence class and serves as a source of at least one test input.**Input domain**partitioned into four sub-domains. Input domain 2 1 3 4 Equivalence Partitioning Four test inputs, one selected from each sub-domain. Too many test inputs.**How to partition?**• Inputs to a program provide clues to partitioning. • Example 1: • Suppose that program P takes an integer X as input X. • For X<0 the program is required to perform task T1 and for X>=0 task T2.**How to partition? (contd.)**• The input domain is prohibitively large because X can assume a large number of values. • However, we expect P to behave the same way for all X<0. • Similarly, we expect P to perform the same way for all values of X>=0. • We therefore partition the input domain of P into two sub-domains.**One test case:**X=-3 Equivalence class X<0 X>=0 Equivalence class Another test case: X=15 Two sub-domains All test inputs in the X<0 sub-domain are considered equivalent. The assumption is that if one test input in this sub-domain reveals an error in the program, so will the others. This is true of the test inputs in the X>=0 sub-domain also.**Non-overlapping Partitions**• In the previous example, the two equivalence classes are non-overlapping. In other words the two sub-domains are disjoint. • When the sub-domains are disjoint, it is sufficient to pick one test input from each equivalence class to test the program.**Non-overlapping Partitions**• An equivalence class is considered coveredwhen at least one test has been selected from it. • In partition testing our goal is to cover all equivalence classes.**Overlapping Partitions**• Example 2: • Suppose that program P takes three integers X, Y and Z. It is known that: • X<Y • Z>Y**X<Y, Z<=Y**X=2, Y=3, Z=1 X>=Y, Z<=Y X=15, Y=4, Z=1 X<Y X<Y, Z>Y X=3, Y=4, Z=7 X>=Y Z>Y Z<=Y X>=Y, Z>Y X=15, Y=4, Z=7 Overlapping partitions**Overlapping Partition-Test Selection**• In this example, we could select 4 test cases as: • X=4, Y=7, Z=1 satisfies X<Y • X=4, Y=2, Z=1 satisfies X>=Y • X=1, Y=7, Z=9 satisfies Z>Y • X=1, Y=7, Z=2 satisfies Z<=Y • Thus, we have one test case from each equivalence class.**Overlapping Partitions-Test Selection**• However, we may also select only 2 test inputs and satisfy all four equivalence classes: • X=4, Y=7, Z=1 satisfies X<Y and Z<=Y • X=4, Y=2, Z=3 satisfies X>=Y and Z>Y • Thus, we have reduced the number of test cases from 4 to 2 while covering each equivalence class.**Partitioning using non-numeric data**• In the previous two examples the inputs were integers. One can derive equivalence classes for other types of data also. • Example 3: • Suppose that program P takes one character X and one string Y as inputs. P performs task T1 for all lower case characters and T2 for upper case characters. Also, it performs task T3 for the null string and T4 for all other strings.**X: lc, Y: not null**X: UC, Y: not null X: lc X:UC X: lc, Y: null Y: not null Y: null X: UC, Y: null Partitioning using non-numeric data lc: Lower case character UC: Upper case character null: null string.**Non-numeric Data**• Once again we have overlapping partitions. • We can select only 2 test inputs to cover all four equivalence classes. These are: • X: lower case, Y: null string • X: upper case, Y: not a null string**Guidelines for equivalence partitioning**• Input condition specifies a range: create one for the valid case and two for the invalid cases. • e.g. for a<=X<=b the classes are • a<=X<=b (valid case) • X<a and X>b (the invalid cases)**Guidelines (contd.)**• Input condition specifies a member of a set: create one for the valid value and one for the invalid (not in the set) value. • Input condition specifies a value: create one for the valid value and two for incorrect values (below and above the valid value). This may not be possible for certain data types, e.g. for boolean.**Sufficiency of Partitions**• In the previous examples we derived equivalence classes based on the conditions satisfied by the input data. • Then we selected just enough tests to cover each partition. • Think of the advantages and disadvantages of this approach!**Boundary Value Analysis (BVA)**• Another way to generate test cases is to look for boundary values. • Suppose a program takes an integer X as input. • In the absence of any information, we assume that X=0 is a boundary. Inputs to the program might lie on the boundary or on either side of the boundary.**BVA (contd.)**• This leads to 3 test inputs: • X=0, X=-20, and X=14. Note that the values -20 and 14 are on either side of the boundary and are chosen arbitrarily. • Notice that using BVA we get 3 equivalence classes. One of these three classes contains only one value (X=0), the other two are large!**14**5 2 1 y2 10 11 6 9 8 13 12 y1 3 4 7 x1 x2 BVA (contd.) • Now suppose that a program takes two integers X and Y and that x1<=X<=x2 and y1<=Y<=y2.**BVA (contd.)**• In this case the four sides of the rectangle represent the boundary. • The heuristic for test selection in this case is: • Select one test at each corner (1, 2, 3, 4). • Select one test just outside of each of the four sides of the boundary (5, 6, 7, 8)**BVA (contd.)**• Select one test just inside of each of the four sides of the boundary (10, 11, 12, 13). • Select one test case inside of the bounded region (9). • Select one test case outside of the bounded region (14). • How many equivalence classes do we get?**BVA (contd.)**• In the previous examples we considered only numeric data. • BVA can be done on any type of data. • For example, suppose that a program takes a string S and an integer X as inputs. The constraints on inputs are: • length(S)<=100 and a<=X<=b • Can you derive the test cases using BVA?**BVA Applied to Output Variables**• Just as we applied BVA to input data, we can apply it to output data. • Doing so gives us equivalence classes for the output domain. • We then try to find test inputs that will cover each output equivalence class.**Finite State Machines (FSMs)**• A state machine is an abstract representation of actions taken by a program or anything else that functions! • It is specified as a quintuple: • A: a finite input alphabet • Q: a finite set of states • q0: initial state which is a member of Q.**FSMs (contd.)**• T: state transitions which is a mapping Q x A--> Q • F: A finite set of final states, F is a subset of Q. • Example: Here is a finite state machine that recognizes integers ending with a carriage return character. • A={0,1,2,3,4,5,6,7,8,9, CR} • Q={q0,q1,q2} • q0: initial state**FSMs (contd.)**• T: {((q0,d),q1),(q1,d),q1), (q1,CR),q2)} • F: {q2} • A state diagram is an easier to understand specification of a state machine. For the above machine, the state diagram appears on the next slide.**State transitions indicated**by labeled arrows from one state the another (which could be the same). Each label must be from the alphabet. It is also known as an event. d CR d Final state indicated by concentric circles. q2 q0 q1 States indicated by circles. State diagram d: denotes a digit**q2**q0 q1 State Diagram-Actions x/y: x is an element of the alphabet and y is an action. d/i=i+d*10j ; j=j+1 d/ i= d j = 1 CR/output i i is initialized to d and j to 1 when the machine moves from state q0 to q1. i is incremented by 10j*d and j by 1 when the machine moves from q1 to q1. The current value of i is output when a CR is encountered. Can you describe what this machine computes?**State Machine: Languages**• Each state machine recognizes a language. • The language recognized by a state machine is the set S of all strings such that: • when any string s in S is input to the state machine the machine goes through a sequence of transitions and ends up in the final state after having scanned all elements of s.**q2**q0 q1 State diagram-Errors d/i=i+d*10j ; j=j+1 d/ i= d j = 1 CR/output I CR/output error reset q4 q4 has been added to the set of states. It represents an error state. Notice that reset is a new member added to the alphabet.**State Diagram-Code**• A state diagram can be transformed into a program using case analysis. Let us look at a C program fragment that embodies logic represented by the previous state diagram. • There is one function for each action. • digit is assumed to be provided by the lexical analyzer.**Program for “integer” state machine**/* state is global, with values q0, q1, q2. i is also global.*/ void event_digit() case q0: i=digit; j = 1 ; /* perform action. */ state=q1; /* set next state. */ break; /* event digit is done. */ case q1: i=i+10^j*digit; j++; /* Add the next digit. */ state=q1; break; /*…complete the program. */ { switch (state)**Checking State Diagrams**• Unreachable state: One that cannot be reached from q0 using any sequence of transitions. • Dead state: One that cannot be left once it is reached.**Test Requirements**• Every state must be reached at least once, Obtain 100% state coverage. • Every transition must be exercised at least once. Obtain 100% transition coverage.**Example: Test Requirements**• For the “integer” state machine: • state machine transitions: • event digit in state q0 • event CR in state q0 • event digit in state q1 • event CR in state q1 • event reset in state q4**More testing of state machines?**• Yes, it is possible! • When we learn about path coverage we will discuss how more test requirements can be derived from a state diagram.**Test Specifications**• As before, test specifications are derived from test requirements. • In the absence of dead states, all states and transitions might be reachable by one test. • It is advisable not to test the entire machine with one test case.**Decision Tables**• Requirements of certain programs are specified by decision tables. • A decision table is useful when specifying complex decision logic**Decision Tables**• A decision table has two parts: • condition part • action part • The two together specify under what condition will an action be performed.**Decision Table-Nomenclature**• C: denotes a condition • A: denotes an action • Y: denotes true • N:denotes false • X: denotes action to be taken. • Blank in condition: denotes “don’t care” • Blank in action: denotes “do not take the action”**Bank Example**• Consider a bank software responsible for debiting from an account. The relevant conditions and actions are: • C1: The account number is correct • C2: The signature matches • C3: There is enough money in the account • A1: Give money • A2: Give statement indicating insufficient funds • A3: Call vigilance to check for fraud!**Example (contd.)**• A2 is to be performed when C1 and C2 are true and C3 is false. • A1 is to be performed when C1, C2, and C3 are true. • A3 is to be performed when C1 is true and C2 is false.**Default Rules**• Are all possible combinations of conditions covered? • No! Which ones are not covered? • We need a default action for the uncovered combinations. A default action could be an error report or a reset.