Software Testing - Path Testing & Data Flow Testing

1. Subject Name: Software Testing Subject Code: 10CS842 Prepared By: Tamilarasi.R Department: CSE Date

2. Unit –III Path Testing, Data Flow Testing

3. Topics  DD paths Test coverage metrics Basis path testing, Guidelines and observations Definition-Use testing Slice-based testing Guidelines and observations.

4. Boundary Value Analysis • Boundary value analysis focuses on the boundary of the input space to identify test cases. • The rationale behind boundary value analysis is that errors tend to occur near the extreme values of an input variable. • Programs written in non-strongly typed languages are more appropriate candidates for boundary value testing. • In our discussion we will assume a program P accepting two inputs x1 andx2 such that a ≤ y1≤ b and c≤ y2≤ d

5. Path testing • Path testing is a “design structural testing” in that it is based on detailed design & the source code of the program to be tested. • The methodology uses the graphical representation of the source code • Thus it is very much “control flow” or “path” oriented • This methodology has been available since the mid-1970’s and is an important White Box testing technique

6. Transforming Code to Diagram • In early chapter (chapter 4) on graph theory, we mentioned condensation graph as a graph that was derived by “condensing” a component (or set of code) into a node as a way to simplify the graphical diagram of source code. • We will : • address the issue of condensation later and • assume that we have a condensed graph and discuss path analysis first

7. Path Analysis • Why path analysis for test case design? • Provides a systematic methodology of White Box testing. • Reproducible • Traceable • Countable • What is path analysis? • Analyzes the number of paths that exist in the system • Facilitates the decision process of how many paths to include in the test Reasons similar to why document test cases Path, as previously defined in Graph Theory, is a sequence of connected nodes

8. Linearly Independent Path • A path through the system is Linearly Independent** from other paths only if it includes some segment or edge that is not covered in the other path. S1 - The statements are represented by the rectangular and diamond blocks. - The segments between the blocks are the edges, labeled with numbered circles. 1 2 C1 Path1 : S1 – C1 – S3 Path2 : S1 – C1 – S2 – S3 OR Path1: edges (1,4) Path2: edges (1,2,3) 4 S2 3 S3 Path1 and Path2 are linearly independent because each includes some edge that is not included in the other. (note: not necessarily nodes) ** This definition will require more explanation later.

9. Another Example of Linearly Independent Paths S1 1 S2 8 C1 2 Path1: edges (1,2,8) Path2: edges (1,5,3,9) Path3: edges (1,5,6,4,10) Path4: edges (1,5,6,7) Note that these are all linearly independent 5 C2 3 S3 9 6 C3 4 10 S4 7 S5

10. Statement Coverage Method • Count all the linearly independent paths • Pick the minimum number of linearly independent paths that will include all the statements(S’s and C’s in the diagram) S1 Path1 : S1 – C1 – S3 : edges (1, 4) Path2 : S1 – C1 – S2 – S3 : edges (1, 2. 3 ) 1 2 C1 4 S2 3 S3 Are both Path1 and Path2 needed to cover all the statements: (S1,C1,S2,S3) ? What would you say?

11. Another Example of Statement Coverage S1 1 S2 8 C1 2 The 4 Linearly Independent Paths Covers: Path1: includes S1-C1-S2-S5 : edges (1,2,8) Path2: includes S1-C1-C2-S3-S5 : edges ( 1,5,3,9) Path3: includes S1-C1-C2-C3-S4-S5 : edges (1,5,6,4,10) Path4: includes S1-C1-C2-C3-S5 : edges (1,5,6,7) 5 C2 3 S3 9 6 C3 4 10 S4 7 S5 For 100% Statement Coverage, all we need are 3 paths : Path1, Path2, and Path3 to cover all the statements (S1,C1,S2,C2,S3,C3,S4,S5) - - - no need for Path4 - - - - !!

12. Statement Coverage Now - - - What do you think of a Software Company which boasts that they test every statement in the software before release? Try this on some students or other professors and see their reaction.

13. Branch Coverage Method(Also DD-Paths of our text book) • Identify all the decisions • Count all the branches from each of the decisions ( out-degree of the decision node) • Pick the minimum number of paths that will cover all the branches from the decisions.

14. Branch Coverage Method One decision C1 : B1 : Path1 : C1 – S3 B2 : Path2 : C1 – S2 S1 1 Branch 1 C1 Branch 2 2 linearly independent paths cover : B1 : Path1 : C1 – S3: B2 : Path2 : C1 – S2 – S3 : 2 4 S2 3 S3 Are both Path1 and Path2 needed to cover Branch1 and Branch2 from C1?

15. Another Example of Branch Coverage The 3 Decisions and the Branches: C1: - B1 : C1- S2 - B2 : C1- C2 C2: - B3 : C2 – S3 - B4 : C2 – C3 C3: - B5 : C3 – S4 - B6 ; C3 – S5 S1 1 S2 8 C1 2 5 C2 3 S3 9 6 C3 4 10 S4 The 4 Linearly Independent Paths Cover: Path1: includes S1-C1-S2-S5 Path2: includes S1-C1-C2-S3-S5 Path3: includes S1-C1-C2-C3-S4-S5 Path4: includes S1-C1-C2-C3-S5 7 S5 We need: Path1 to cover B1, Path2 to cover B2 and B3, Path3 to cover B4 and B5, Path4 to cover B6

16. Branch Coverage How do you feel about a Software Company who states that they test every branch in the Software before release? Does All Branch Coverage subsume All Statements Coverage?

17. McCabe’s Cyclomatic Number • Is there a way to know how many linearly independent paths exist? • McCabe’s Cyclomatic number used to study program complexity may be applied. There are 3 ways to get the Cyclomatic Complexity number from a flow diagram. • # of binary decisions + 1 • # of edges - # of nodes + 2 • # of closed regions + 1

18. McCabe’s Cyclomatic Complexity NumberEarlier Example We know there are 2 linearly independent paths from before: Path1 : C1 – S3 Path2 : C1 – S2 – S3 S1 1 • McCabe’s Cyclomatic Number: • a) # of binary decisions +1 = 1 +1 = 2 • b) # of edges - # of nodes +2 = 4-4+2 = 2 • c) # of closed regions + 1 = 1 + 1 = 2 C1 2 4 S2 Closed region 3 S3

19. McCabe’s Cyclomatic Complexity NumberAnother Example • McCabe’s Cyclomatic Number: • a) # of binary decisions +1 = 2 +1 = 3 • b) # of edges - # of nodes +2 = 7-6+2 = 3 • c) # of closed regions + 1 = 2 + 1 = 3 S1 1 4 C1 2 C2 5 S2 There are 3 Linearly Independent Paths Closed Region Closed Region S4 7 6 3 S3

20. An example of 2n total path Since for each binary decision, there are 2 paths and there are 3 in sequence, there are 23 =8 total “logical” paths path1 : S1-C1-S2-C2-C3-S4 path2 : S1-C1-S2-C2-C3-S5 path3 : S1-C1-S2-C2-S3-C3-S4 path4 : S1-C1-S2-C2-S3-C3-S5 path5 : S1-C1-C2-C3-S4 path6 : S1-C1-C2-C3-S5 path7 : S1-C1-C2-S3-C3-S4 path8 : S1-C1-C2-S3-C3-S5 S1 1 C1 2 3 S2 4 C2 5 6 S3 How many Linearly Independent paths are there? Using Cyclomatic number = 3 decisions +1 = 4 One set would be: path1 : includes edges (1,2,4,6,9) path2 : includes edges (1,2,4,6,8) path3 : includes edges (1,2,4,5,7,9) path5 : includes edges (1,3,6,9) 7 C3 9 8 S5 S4 Note 1: with just 2 paths ( Path1 and Path8) all the statementsare covered. Note2: with just 2 paths ( Path1 and Path8) all the branchesare covered.

21. Example with a Loop Total number of ‘logical’ paths may be “infinite” (very large) because of the loop S1 1 Linearly Independent Paths = 1 decision +1 = 2 path1 : S1-C1-S3 (segments 1,4 ) path2 : S1-C1-S2-C1-S3 (segments 1,2,3,4 ) C1 4 S3 2 “One path” will cover all statements: path2 : S1-C1-S2-C1-S3 (S1,C1,S2,S3) S2 3 “One path” will cover all branches: path2 : S1-C1-S2-C1-S3 branch1 (C1-S2) and branch 2 (C1-S3) *** Note***: To get one path --- S2 must somehow change C1

22. Example with a Loop (cont.) Total number of ‘logical’ paths may be “infinite” (very large) because of the loop S1 1 Linearly Independent Paths = 1 decision +1 = 2 path1 : S1-C1-S3 (segments 1,4 ) path2 : S1-C1-S2-C1-S3 (segments 1,2,3,4 ) C1 4 S3 2 “One path” will cover all branches: path2 : S1-C1-S2-C1-S3 branch1 (C1-S2) and branch 2 (C1-S3) S2 3 Should we consider the loop boundaries in C1? & also bring in our boundary value testing thoughts?

23. More on Linearly Independent Paths • In discussing dimensionality, we talks about orthogonal vectors (or “basis” of vector space) . • (e.g.) Two dimensional space has two orthogonal vector from which all the other vectors in two dimension can be obtained via “linearcombination” of these 2 vectors: • [1,0] • [0,1] e.g. [2,4] = 2[1,0] + 4[0,1] [2,4] [1,0] [0,1]

24. More on Linearly Independent Paths • A set of paths is considered to be a Linearly Independent Set if every path may be constructed as a “linear combination” of paths from the linearly independent set. For example: We already know: a) there are a total of 22=4 logical paths. b) 1 path (path4) will cover all statements c) 2 paths will cover all branches. d) 2 branches +1 = 3 linearly independent paths. C1 2 1 S1 1 2 3 4 5 6 3 We pick: path1, path2 and path3 as the Linearly Independent Set 1 path1 1 1 C2 5 path2 1 1 4 path3 1 1 1 S2 path4 1 1 1 1 6 path 4 = path3 + path1 – path2 = (0,1,1,1,0,0)+(1,0,0,0,1,1)- (1,0,0,1,0,0) = (1,1,1,1,1,1) - (1,0,0,1,0,0) = (0,1,1,0,1,1)

25. Other Sets of Linearly Independent Paths 1 2 3 4 5 6 1 path1 1 1 path2 1 1 path3 1 1 1 path4 1 1 1 1 • Consider the set of linearly independent paths: 1, 2 & 4 instead. • Can we get Path3 = (0,1,1,1,0,0)? • Consider path3 = path2 + path4 – path1 • = (1,0,0,1,0,0,)+(0,1,1,0,1,1) – (1,0,0,0,1,1) = (0,1,1,1,0,0) • Consider another set of linearly independent paths: 2, 3 & 4 instead. • Can we get Path 1 = (1,0,0,0,1,1))? • Consider path1 = path2 + path4 – path3 • = (1,0,0,1,0,0,)+(0,1,1,0,1,1) – (0,1,1,1,0,0) = (1,0,0,0,1,1)

26. More on Linearly Independent Paths We already know: a) there are a total of 22=4 logical paths. b) 1 path (path4) will cover all statements c) 2 paths will cover all branches. d) 2 branches +1 = 3 linearly independent paths. C1 2 1 1 2 3 4 5 6 S1 1 path1 1 1 3 path2 1 1 C2 5 path3 1 1 1 4 path4 1 1 1 1 S2 6 Note : Although path1 and path3 are linearly independent, they do NOT form a Linearly Independent Set because no linear combination of path1 and path3 can get , say, path4.

27. More on Linearly Independent Paths • Because the Linearly Independent Set of paths display the same characteristics as the mathematical concept of basisin n-dimensional vector space, the testing using the Linearly Independent Set of paths is sometimes called the “basis” testing. • The main notion is that since the “linear independent set of paths” as a set can span all the paths for the design/code construct, then basis testing covers the “essence” of the whole structure. Then ------ is there a way to find a Linearly Independent Set ?

28. An Algorithm to Find the Basis Set • Select a baseline path, an arbitrary, normal execution path that contains as many decisions as possible. • Retrace the baseline path and “flip” each of the decision encountered; each flip creates a new path. Continue until all the decisions are flipped • The basis set is composed of all the paths generated from steps 1 and 2 above Cyclomatic # = 2+ 1 = 3; So there are 3 linearly independent paths C1 2 1 S1 1. pick baseline path P1: C1 –S1- C2 – S2: <0,1,1,0,1,1> 2. flip C1 P2 : C1 – C2 – S2 : <1,0,0,0.1,1> 3. flip C2 P3: C1 - C2 : <1,0,0,1,0,0> 3 C2 5 4 Can we get the 4th path : C1 – S1 – C2 : <0,1,1,1,0,0> from the above basis set? How about : (P1 + P3) – P2 ? (P1 + P3) – P2 = (<0,1,1,0,1,1> + <1,0,0,1,0,0>) - <1,0,0,0,1,1> = <1,1,1,1,1,1> - <1,0,0,0,1,1> = <0,1,1,1,0,0> = P4 S2 6

29. Total Possible Logical Paths can be Big! s1 There are 5 choices each time we process through this loop. For passing through the loop n times we have 5n possibilities of logical paths. If we go through the loop just 3 times, we have (5)3 = 125 possible paths! c1 1 c2 c3 2 4 s2 5 3 s3 You also note that for n loops there are 3n decisions, and we have 3*3 = 9 decisions for looping 3 times. c4 s4

30. Paths Analysis • Interested in total number of all possible combinations of “logical” paths • Interested in Linearly Independent paths • Interest in Branch coverage or DD-path • Interested in Statement coverage Which one do you think is the largest set, next largest, - - - , etc.? Compare this list with page 136 of your text - - - what are we missing?

31. Definition for DD-path • DD (decision-decision) path is a path of nodes in a directed graph. • A chain is a path in which: • Initial and terminal nodes are distinct (not just one node) • All interior nodes have in-degree = 1 and out-degree =1 • A DD-path is a chain in a program graph such that the following are included in the chain: • It consists of a single node with in-degree = 0 (initial node) • It consists of a single node with out-degree = 0 (terminal node) • It consists of a single node with in-deg => 2 or out-deg => 2 • It consists of a single node with in-deg = 1 and out-deg = 1 • It is a maximal chain of length => 1.

32. Condensation of Code to Table then to Graph Psuedo-code Path/node name DD-path Def. code statement 1. Program Triangle 2. Dim a, b,c As Integer 3. Dim IsTriangle As Boolean 4. Output ( “enter a,b, and c integers”) 5. Input (a,b,c) 6. Output (“side 1 is”, a) 7. Output (“side 2 is”, b) 8. Output (”side 3 is”, c) 9. If (a<b+c) AND (b<a+c) And (c<b+a) 10. then IsTriangle = True 11. else IsTriangle = False 12. endif 13. If IsTriangle 14. then if (a=b) AND (b=c) 15. then Output (“equilateral”) 16. else if (a NE b) AND (a NE b) AND (b NE c) 17. then Output ( “Scalene”) 18. else Output (“Isosceles”) 19. endif 20. endif 21. else Output (“not a triangle”) 22. endif 23. end Triangle2 Skip 1- 3 (or w/4) 4 first 1 5 – 8 A 5 9 B 3 10 C 4 11 D 4 12 E 3 13 F 3 14 H 3 15 I 4 16 J 3 17 K 4 18 L 4 19 M 3 20 N 3 21 G 4 22 O 3 23 last 2 Def of DD-paths on Page 140

33. Condensation Graph from Table - Statements coverage - 4 paths - Branch (DD-path) coverage - 4 paths - Cyclomatic # = 4+1 = 5 - 5 lin. Ind paths - All combinations - 8 paths first A B C D E F H G J I K L M N O Last

34. Closer Look into Path Testing • Look at 2 paths from A to E. The paths of either it is a triangle or not a triangle. • But there are many combinations to get “not triangle” ; so we still need to consider utilizing boundary values and equivalence class. Just one test case to cover the path may not be enough • Look at the path that leads from D to L (D= not triangle and L= Isosceles triangle). Is that path possible - - - can you generate a test case for that path? • Look at the path from C to G (C = Is a triangle and G = output “not triangle”) - - - is this a possible path? • There may be logical dependencies that prevent us from generating test case to traverse certain paths. Thus we also need to consider employing decision tables from Black Box testing technique.

35. Slicing • The notion of a program slice is useful in software testing, program debugging, automatic parallelization, and program integration • A slice of a program is taken with respect to a program point P and a variable x. The slice consists of all program statements that may affect the value x at program point P. The tuple <P, x> is called a slicing criterion • A Forward Slice of a program with respect to a program point P and variable x consists of all statements of the program that may be affected by the value of x at point P • One way to compute a slice is by constructing a Program Dependence Graph (PDG) and then appropriately traverse this graph

36. Program Dependency Graphs (1) • Directed graphs. Three types of vertices: • Entry vertex • Initial-state(x) for every variable x such that, there exists a path in the CFG on which x is used before it is defined. It represents an assignment to x from the initial state • Final-Use(x) for every variable x named in P’s end statement. It represents an access to the final value of x computed by P

37. Program Dependence Graphs (2) • Three types of edges: • Control Dependence edges labeled as True or False. The source is either a predicate or the entry vertex. A control dependence means that during program execution, if the label of the edge matches the execution result of the source vertex predicate then, the target vertex will be eventually executed • Data Dependence between two nodes iff one is defining a variable and the other is using the variable. Two types of data dependence edges: • Flow dependence • Loop carried • Loop independent • Def-order dependence

38. Control Dependence Edges • A Program Dependence Graph of a program P contains a Control Dependence edge from vertex v1 to vertex v2 (v1c v2 iff one of the following holds: • Vertex v1 is the entry vertex and v2 represents a component of program P that is not nested within a loop or conditional; these edges are labeled True • Vertex v1 represents a control dependence and v2 represents a component of P immediately nested within a loop or a conditional statement whose predicate is represented by v1. If v1 is a test predicate of a loop structure the vertex is labeled True. If it is a test predicate in a conditional statement the edge is labeled according to the boolean result of the predicate.

39. Data Dependence Edges • A Program Dependence Graph of a program P contains a Data Dependence edge from vertex v1 to vertex v2 (v1f v2 iff all of the following holds: • Vertex v1 is a vertex that defines variable x • Vertex v2 is a vertex that uses variable y • Variables x and y are either the same variable or aliases • Control can reach v2 after v1, via an execution path along which there is no other intervening definition of x or y. That is there is a path in the program’s CFG by which definition of x at v1 reaches the use of x at v2. • Data flow dependence edges can be further classified as loop carried and loop independent

40. Loop Carried Data Dependence • Loop carried dependencies between vertex v1 and vertex v2 denoted as v1lc(L) v2occur when in addition to the other criteria for data dependence we have: • There is an execution path that includes a back edge to the predicate of the loop L • Both vertex v1 and vertex v2 are enclosed in the loop L • If there is no back edge to the predicate the data dependence is called loop independent

41. Def-Order Dependence • A PDG contains a def-order dependence edge from vertex v1 to vertex v2 with witness vertex v3 denoted as v1do(v3) v2 iff all of the following hold: • Vertex v1 and v2 define variables x1 and x2 respectively • Variables x1 and x2 are either aliases or the same variable • Vertex v1 and v2 and in the same branch of any conditional statement that encloses both of them • There exists a program component v3 that v1 f v2 and v2 f v3 • Vertex v1 occurs to the left of v2 in the program’s AST (i.e .the statement related to v1 executes before the statement related to vertex v2)

42. Example Legend: Control dependence Loop independent data dependence Loop carried data dependence program Sum sum = 0; i=1; while i < 10) do sum = sum +i; i = i+1; enddo end Def-order dependence ENTRY T T T T T FinalUse(sum); FinalUse(i); while i < 10; sum = 0 i = 1; T T sum=sum+1; i = i +1;

43. Computing a Slice Using a PDG (1) • For vertex S of the Program Dependence Graphs G of a program P, the slice of G with respect to S is a graph (denoted as G/S) containing all vertices of which S has a transitive flow or control dependence (i.e. all vertices that can reach S via flow or control edges) • The edges in the slice graph G/S are data dependence (loop independent), control dependence, and ded-order edges of the original graph G that have source and targets in vertices in G/S

44. Computing a Slice Using a PDG (2) • Therefore the vertices of a slice G/S are: V(G/S) = {w | w in V(G) and w * S} • We can extend the definition to a slice with respect to a set of vertices S = Ui Si V(G/S) = V(G/ Ui Si) = Ui (V(G/Si)) • The edges in a slice are: E(G/S) = {u f w | u  w in E(G) and u, w in V(G/S)} U {u c w | u  w in E(G) and u, w in V(G/S)} U {u do w | u  w in E(G) and u, w in V(G/S)} c,f

45. Algorithm for Intra-Procedural Slice Using a PDG Program ComputeSlice(G, S) G: a program dependence graph S: a set of vertices in G WorkList: a set of vertices in G v, wc: verices in G Begin WorkList = S; while Worklist ≠ 0 do begin Select and remove vertex v from WorkList; Mark v For each unmarked vertex w such that w f v or w c v or w do v do begin Insert w in WorkList end end End

46. Example (1) ENTRY T T T T T FinalUse(sum); FinalUse(i); sum = 0 while i < 10; i = 1; T T sum=sum+1; i = i +1; Slice based of the criterion FinalUse(i)

47. Example (2) program SumSlice i=1; while i < 10) do i = i+1; enddo end program Sum sum = 0; i=1; while i < 10) do sum = sum +i; i = i+1; enddo end Slice on FinalUse(i) ENTRY T T T i = 1; while i < 10; FinalUse(i); T i = i +1;

48. Slicing using Control Flow Graphs • Given a program P, and a program graph G(P) in which statements and statement fragments are numbered, and a set V of variables in P, the slice of the set of variables V at statement fragment n, written as S(V, n), is the set node numbers of all statement fragments W in P prior to n that contribute to the values of variables in V at point n • Contribute means that a statement in S(V, n) will affect the c-use or a p-use of a variable v in V at statement n. Actually, we refine here the concepts of c-use and p-use. More specifically, • P-use: used in a predicate • C-use: used in a computation • O-use: used for output • L-use: used for location (pointers, array indices) • I-use: iteration (loop counters, internal counters) • Also we can distinguish two definition nodes: • I-def: defined by input • A-def: defined by assignment

49. Slices • We can consider that in a slice S(V. n), V has only one element say variable v. If n is a defining node for v then is included the slice. If n is a usage node for v is not included in the slice • p-uses, and c-uses of other variables not in V are included as long their execution affects the value of variable v at point n • We chose to exclude O-uses, L-uses, and I-uses from the slice

50. Slices – Points of Interest • Do not make slices with a criterion (V, n) where variables of V do not appear in n • Consider making slices with respect of one variable not a set of variables • Consider making slices for all A-def nodes (focus is on the variables in the right hand side of an assignment) • Consider making slices for p-use nodes • Consider making slices compilable