structural testing n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Structural Testing PowerPoint Presentation
Download Presentation
Structural Testing

Loading in 2 Seconds...

play fullscreen
1 / 28

Structural Testing - PowerPoint PPT Presentation


  • 565 Views
  • Uploaded on

Structural Testing. Structural testing measures Flowcharts Structural testing and flowcharts Minimal test suites. Program Flowcharts. main () { int x, pos, even; printf("Enter an integer:"); scanf("%d", &x); pos = 0; even = 0;

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Structural Testing' - paul2


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
structural testing
Structural Testing
  • Structural testing measures
    • Flowcharts
    • Structural testing and flowcharts
    • Minimal test suites
program flowcharts
Program Flowcharts

main () {

int x, pos, even;

printf("Enter an integer:");

scanf("%d", &x);

pos = 0;

even = 0;

if (x > 0) {pos = 1;}

if (x%2 == 0) {even = 1;}

printf ("%d, %d, %d\n", x, pos, even);

return 0;

}

  • Example of code:
    • Intended to set pos = 1 iff x is positive, even = 1 iff x is even
  • Draw a graph of the statements and decisions in the code:
    • Statements = boxes
    • Decisions = diamonds
  • This graph called a “flowchart”
flowchart example

Pos=1

Pos=0

Even=0

Flowchart Example

Printf()

T

X>0

F

Scanf()

T

X%2=0

Even = 1

F

Printf()

End

flowchart example1
Flowchart Example
  • A flowchart is really a labeled directed graph
    • Boxes and diamonds = nodes
    • Arrows = edges
  • Edges indicate parts of “paths” which may be followed through code
  • Labels on edges going out of diamonds indicate what happens when decision in diamond is true/false
flowcharts example 2 code
Flowcharts: Example 2 (Code)
  • Code:
    • Changes lower-case letters in s to uppercase

i = 0;

while (s[i] != 0) {

if (s[i] >= ‘a’) && (s[i] <= ‘z’)) {

s[i]=s[i] – 32;

}

i++;

}

structural testing and flowcharts
Structural Testing and Flowcharts
  • What does this have to do with testing?
  • If test suite executes all statements, then test suite visits every box and diamond in the flowchart
    • We say the test suite covers every node
    • We can also say the test suite covers every statement
  • This does not mean that every arrow (edge) in the flowchart has been followed
  • Example: consider the first example flowchart
    • We run it on one test case: x=2
    • All nodes visited
    • However, neither F branch has ever been taken
  • If every arrow in the flowchart has been taken by some case in the test suite:
    • We say the test suite covers every edge
    • We can also say the test suite covers every decision
edge vs node coverage
Edge vs. Node Coverage
  • Edge cover is more thorough than node coverage
    • Catches more problems
  • Example: consider first example flowchart
    • Change statements pos =0; even = 0 in the first column to pos = 1; even = 1
    • Now, program always sets pos and even to 1
    • But we wanted pos to indicate whether x is positive, even to indicate whether x is even
    • If we just test with x = 2
    • If we test with both x = 2 and x = -1:
      • We cover every edge
      • We also force a failure to happen, program reports wrong results for x = -1
  • Thus, edge coverage can reveal failures where node coverage cannot.
coverage terminology
Coverage Terminology
  • If a test suite executes 100% of statements in program, we say it achieves
    • 100% node coverage or 100% statement coverage
  • If a test suite follows 90% of edges in flowchart, we say it achieves 90% edge coverage (etc)
  • “Line Coverage”: what gcov measures
    • Any line containing code is measured
      • Considered covered if any code on line is executed
    • Not exactly the same thing as statement coverage
      • Can have more than one statement on a line
    • However, 100% statement coverage implies 100% line coverage
  • Edge coverage also called decision coverage
    • Tests each decision in an if, while, etc. both ways
strength of coverage measures
Strength of Coverage Measures
  • 100% node coverage implies 100% line coverage
    • However, 100% line coverage does not imply 100% node coverage
  • 100% edge coverage implies 100% node coverage
    • If followed every edge, must have visited every node
    • However, as we saw, 100% node coverage does not necessarily imply 100% edge coverage
  • Thus we say:
    • Node (statement) coverage is strong than line coverage
    • Edge (decision) coverage is stronger than node coverage
  • We can draw a graph:
    • Arrow from A to B: B is a stronger coverage measure than A

Edge

(Decision)

Node

(Statement)

Line

minimal test suites
Minimal Test Suites
  • By throwing more and more test cases into a test suite, we may achieve higher coverage
  • However, longer test suites take longer to run
  • Hence, we often want to find some minimal test suite
    • Important to define what minimal means i.e. minimal test cases? Minimal number of input used? Minimum time to run?
  • Example: first flowchart example:
    • Test suite: x=2, x=1, x=-2 covers all edges
    • However, it has 3 test cases
    • Test suite: x=2, x=-1 covers al edges and has only 2 test cases
    • Cannot have a minimal test suite which covers all edges and has only 1 test case
      • Have to execute each decision both ways
  • Hence, x=2, x=-1 is a minimal test suite for 100% edge coverage
    • Minimal in the sense of needing the fewest test cases
coverage and test suites another example
Coverage and Test Suites: Another Example
  • Consider again the case-conversion code flowchart (Example 2)
  • Find “minimal” test suite which achieves:
    • 100 % node coverage
    • 100% edge coverage
  • Decide what minimalmeans in this context!
case conversion flowchart coverage
Case Conversion Flowchart: Coverage
  • Clearly we can achieve 100% node coverage with one-char test case
    • Save, “a”
  • This test case also follows both arrows from the top diamond
    • This is because:
      • First time through loop (i==0), T branch taken
      • Next time condition is evaluated (i==1), s[i] is the null (end of string) character
      • Hence, F branch is also taken
  • However, F branch of other diamond not taken
  • To cover all edges, we need test suite like
    • (A) s= “a”, s=“X”; or
    • (B) s = “aX”
case conversion flowchart minimality
Case Conversion Flowchart: Minimality
  • Which of (A) and (B) is “minimal”
    • Depends on how we define minimality
    • (A) has 2 test cases, but each has just 1 character
    • (B) has just 1 test case, but it has 2 characters
  • We should always define precisely what we mean by minimality for a given test suite
    • Here, makes more sense to accept (B) as minimal
    • Don’t’ have to run program as many times, incur overhead of starting up process again
  • Hence, for this problem, we should say something like:
    • Test suite X will be considered smaller than test suite Y if either:
      • X has few test cases than Y; or
      • X has the same number of test cases as Y, but the total number of characters in X is smaller
    • Saves us from having to accept s = “aaaaaaXXXXXX” as minimal
stronger coverage measures
Stronger Coverage Measures
  • So far we have been writing compound decisions
    • (e.g. (c >=‘a’) && (c <=‘z’)) inside a single diamond
  • Decision: the whole thing in parentheses after the “if”, while”, etc
    • Consider the code:if ((c>=‘a’) && (c<=‘z’)) {..}
    • (c >=‘a’) && (c<=‘z’) is the decision
  • Conditions: the individual terms of the condition, connected by logical operators
    • (c>=‘a’) && (c<=‘z’) are the two conditions
  • If we separate each condition with the decision into a separate diamond:
    • We can get a better reflection of what the program does
splitting up diamonds flowcharts

s[i]>=‘a’

s[i]<=‘z’

s[i]>=‘a’ && s[i]<=‘z’

Splitting Up Diamonds: Flowcharts
  • Example:
    • Instead of writing

we write

    • Now we have two new arrows
    • To cover all arrows, we have to cover the two ones as well

F

T

F

T

F

T

short circuit condition coverage
Short Circuit Condition Coverage
  • If we split up all diamonds and then cover all arrows:
    • We are achieving more thorough coverage than just edge (decision) coverage can give us
  • This is called short circuit (SC) condition coverage
    • Recall definition of short circuit evaluation
      • Evaluation of a Boolean expression is cut short as soon as it can be determined to be true or false
    • Applies to language with short circuit evaluation of decisions, e.g. C, C++, Java
  • Similar but not identical to another notion called just “condition coverage”
    • Condition coverage applies to languages without short circuit evaluation of decisions, e.g. Visual Basic
    • We will not study this coverage measure
achieving sc condition coverage
Achieving SC Condition Coverage
  • Consider the case conversion example
  • Test case “aX” is sufficient to achieve 100% edge coverage
    • Whole decision (c>=‘a’) && (c<=‘z’) is tested both ways: true for ‘a’ and false for ‘X’
  • Is not sufficient to achieve 100% SC condition coverage
    • Have not take the F arrow from the s[i] <= ‘z’ diamond
  • To take that arrow we need a character that is:
    • Greater than or equal to ‘a’
    • Greater than ‘z’
  • We need one such ASCII character
    • The test case “aX~” will achieve 100% SC condition coverage
achieving sc condition coverage in general
Achieving SC Condition Coverage in General
  • In general, for a decision A && B to achieve SC condition coverage, we need to design test cases to make:
    • A true, B true
    • A true, B false
    • A false
  • We will not even be able to get to the evaluation of B unless A is true as in first two cases (due to SC evaluation)
  • Similarly, for a decision A || B to achieve SC condition coverage, we need to design test cases to make:
    • A false, B true
    • A false, B false
    • A true
  • We will not even be able to get to the evaluation of B unless A is false as in the first two cases
sc condition coverage vs edge coverage
SC Condition Coverage vs. Edge Coverage
  • If we have achieved SC condition coverage:
    • We must have evaluated each condition of an “if”, “while”, etc. both ways
    • Therefore, we must have evaluated each decision both ways
  • Therefore, SC condition coverage is stronger than edge (decision) coverage
  • Is there anything stronger than SC condition coverage?
path coverage
Path Coverage
  • “100% path coverage”: every possible path through the flowchart of the code has been executed by some test case
    • Path: sequence of nodes visited during an execution
      • Can contain repeated elements if execution has visited a node more than once
    • The strongest possible coverage measure
  • For code with no loops, this is achievable
  • For code with while loops:
    • Going around loop a different number of times means we have taken a different path
  • For some code, we can go around a loop any number of times
  • Therefore, for some code:
    • it is not possible to achieve 100% path coverage
path coverage example 1

Printf()

T

X>0

F

Pos=1

Pos=0

Even=0

Scanf()

T

X%2=0

Even = 1

F

Printf()

End

Path Coverage: Example 1
  • Integer classification example again:
  • Four paths through the code are possible
    • X = 2
    • X = 1
    • X = -2
    • X = -1
  • Draw the paths for each of the above examples!
  • Therefore we need four test cases for 100% coverage
path coverage example 2

s[i]>=‘a’&&s[i]<=‘z’?

s[i]!=0?

i=0

i++

End

s[i]=s[i]-32

Path Coverage: Example 2
  • Case conversion example again
  • Every string of a different length will follow a different path

F

T

F

T

path coverage for loops
Path Coverage for Loops
  • Path coverage for many while loops is impossible
  • Most programs have while loops!
  • Most have at least one that can be followed an arbitrary number of times:
    • e.g. A top level loop where we read a command and execute it
  • Therefore, for most whole programs, we don’t even attempt 100% path coverage
  • Path coverage is still useful for small sections of code
strength of coverage measures1
Strength of Coverage Measures

Path (strongest)

Edge

(Decision)

Node

(Statement)

Line (weakest)

Where arrow from A to B means “B is stronger than A”

testing loops
Testing Loops
  • If we have loops, we can’t test all possible paths
  • However, just decision coverage may not spot usual kinds of errors
  • Often code is wrong because programmer didn’t consider what would happen when:
    • Loop decision is false right from the start
    • Loop decision is true once, then false
  • Sometimes there is a maximum number (max) of times loop can be executed
    • Example: loop may stop at end of an array, or when some condition is true
  • In these cases, code may be wrong because programmer didn’t consider what would happen when
    • Loop decision is true max times
    • Loop decision is true max-1 times
loop coverage
Loop Coverage
  • It is therefore useful to write test cases which:
    • Execute loop 0 times
    • Execute loop 1 time
    • Execute loop more than 1 time
    • Execute loop max times (if max exists)
    • Execute loop max -1 times (if max exists)
  • These tests are often referred to as loop coverage
    • However, there is no real formal definition of loop coverage as with node, edge etc. coverage
loop coverage exercise
Loop Coverage: Exercise
  • Provide adequate loop coverage for the following: (code finds the rightmost dot in a string, used to find extension for filename)
  • i = strlen(fname) -1;
  • while (Ii >0 && fname[i] != ‘.’) {i--;}
  • To do “loop coverage”, list some of the possible test cases:
    • (0 times) 
    • (1 time) 
    • (more than 1 time) 
    • (max – 1 times) 
    • (max times) 

Note: max her is (length of the string –1)

  • Note: the code above is almost certainly correct
    • However, problem may not be with the loop itself, but the code that comes after
    • For instance, maybe the code that comes afterward:
      • Expects a non-empty extension
      • Expects a non-empty filename before the extension
      • Can’t handle the case of no extension
    • Loop testing will help find those problems too