software testing quality assurance lecture 10 created by paulo alencar modified by frank xu n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Software Testing & Quality Assurance Lecture 10 Created by: Paulo Alencar Modified by: Frank Xu PowerPoint Presentation
Download Presentation
Software Testing & Quality Assurance Lecture 10 Created by: Paulo Alencar Modified by: Frank Xu

Loading in 2 Seconds...

play fullscreen
1 / 25

Software Testing & Quality Assurance Lecture 10 Created by: Paulo Alencar Modified by: Frank Xu - PowerPoint PPT Presentation


  • 89 Views
  • Uploaded on

Software Testing & Quality Assurance Lecture 10 Created by: Paulo Alencar Modified by: Frank Xu. Overview. Structural Testing Introduction – General Concepts Flow Graph Testing DD-Paths Test Coverage Metrics Basis Path Testing Guidelines and Observations Data Flow Testing Hybrid Methods

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Software Testing & Quality Assurance Lecture 10 Created by: Paulo Alencar Modified by: Frank Xu' - hija


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
overview
Overview
  • Structural Testing
    • Introduction – General Concepts
    • Flow Graph Testing
      • DD-Paths
      • Test Coverage Metrics
      • Basis Path Testing
      • Guidelines and Observations
    • Data Flow Testing
    • Hybrid Methods
    • Retrospective on Structural Testing
path testing
Path Testing
  • Path Testing is focusing on test techniques that are based on the selection of test paths though a program graph. If the set of paths is properly chosen, then we can claim that we have achieved a measure of test thoroughness.
  • The fault assumption for path testing techniques is that something has gone wrong with the software that makes it take a different path than the one intended.
  • Structurally, a path is a sequence of statements in a program unit. Semantically, a path is an execution instance of the program unit. For software testing we are interested in entry-exit paths.
path testing process
Path Testing Process
  • Input:
    • Source code and a path selection criterion
  • Process:
    • Generation of a CFG
    • Selection of Paths
    • Generation of Test Input Data
    • Feasibility Test of a Path
    • Evaluation of Program’s Output for the Selected Test Cases
slide5

V(

G) = 3

1

Basis set:

x = z+5

1, 2, 3, 4, 6, 7

z = 4*3

-

y

1, 2, 3, 4, 5, 4, 6, 7

1, 2, 6, 7

x > z

2

t

Flow Graphs – Use in determining

Paths for Testing - Revisited

x = z+5

6

z = 4*3

-

y

f

R1

R2

if(x > z) goto A;

for( u=0; u < x; u++) {

3

z = z+1;

y = z+k

};

5

t

f

u = 0

A: y = z + k

z = z+1

R3

7

u++

4

u < x

Example of a simple control flowgraph.

program graph to dd path graph

4

5

6

7

8

9

11

10

12

13

14

21

15

16

17

18

20

22

19

23

Program Graph to DD-Path Graph

first

B

D

C

E

F

H

G

I

J

K

L

N

O

M

last

topological paths
Topological Paths
  • We can identify eight possible paths through the previous diagram

1. first – A – B – C – E – F – G – O – last

2. first – A – B – D – E – F – G – O – last

3. first – A – B – C – E – F – H – I – N – O – last

4. first – A – B – D – E – F – H – I – N – O – last

5. first – A – B – C – E – F – H – J – K – M – N – O – last

6. first – A – B – D – E – F – H – J – K – M – N – O – last

7. first – A – B – C – E – F – H – J – L – M – N – O – last

8. first – A – B – D – E – F – H – J – L – M – N – O – last

feasible paths
Feasible Paths
  • If we assume that the logic of the program dictates that “if node C is traversed, then we must traverse node H, and if node D is traversed, then we must traverse Node G”
    • These constraints will eliminate Paths 1, 4, 6 and 8
    • We are left to consider four feasible paths:

2. first – A – B – D – E – F – G – O – last

3. first – A – B – C – E – F – H – I – N – O – last

5. first – A – B – C – E – F – H – J – K – M – N – O – last

7. first – A – B – C – E – F – H – J – L – M – N – O – last

slide9

first

A

A

E

B

C

G

F

D

I

H

L

K

J

last

  • Let’s consider the Triangle program, its CFG, and its DD-Path graph.

[Jorgensen: Software Testing A Craftsman’s Approach]

1. program triangle (input, output);

2. VAR a,b,c: integer;

3. IsATriangle: boolean;

4. BEGIN

5. writeln(“Enter integers: “);

6. readln(a,b,c);

7. Writeln(“Side A is: “, a, “Side B is: “, b, “Side C is: “, c);

8. IF(a <b+c) AND (b < a+c) AND (c<a+b) THEN

9. IsATriangle = True;

10. ELSE IsATriangle = False;

11. IF IsATriangle)

12. THEN

13. BEGIN

14. IF (a=b) XOR (a=c) XOR (b=c) AND NOT((a=b) AND (a=c)

15. THEN writeln(“Equilateral”);

16. IF (a=b) AND (b=c)

17. THEN writeln(“Equilateral”);

18. IF(a <> b) AND (a <> c) AND (a<>c)

19. THEN writeln(“Scalene”);

20. END

21. ELSE Writeln(“Not a Triangle”);

22. END

P1: A-B-D-E-G-I-J-K-Last

P2: A-C-D-E-G-I-J-K-Last

P3: A-B-D-L-Last

P4: A-B-D-E-F-G-I-J-K-Last

P5: A-B-D-E-F-G-H-I-J-K-Last

P6: A-B-D-E-F-G-H-I-K-Last

slide10

first

A

A

E

B

C

G

F

D

I

H

L

K

J

last

  • Let’s consider the Triangle program, its CFG, and its DD-Path graph.

[Jorgensen: Software Testing A Craftsman’s Approach]

1. program triangle (input, output);

2. VAR a,b,c: integer;

3. IsATriangle: boolean;

4. BEGIN

A5. writeln(“Enter integers: “);

A6. readln(a,b,c);

A7. Writeln(“Side A is: “, a, “Side B is: “, b, “Side C is: “, c);

A8. IF(a < b+c) AND (b < a+c) AND (c < a+b) THEN

B9. IsATriangle = True;

C10. ELSE IsATriangle = False;

D11. IF IsATriangle)

D12. THEN

E13. BEGIN

E14. IF (a=b) XOR (a=c) XOR (b=c) AND NOT((a=b) AND (a=c)

F15. THEN writeln(“Equilateral”);

G16. IF (a=b) AND (b=c)

H17. THEN writeln(“Equilateral”);

I18. IF(a <> b) AND (a <> c) AND (a<>c)

J19. THEN writeln(“Scalene”);

K20. END

L21. ELSE Writeln(“Not a Triangle”);

22. END

Node B is traversed  D, E are traversed

Node C is traversed  D, L are traversed

Node E is traversed  F, H, J are traversed

Node F is traversed  H, J are traversed

Node H is traversed  F, J are traversed

Node J is traversed  F, I are traversed

Logically feasible paths:

P1: A-C-D-L-Last

P2: A-B-D-E-F-G-I-K-Last

P3: A-B-D-E-G-H-I-K-Last

P4: A-B-D-E-G-I-J-K-Last

structured programming constructs
Structured Programming Constructs

s1

s1

s1

s1

s1

s1

s2

s3

s4

s2

s3

s2

s2

s2

s2

s3

s5

s4

s3

Sequence if-then-else case if-then loop-pre-test loop-post-test

violations of structured programming constructs
Violations of Structured Programming Constructs

s1

s1

s1

s1

s0

s0

s2

s2

s2

s2

s0

s1

s3

s3

s3

s3

Branching into loop Branching out of loop Branching into decision Branching out of decision

essential complexity
Essential Complexity
  • Essential complexity. This measures how much unstructured logic exists in a module (e.g., a loop with an exiting GOTO statement)
  • The essential complexity can be computed on a program graph that has been condensed by collapsing structured programming constructs to a single node, until no more structured constructs can be found
condensing a program graph
Condensing A Program Graph

first

first

a

a

d

A

b

A

b

E

B

C

H

c

G

F

D

L

c

I

H

last

L

K

J

last

e

first

a

first

L

d

V(G) = 1

e

last

some fundamental path selection criteria
Some Fundamental Path Selection Criteria
  • Exercise every statement or instruction at least once (statement coverage) C0
  • Exercise every branch and case statement, in each direction, at least once (branch coverage) C1
  • Exercise each condition in a decision (condition coverage)
  • Exercise each condition in a decision with all possible outcomes C1P
  • Exercise every compound predicate outcome (MCC) CMCC
  • Exercise every path from entry to exit C∞
example
Example

1

2

Statement Coverage C0:

SCPath1: 1-2-3(F)-10(F)-11-13

SCPath2: 1-2-3(T)-4(T)-5-6(T)-

7(T)-8-9-3(F)-10(T)-12-13

F

T

3

F

10

4

F

T

T

5

12

11

Branch or Decision Coverage C1:

BCPath1: 1-2-3(F)-10(F)-11-13

BCPath2: 1-2-3(T)-4(T)-5-6(T)-

7(T)-8-9-3(F)-10(T)-12-13

BCPath3: 1-2-3(T)-4(F)-10(F)-11-13

BCPath4: 1-2-3(T)-4(T)-5-6(F)-9-3(F)-

10(F)-11-13

BCPath5: 1-2-3(T)-4(T)-5-6(T)-7(F)-9-

3(F)-10(F)-11-13

6

F

T

13

7

F

T

8

9

condition coverage
Condition Coverage
  • Requires that each condition in a compound decision takes on all possible outcomes at least once
  • Condition Coverage does not require that each decision takes on all possible outcomes
  • Example: if Not (A or B) then C

Setting A = True, B = False, then A = False and B = True satisfies Condition Coverage, but statement C will never be executed

multiple condition coverage
Multiple Condition Coverage
  • Requires that each possible combination of inputs be tested for each decision.
  • Example: “if (A and B)” requires 4 test cases:

A = True, B = True

A = True, B = False

A = False, B = True

A = False, B = False

  • For n conditions, 2n test cases are needed, and this grows exponentially with n
modified mcc
Modified MCC
  • Requires that each condition be shown to independently affect the outcome of a decision
  • Generally, for n conditions, MC/DC requires only n + 1 test cases
  • Example: if (A or B) then C

3 test cases: A = True, B = False (A dominates)

A = False, B = True (B dominates)

A = False, B = False (false outcome)

examples
Examples
  • To test if (A or B)

A: T F F

B: F T F

  • To test if (A and B)

A: F T T

B: T F T

  • To test if (A xor B)

A: T T F

B: T F T

guidelines and observations
Guidelines and Observations
  • Functional testing focuses on properties that are “too far” or disassociated from the source code being tested
  • Path testing focuses too much on the graph and not the logic of the code
  • Path testing very useful to measure the quality of the testing (coverage criteria)
  • Basis path testing is giving us a lower bound of how much testing is necessary
  • Path testing is providing a set of metrics that act as cross checks to functional testing
    • When a program path is traversed by several functional test cases we suspect redundancy
    • When we fail to attain DD-Path graph coverage, we know that there are gaps is the functional test cases
  • Coverage metrics can be used as
    • Setting minimum testing standards and as
    • Mechanism to selectively test portions of the code more rigorously than others
guidelines and observations1

S

P

T

Guidelines and Observations
  • We can distinguish between
    • Specified program behavior (S),
    • Programmed behavior (feasible paths) (P), and
    • Topologically feasible paths (T)
guidelines and observations2

S

P

T

Guidelines and Observations
  • Region 1 contains specified behavior that is implemented by feasible paths – most desirable
  • Every feasible path is topologically possible so the parts 2,6 of set P should be empty
  • Region 3 contains feasible paths that corresponds to unspecified behavior. Such extra functionality must be inspected
  • Regions 4 and 7 contain the infeasible paths. Region 4 is problematic as it corresponds to specified behavior that has almost been implemented (topologically possible, yet infeasible program paths) – coding errors
  • Region 7 refers to unspecified, infeasible, yet topologically possible paths – potential maintenance problem – could become Region3

2

6

5

1

4

3

7

software testing standards
Software Testing Standards
  • IEEE 1008 Software Unit Testing
  • IEEE 1044 Classification for Software Anomalies
  • ISO/IEC 12207 Information Technology--Software Life Cycle Processes
  • ISO/IEC TR 15271 Guide for ISO/IEC 12207 (Software Life Cycle Processes)
  • AECL CE-1001-STD REV.1 Standard for Software Engineering of Safety Critical Software
  • BSI BS-7738 Specification for Information Systems Products Using SSADM
  • (Structured Systems Analysis and Design Method)
  • BSI BS-7925-1 Software Testing - Vocabulary
  • BSI BS-7925-2 Standard for Software Component Testing
  • DEF 00-55 (Part 1)/1 The Procurement of Safety Critical Software in Defence Equipment-Requirements
  • DIN VDE 0801 Principles for Computers in Safety-Related Systems
  • ESA PSS-05-01 Guide to the Software Engineering Standards, Issue 1
  • German Process-Model (V-Model) Software Life-Cycle Process Model
  • IAEA TRS-372 Development and Implementation of Computerized Operator Support Systems in Nuclear Installations
  • IEC 60601-1-4 Medical Electrical Equipment--Part 1: General Requirements for Safety-4.Collateral Standard: Programmable Electrical Medical Systems
  • IEC 60880 Software for Computers in the Safety Systems of Nuclear Power Stations
  • IEE 5 Software Inspection Handbook
  • NCC STARTS Purchasers' Handbook--Procuring Software-Based Systems
  • NRC NUREG/BR-0167 Software Quality Assurance Program and Guidelines
  • RTCA DO-178B/ED-12B Software Considerations in Airborne Systems and Equipment Certification