1 / 94

Black-Box Testing Techniques II

Black-Box Testing Techniques II. Software Testing and Verification Lecture 5. Prepared by Stephen M. Thebaut, Ph.D. University of Florida. Black-Box Test Case Design Techniques Considered. Partition testing Combinatorial Approaches Boundary Value Analysis Intuition & Experience.

cassidy-rosales
Download Presentation

Black-Box Testing Techniques II

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Black-Box Testing Techniques II Software Testing and Verification Lecture 5 Prepared by Stephen M. Thebaut, Ph.D. University of Florida

  2. Black-Box Test Case Design Techniques Considered • Partition testing • Combinatorial Approaches • Boundary Value Analysis • Intuition & Experience

  3. Cause-Effect Analysis • Cause-Effect Analysis is a combinatorial approach that can be viewed as a logical extension of partition testing. • It extends the idea of partitioning a multi-dimensional input space by providing a systematic means for generating test case templates to cover different combinations of input “Causes” resulting in output “Effects.”

  4. Cause-Effect Analysis • Cause-Effect Analysis is a combinatorial approach that can be viewed as a logical extension of partition testing. • It extends the idea of partitioning a multi-dimensional input space by providing a systematic means for generating test case templates to cover different combinations of input “Causes” resulting in output “Effects.”

  5. Causes and Effects • A CAUSE may be thought of as a distinct input condition, or an “equivalence class” of input conditions. • An EFFECT may be thought of as a distinct output condition or change in program state.

  6. Causes and Effects • A CAUSE may be thought of as a distinct input condition, or an “equivalence class” of input conditions. • An EFFECT may be thought of as a distinct output condition or change in program state.

  7. Causes and Effects • Causes and Effects are represented as Boolean variables. • The logical relationships among them CAN (but need not) be represented as one or more Boolean graphs.

  8. Causes and Effects • Causes and Effects are represented as Boolean variables. • The logical relationships among them CAN (but need not) be represented as one or more Boolean graphs. Л Causes Effects V

  9. C-E Analysis Process Steps • Identify Causes and Effects • The most critical and usually the most difficult step • Choose an appropriate level of abstraction. • Divide and conquer as necessary. • Effects may or may not be mutually exclusive.

  10. C-E Analysis Process Steps • Identify Causes and Effects • The most critical and usually the most difficult step • Choose an appropriate level of abstraction. • Divide and conquer as necessary. • Effects may or may not be mutually exclusive.

  11. C-E Analysis Process Steps • Identify Causes and Effects • The most critical and usually the most difficult step • Choose an appropriate level of abstraction. • Divide and conquer as necessary. • Effects may or may not be mutually exclusive.

  12. C-E Analysis Process Steps • Identify Causes and Effects • The most critical and usually the most difficult step • Choose an appropriate level of abstraction. • Divide and conquer as necessary. • Effects may or may not be mutually exclusive.

  13. C-E Analysis Process Steps • Identify Causes and Effects • The most critical and usually the most difficult step • Choose an appropriate level of abstraction. • Divide and conquer as necessary. • Effects may or may not be mutually exclusive.

  14. C-E Analysis Process Steps (cont’d) • Deduce Logical Relationships and Constraints • Relationships take the form of conditionals and utilize the logical operators AND, OR, and NOT. • Constraints describe relationships among Causes that allow for the identification of infeasible (i.e., impracticable) combinations.

  15. C-E Analysis Process Steps (cont’d) • Deduce Logical Relationships and Constraints • Relationships take the form of conditionals and utilize the logical operators AND, OR, and NOT. • Constraints describe relationships among Causes that allow for the identification of infeasible (i.e., impracticable) combinations.

  16. C-E Analysis Process Steps (cont’d) • Deduce Logical Relationships and Constraints • Relationships take the form of conditionals and utilize the logical operators AND, OR, and NOT. • Constraints describe relationships among Causes that allow for the identification of infeasible (i.e., impracticable) combinations. (cont’d)

  17. C-E Analysis Process Steps (cont’d) • Deduce Logical Relationships and Constraints (cont’d) • Boolean graphs provide a convenient and economical way to visualize relationships and constraints.

  18. C-E Analysis Process Steps (cont’d) • Identify an appropriate Test Case Selection Strategy • Determines the number and nature of Cause-combinations to be considered. • Strategies can be designed to meet a variety of coverage requirements/ cost constraints.

  19. C-E Analysis Process Steps (cont’d) • Identify an appropriate Test Case Selection Strategy • Determines the number and nature of Cause-combinations to be considered. • Strategies can be designed to meet a variety of coverage requirements/ cost constraints.

  20. C-E Analysis Process Steps (cont’d) • Identify an appropriate Test Case Selection Strategy • Determines the number and nature of Cause-combinations to be considered. • Strategies can be designed to meet a variety of coverage requirements/ cost constraints.

  21. C-E Analysis Process Steps (cont’d) • Construct a Test Case Coverage Matrix • Typically involves tracing through the Cause-Effect relationships to identify combinations of Causes resulting in each Effect according to the selection strategy chosen. • This can be extremely tedious if done manually...

  22. C-E Analysis Process Steps (cont’d) • Construct a Test Case Coverage Matrix • Typically involves tracing through the Cause-Effect relationships to identify combinations of Causes resulting in each Effect according to the selection strategy chosen. • This can be extremely tedious if done manually...

  23. C-E Analysis Process Steps (cont’d) • Construct a Test Case Coverage Matrix • Typically involves tracing through the Cause-Effect relationships to identify combinations of Causes resulting in each Effect according to the selection strategy chosen. • This can be extremely tedious if done manually...

  24. Question… To what extent do you think CASE support might be applicable to each step in the process? For which steps do you think it might be most important?

  25. Question… To what extent do you think CASE support might be applicable to each step in the process? For which steps do you think it might be most important? (We’ll come back to this after illustrating the process steps with some examples.)

  26. Illustration of Step 1 (Identify Causes and Effects) The first input is a yes/no response to the question “Do you reside within the city?” The second input is gross pay for the year in question. A non-resident will pay 1% of the gross pay in city tax. Residents pay on the following scale: - If gross pay is no more than $30,000, the tax is 1%. - If gross pay is more than $30,000, but no more than $50,000, the tax is 5%. - If gross pay is more than $50,000, the tax is 15%.

  27. Guidelines for identifying Causes and Effects • Underline words or phrases in the specification that correspond to input/output conditions or changes in state.

  28. Guidelines for identifying Causes and Effects (cont’d) The first input is a yes/no response to the question “Do you reside within the city?” The second input is gross pay for the year in question. A non-resident will pay 1% of the gross pay in city tax. Residents pay on the following scale: - If gross pay is no more than $30,000, the tax is 1%. - If gross pay is more than $30,000, but no more than $50,000, the tax is 5%. - If gross pay is more than $50,000, the tax is 15%.

  29. Guidelines for identifying Causes and Effects (cont’d) • List each Cause and Effect. • Assign a unique number to each (use different number ranges to differentiate Causes from Effects).

  30. Guidelines for identifying Causes and Effects (cont’d) • List each Cause and Effect. • Assign a unique number to each (use different number ranges to differentiate Causes from Effects).

  31. Guidelines for identifying Causes and Effects (cont’d) The first input is a yes/no response to the question “Do you reside within the city?” The second input is gross pay for the year in question. A non-resident will pay 1% of the gross pay in city tax. Residents pay on the following scale: - If gross pay is no more than $30,000, the tax is 1%. - If gross pay is more than $30,000, but no more than $50,000, the tax is 5%. - If gross pay is more than $50,000, the tax is 15%.

  32. Illustration of Step 1 (cont’d) Ignoring, again, the unspecified responses to “invalid” inputs, we have: Causes:Effects: (1) Non-Resident (11) 1% tax (2) Resident (12) 5% tax (3) $0  Gross Pay  $30K (13) 15% tax (4) $30K  Gross Pay  $50K (5) Gross Pay  $50K

  33. Illustration of Step 2 (Deduce Logical Relationships and Constraints) The first input is a yes/no response to the question “Do you reside within the city?” The second input is gross pay for the year in question. A non-resident will pay 1% of the gross pay in city tax. Residents pay on the following scale: - If gross pay is no more than $30,000, the tax is 1%. - If gross pay is more than $30,000, but no more than $50,000, the tax is 5%. - If gross pay is more than $50,000, the tax is 15%.

  34. What are the constraints? Causes:Effects: (1) Non-Resident(11) 1% tax (2) Resident(12) 5% tax (3) $0  Gross Pay  $30K(13) 15% tax (4) $30K  Gross Pay  $50K (5) Gross Pay  $50K

  35. Constraints deducible from spec, problem domain knowledge, etc. • [(1) Л ¬(2)] V [¬(1) Л (2)] (i.e., one, and only one of (1) and (2) must be true.) • [(3) Л ¬(4) Л ¬(5)] V [¬(3) Л (4) Л ¬(5)] V [¬(3) Л ¬(4) Л (5)] • [(11) Л ¬(12) Л ¬(13)] V [¬(11) Л (12) Л ¬(13)] V [¬(11) Л ¬(12) Л (13)]

  36. Constraints deducible from spec, problem domain knowledge, etc. • [(1) Л ¬(2)] V [¬(1) Л (2)] (i.e., one, and only one of (1) and (2) must be true.) • [(3) Л ¬(4) Л ¬(5)] V [¬(3) Л (4) Л ¬(5)] V [¬(3) Л ¬(4) Л (5)] • [(11) Л ¬(12) Л ¬(13)] V [¬(11) Л (12) Л ¬(13)] V [¬(11) Л ¬(12) Л (13)]

  37. Constraints deducible from spec, problem domain knowledge, etc. • [(1) Л ¬(2)] V [¬(1) Л (2)] (i.e., one, and only one of (1) and (2) must be true.) • [(3) Л ¬(4) Л ¬(5)] V [¬(3) Л (4) Л ¬(5)] V [¬(3) Л ¬(4) Л (5)] • [(11) Л ¬(12) Л ¬(13)] V [¬(11) Л (12) Л ¬(13)] V [¬(11) Л ¬(12) Л (13)]

  38. Constraints deducible from spec, problem domain knowledge, etc. • [(1) Л ¬(2)] V [¬(1) Л (2)] (i.e., one, and only one of (1) and (2) must be true.) • [(3) Л ¬(4) Л ¬(5)] V [¬(3) Л (4) Л ¬(5)] V [¬(3) Л ¬(4) Л (5)] • [(11) Л ¬(12) Л ¬(13)] V [¬(11) Л (12) Л ¬(13)] V [¬(11) Л ¬(12) Л (13)]

  39. What are the logical relationships? The first input is a yes/no response to the question “Do you reside within the city?” The second input is gross pay for the year in question. A non-resident will pay 1% of the gross pay in city tax. Residents pay on the following scale: - If gross pay is no more than $30,000, the tax is 1%. - If gross pay is more than $30,000, but no more than $50,000, the tax is 5%. - If gross pay is more than $50,000, the tax is 15%.

  40. Conditionals deducible from specification and constraints • From the specification we have: (1) => (11) [(2) Л (3)] => (11) [(2) Л (4)] => (12) [(2) Л (5)] => (13)

  41. Conditionals deducible from specification and constraints (cont’d) • Which, in light of the identified constraints, simplify to: [(1) V (3) => (11) [(2) Л (4)] => (12) [(2) Л (5)] => (13)

  42. Boolean Graph Representation (1) (11) (3) (4) (12) (2) (13) (5) V Л Л

  43. Boolean Graph Representation Non-Res (1) (11) 1% tax [0,30K] (3) (30K,50K] (4) (12) 5% tax Res (2) (13) 15% tax >50K (5) V Л Л

  44. Boolean Graph Representation Non-Res (1) (11) 1% tax [0,30K] (3) (30K,50K] (4) (12) 5% tax Res (2) (13) 15% tax >50K (5) V O O Л O Л

  45. Cause/Effect Constraints Exclusive Inclusive E “at most one” I “at least one” One & Only One Requires A A =>B O “one and only one” B

  46. Illustration of Step 3 (Identify Test Case Selection Strategy) • Simple (but extreme) strategies: • “All Feasible Combinations of Cause Values” (AFCCV) • “All Effects” (AE) • For the relationships depicted in our graph, how many test cases would be required to achieve AFCCV coverage? AE coverage?

  47. Illustration of Step 3 (Identify Test Case Selection Strategy) • Simple (but extreme) strategies: • “All Feasible Combinations of Cause Values” (AFCCV) • “All Effects” (AE) • For the relationships depicted in our graph, how many test cases would be required to achieve AFCCV coverage? AE coverage?

  48. Illustration of Step 3 (Identify Test Case Selection Strategy) • Simple (but extreme) strategies: • “All Feasible Combinations of Cause Values” (AFCCV) • “All Effects” (AE) • For the relationships depicted in our graph, how many test cases would be required to achieve AFCCV coverage? AE coverage?

  49. Illustration of Step 3 (Identify Test Case Selection Strategy) • Simple (but extreme) strategies: • “All Feasible Combinations of Cause Values” (AFCCV) • “All Effects” (AE) • For the relationships depicted in our graph, how many test cases would be required to achieve AFCCV coverage? AE coverage?

  50. AFCCV and AE Coverage Non-Res (1) (11) 1% tax [0,30K] (3) (30K,50K] (4) (12) 5% tax Res (2) (13) 15% tax >50K (5) V O O Л O Л

More Related