1 / 17

Testing and Test Case Development

Testing and Test Case Development. A “primitive” method of testing, with NO test preparation , may include the following steps : Initiate the system Provide the requested input(s) {unplanned value} Observe the outputs/results Record and analyze the results

Download Presentation

Testing and Test Case Development

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Testing and Test Case Development • A “primitive” method of testing, with NO test preparation, may include the following steps : • Initiate the system • Provide the requested input(s) {unplanned value} • Observe the outputs/results • Record and analyze the results • Note that such testing (ad hoc or random) is very difficult to analyze: • Is the test coverage complete? • How much duplication exist?

  2. Checklist Testing • A more orderly testing than ad hoc testing is to question the “coverage” in some systematic way. • A Check-listof coverage is developed: • The test cases designed & developed are based on the checklist items which maycover different items such as: • Major Functionalities (black-box) • Modules in the system (white-box) • All Error Messages or All UI Screens (white-box) • The test execution may be terminated & analyzed via: • the coverage - the % of the checklist test cases executed • the coverage - the % of checklist items executed successfully

  3. Checklist Example • An application “functional” checklist (perhaps derived from Requirements doc.) of an on-line-order software may include the following: • Entering the order items • Validating the order items • Checking availability/price of order items • Summarizing the order items and the total price into an invoice • Display and validate the invoice You may “check-off” the items as you tested each of the above functionalities.

  4. More on Checklist • A checklist may be further expanded to form a “tree” of checklists to provide some depth. • Several checklists may be combined to form a “matrix” of checklists. Some logical intersection of checklists may be developed into a “new” checklist. • Some drawbacks of checklist: • Too high level to cover some of the details • One high level item may require extensive analysis (complex scenario and thus require several related test cases to test a scenario.) • There may exist “overlaps” at high level • Complex interactions may also exist besides overlapping • Too difficult to combine multiple views such as black-box and white-box testing even with combined checklists. Can we improve on checklist ? ----

  5. Partition • We care about partition because we want to divide our test cases up so that they (a)uniquely(with non or minimal duplication) cover(b) all the situations of interest. • The formal mathematical definition of a partition is: • Consider a set S • The set S is partitioned into subsets C1, C2, ----, Cn if: • For any x, y, x≠y then Cx ∩ Cy = Ø (there is no overlap between Ci’s) • ∑ Ci = S (union of all Ci’s is S)

  6. Partition of set S of marbles “by size” C1 C5 C3 C2 C4 • Note that: 1) For any x, y, x≠y then Cx ∩ Cy = Ø (there is no overlap between Ci’s) • 2) ∑ Ci = S (union of all Ci’s is S)

  7. Some More “Math” on Partition • Each partition, Cx, of set S is considered an equivalence class. (this is because every member inside Cx is the “same” with respect to the property, size.) A relation “= size” used to partition the set S. • Another example: the set S may be a set of dogs, and we partitioned S by the color of the dogs {black, white, brown, buff, gray, multi-color}. Then every dog in the partition “black” is the “same” with respect to that color. • The relation for members in an equivalence class satisfy the following rules. Let R represent the “equivalence” relation then: • R(a,b) => R(b,a) [symmetric] {e.g.} Samecolor(a,b) => Samecolor(b,a) • R(a,b) Λ R(b,c) => R(a,c) [transitive] {e.g.} { Samecolor(a.b) Λa Samecolor (b,c) } => Samecolor(a,c) • R(a,a) [reflexive] {e.g.} Samecolor (a,a)

  8. Example of Testing Program Q • Consider a program Q that i) reads inputs a, b, c and ii) determines the roots, r1 and r2, for the quadratic equation ax2 +bx +c = 0. • From high school math, we remember the 2 roots for this equation is given by: r1 = [ -b + √(b2 -4ac) ]/2a and r2 = [ -b - √(b2 -4ac) ]/2a • To test this program , we may consider all kinds of approaches • How should we decide on the test cases for our testing of this program?

  9. 1st Example of Test cases for the Program Q • Consider the inputs to the program Q: a, b, c • We could divide the three inputs into valid (numerical) and invalid (non-numerical) to see if program Q handles input validation properly • Then we would have 2 cases for each input variable or 2 x 2 x 2 = 8 test cases to cover all the possible “valid-invalid” input combinations: • (a = valid, b= valid, c= valid) • (a = valid, b= valid , c= invalid) • (a = valid, b= invalid, c= valid) • (a= valid, b= invalid, c= invalid) • (a= invalid, b= valid, c= valid) • (a= invalid, b= valid, c= invalid) • (a= invalid, b= invalid, c= valid) • (a= invalid, b= invalid, c= invalid) • Note: that this is a checklist and is also a partitioning of the input based on our coming up with the relation of “valid and invalid’ inputs

  10. Partitioning Input, (a,b,c), into 8 Equivalence Categories • (a = valid, b= valid, • c= valid) • (a = invalid, b= valid, • c= valid) • (a = invalid, b= valid, • c= invalid) • (a = valid, b= valid, • c= invalid) • (a = invalid, b= invalid, • c= valid) • (a = valid, b= invalid, • c= valid) • (a = valid, b= invalid, • c= invalid) • (a = invalid, b= invalid, • c= invalid) • Note that within each equivalence category there may be infinite choices! • Note that Choosing 1 from each category covers a) all 8 categories and • b) each category is unique ---- partition

  11. 2nd Example of Test cases for the Program Q • Again, consider the inputs to the program Q: a, b, c • We consider the situation of =0, >0 and <0 values for each input to test the implementation of algorithm for root • Then we will have 3 x 3 x 3 = 27 test cases to cover all the possible combinations: (a=0, b=0, c =0) (a>0, b=0, c =0) (a<0, b=0, c =0) (a=0, b=0, c >0) (a>0, b=0, c >0) (a<0, b=0, c >0) (a=0, b=0, c <0) (a>0, b=0, c <0) (a<0, b=0, c <0) (a=0, b>0, c =0) (a>0, b>0, c =0) (a<0, b>0, c =0) (a=0, b>0, c >0) (a>0, b>0, c >0) (a<0, b>0, c >0) (a=0, b>0, c <0) (a>0, b>0, c <0) (a<0, b>0, c <0) (a=0, b<0, c =0) (a>0, b<0, c =0) (a<0, b<0, c =0) (a=0, b<0, c >0) (a>0, b<0, c >0) (a<0, b<0, c >0) (a=0, b<0, c <0) (a>0, b<0, c <0) (a<0, b<0, c <0) • Note that this is also a checklist and is also a partitioning of the input based on our coming up with the relation of =0, >0, and <0 values for the 3 inputs

  12. 3rd Example of Test cases for the Program Q • This time consider the outputs to the program Q: r1, r2 • We test the two major categories of possible outputs: a) r1 = r2 and b) r1 ≠ r2 • If (b2 -4ac) = 0, then r1 = r2 because r1 = (-b + 0)/2a and r2 = (-b – 0)/2a • If (b2 -4ac) >0, then r1 ≠ r2 • We have partitioned our “outputs” into 2 partitions : r1=r2 and r1 ≠ r2 ; this in turn partitioned our inputs of a,b,c into: • set of (a,b,c)’s such that b2 = 4ac • set of (a,b,c)’s such that b2 > 4ac 1)Note that this is a checklist and also a partition of 2 sets of input (a,b,c)’s based on covering the 2 partitions of outputs where r1 = r2 and r1 ≠ r2. 2)Note that your text includes the case of output being imaginary number.

  13. Partitions • Test Case partitioning technique may be applied over different types of properties of interest: • Inputs • Outputs • Functionalities • Usages • Logical paths • etc. • We can combine the different partitioning (Examples 1, 2, and 3) to cover different aspect of Program Q: input validation, algorithm correctness, output coverage)

  14. Usage Based Testing • Usage Based testing may be viewed as a type of partitioning. • We formulate a set of user-operational profiles (partitioning the users by some characteristics: e.g. user role & access of different functionalities by role) • We may attach probabilities to the user profile (e.g. different probabilities of access to different functionalities based on the role.) • We may formulate a resource usage profile (partitioning the resources by some characteristics: e.g. number of times accessed by resources) • We may attach frequency of usage count to each resource (e.g. develop a statistical usage frequency distribution of the resources) The idea is to test more on the more heavily “used” categories because more heavily used category is more “important”

  15. User-Operational Profile Example(naïve user versus experienced user) (20/20) (20/20) prob = .25 √ term naïve users (20/80) default f1 f4 (5/5) prob = .063 (5/10) f4 term Start (80 users) (5/10) (5/5) prob = .063 (10/60) f1 f6 term experienced users (60/80) (40/40) (40/40) (40/60) prob = .50√√ term f7 f2 Choice of f1,f2,f3 (10/60) (3/10) (3/3) prob = .037 term f4 f3 (7/10) (7/7) prob = .087 Note that 2 usage cases take up 75% of operational profile ! f5 term

  16. Some Interesting Observations/Usageof the User Operational Profile • From a testing priority and resource allocation perspective, we may conduct our testing in a sequence equal to the highest probability scenario (choice of f1,f2,f3 - f2 – f7 ) to the lowest probability scenario (choice of f1,f2,f3 – f3 – f4). • If we want to claim some form of “coverage,” we may use the user-profile and test all the scenarios thoroughly except (choice of f1,f2,f3 – f3 – f4) and claim 96% coverage reliability.

  17. Resource-Usage Profile Example(DB-table Usage Frequency distribution) Usage/Month 9k times employeetable is “used” the most; thus more test cases should be created to for accessing that table 6k times 3ktimes employee payroll benefits dependent tax tax DB - Tables

More Related