1 / 9

Different Testing Methodology

Different Testing Methodology. Inspection and Review (People Intensive) Mostly applicable --- document s and non-machine executable material requires preparation and some training in looking for problem, recording, follow-up, etc. Formal proof of correctness (Very Skill Intensive)

janine
Download Presentation

Different Testing Methodology

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Different Testing Methodology • Inspection and Review (People Intensive) • Mostly applicable --- documentsand non-machine executable material • requires preparation and some training in looking for problem, recording, follow-up, etc. • Formal proof of correctness (Very Skill Intensive) • Mostly applicable --- very small algorithmic solutions • requires formal training in theorem proving • Running code (Machine Intensive & People) • Applies to all executable material • requires a lot of preparation of test cases and some training in test process, test execution, and test tools

  2. Test Case Generation • How much do you test ? • From code point of view( what is the goal?) • 1 to 100% of executable statements • 1 to 100% of distinct paths ( #of predicates + 1) • 1 to 100% of program logic (loops can be a bear!) • From data point of view(what is the goal ?) • every input variable is assigned a valid value and executed once • every input variable is assigned a valid and an invalid value and executed once • every input variable is assigned a value outside of its boundary, on the boundary, and inside of its boundary and executed. (boundary value testing)

  3. Total # of Paths .vs. Total Executable Statement Coverage S1 1 Since for each binary decision, there are 2 paths and there are 3 in sequence, there are 23 =8 total “logical” paths path1 : S1-C1-S2-C2-C3-S4 path2 : S1-C1-S2-C2-C3-S5 path3 : S1-C1-S2-C2-S3-C3-S4 path4 : S1-C1-S2-C2-S3-C3-S5 path5 : S1-C1-C2-C3-S4 path6 : S1-C1-C2-C3-S5 path7 : S1-C1-C2-S3-C3-S4 path8 : S1-C1-C2-S3-C3-S5 All Statement Coverage: S1, C1, S2, C2, S3, C3, S4, S5 With path1 we covered S1, C1, S2,C2, C3, S4 (S3 & S5 uncovered) But with path5 we cover S1, C1, S2, C2, S3, C3, S5 C1 2 3 S2 4 C2 5 6 S3 7 C3 9 8 S5 S4

  4. When do we Stop Testing ? • Most common goal for stopping: • When we have i) executed all of our test cases and when ii) all thefailures which came from the testing are fixed. • Less common goal for stopping: • When we have i) executed a fixed percentage of test cases and when ii) all the failures which came from the testing are fixed • When we have i) executed all the test cases and when ii) only afixed percentage of the failures which came from the testing are fixed • Lesser common goal for stopping: • devising a stopping scheme with a goal and stop when the goal is reached.

  5. A Devised Test Stopping Scheme • Seeding: Faults into the product and NOT tell testers where • assumes that (detected seeded errors)/(total seeded errors) is equal to (detected real errors)/(total real errors) • look at a way to utilize this • dse/tse = dre/tre (from above) • tre (total real errors) = (tse * dre)/dse • if the goal is to have x% of the estimated total real errors found prior to stopping, we can look at the ratio of dre/tre. • If the x% is met or exceeded, then we can STOP • if not, we can continue to run more test cases and monitor the results: • dse increases with no increase in dre (good situation: because tre = (tse*dre)/dse ---- tse is fixed thus dse increase will mean lesser tre) • dre increases with no increase in dse (bad situation: dre increase drags an increase in tre because tre = (tse*dre) /dse ) • both dse and dre increases (depends on the numbers) • Make sure the all seeded faults are removed prior to release

  6. Look at An Example • tse = 20 • dse = 15 • dre = 50 • tre = (20 * 50) / 15 => approximately 67 total real errore estimated • dre/tre = 50/67 = appr. 75 % • if you want to have larger dre/tre ratio such as 80% (more faults are detected and fixed) then : • increase dre (this can mean that there are really more tre if there is no comparable increase in dse) • decrease tre via finding more seeded faults (increasing dse) without running into any real fault. • *** another mechanism is to introduce more total seeded faults(tse) and see if dse keeps on increasing without comparable increase in dre *** • *** this method is not fool proof because enough increase in dre will mean more testing is needed with more tse and the cycle continues.

  7. Test -Fix Cycle • Testing-Fix cycle involves both the testers and the fixers and therefore must be managed. • Problems found must be described and submitted for fix • Resource must be assigned to diagnose (accept or reject) the failures found • Resource must be assigned to fix the failures • Resources must be set aside to accept the fixes and merge them into the next test “build cycle.” • Resource and schedule must be set to include re-test of the fixes

  8. “Possible” Problem Report • Problem report should contain some of these items: • person who found the problem • date of problem submission (problem “open” date) • problem id # • component and version id of the software that was tested • test case(s) used • brief description of the problem symptoms with possibly screen snapshots • tester’s assessment of the problem severity • if a retest of another problem, give the earlier problem id # • Current Status: open; in fix; fixed; closed • closed date if closed

  9. Test- Fix Cycle Any Problem ? Test the Software Submit the problem Yes Software Test Library NO Problem Report DB The Testing Side The Fixing Side Accept Problem ? Send fixed material to test library Update Problem Status Fix Problem Yes NO

More Related