1 / 22

Software Testing Using Model Program

Software Testing Using Model Program. DESIGN BY HONG NGUYEN & SHAH RAZA Dec 05, 2005. Overview Software Testing Using Model Program. Briefly introduction of M-mp testing What is M-mp testing? How it work? Why we need program base testing The problem of software testing Oracle test

agnes
Download Presentation

Software Testing Using Model Program

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Software Testing Using Model Program DESIGN BY HONG NGUYEN & SHAH RAZA Dec 05, 2005

  2. OverviewSoftware Testing Using Model Program • Briefly introduction of M-mp testing • What is M-mp testing? • How it work? • Why we need program base testing • The problem of software testing • Oracle test • N-Version programming • What is N-Version Programming? • Disadvantages • Advantages

  3. Overview (continues) • M-mp testing • Framework for M-mp testing • Asymptotic behavior of M-mp testing • Input domain partitioning • An M-mp Experiment • Background and motivation • Selecting the application • Phases of the experiment • Conclusion

  4. Introduction • What is M-mp testing? • Testing using M Model program • Abbreviated to ‘M-mp testing’ • It is alternative to software testing base on manual outcome prediction • M-mp testing is base on N-Version programming technique • Which is apply specially in the well-known software fault tolerant technique.

  5. Introduction (Continues) • How it work • Model program implements selected part of functional specification of the software to be tested. • The M-mp testing strategy requires that M (M>= 1) • Model programs as well as the program under test, should be independently developed • P and M model program are then subjected to the same test data. • Difference analysis is conducted on the outputs and appropriate corrective action taken • P and the M model program jointly constituted an approximate test oracle (out come correct or not)

  6. Introduction (Continues) • Why we need program base testing • For small number of test data, it may be feasible to work out the expected output manually • For thorough testing which required large test datasets manual testing could be time consuming and error prone • That is why M-mp base testing may be more feasible for those kinds of data set

  7. The problem of software testing • The difficult task in software testing is to assess the correctness to the outcome of a program that is subjected to particular test input • In testing theory, the mechanism to adjudicate on whether or not an output is correct, somewhat neglected this problem is referred to as “the oracle problem. • Testing theory concern with the choice of test and testing methods, usually ignore the oracle problems.

  8. The problem of software testing (continues) • Oracle Test • It means of determining whether a program passed or failed a test.

  9. The problem of software testing (continues) • Oracle test (continues) • P is the program to be tested • Test strategy determines the set of test data to be used • To get the test result we compare P’s output with the oracle ‘s output • For small out put oracle manually base oracle technique is feasible • But for large industry base application manually base oracle testing is not feasible • One of alternative of manually base oracle is N- Version programming

  10. N-Version Programming • N-Version is comprises N independently written version of the software • All has the same functional specification • At runtime voting base on majority agreement is used to decided the probable output • Disadvantage: • Concern in N-Version systems is the fact that correlated failures may limit the practical gain in reliability • N-Version systems should not be casually assumed to be completed reliable.

  11. N-Version (continues) • Disadvantage (continues) • Is the increased development cost because at least three version being required for a voting system to work • Advantage • N-Version design appear to offer more reliability that we can gain from any other way • Even though the cost of failure is high: • It would be more cost effective to build N-Version systems rather than focusing on building ‘one good version’

  12. M-mp Testing • Framework for M-mp testing

  13. M-mp Testing (continues) • Framework (continues) • In M-mp testing P is primary program (the program under test) • mp1 –mpM are M so called ‘model program of P • M-mp testing treat P and mp1-mpM to be same test input • Any disagreement on the output indicates the presence of specification defects Or software faults in at least one of program • Once defects are detected and removed. The program are re-run

  14. M-mp Testing (continues) • Framework (continues) • The cycle is repeat until all disagreements for a particular dataset are resolved. • Asymptotic behavior of M-mp testing • Difference between the M-mp and manual approaches to testing is the cost of outcome verification

  15. M-mp Testing (continues) • Asymptotic behavior of M-mp testing (continues) • On the graph: the cost of M-mp outcome verification is high initially but it increase only slightly. • The cost growth is modeled as linear • In manual approach, the growth of the output verification cost may be expected to be much steeper. • Input domain partitioning • Divided into a standard domain and an exception domain • Exception domain may be split into an incomplete data domain and an invalid data domain. • The standard domain consists of inputs that a program can process ‘as is’

  16. M-mp Testing (continues) • Input domain partitioning (continues) • The invalid domain contains the input that a program can not process it should be rejected with appropriate error message • The incomplete data domain is ‘between’ the standard and invalid data domains • It is made up of inputs that the program has to completed with default values before processing them • An M-mp Experiment • Background and motivation

  17. M-mp Testing (continues) • Background and motivation (continues) • The primary program is a scheduling application, a non-interactive data processing subsystem of a large logistics management system • The program can be characterized as algorithmically complex • In practice, the testing process in the company was well defined and supported by standards and procedures but had several drawback • Designing test cases based on manual outcome prediction was not only difficult, error prone and time consuming • But consequently also unattractive and demoralizing

  18. M-mp Testing (continues) • Selecting the application • The scheduling application was chosen as the primary program for following reasons • Developer was not familiar with the scheduling domain and application • The cost of developing from scratch could be assesses more authentically • The risk of correlated failures caused by common design error was reduces • scheduling application is representative of the family of operation management program • The time the experiment started, the scheduling subsystem had been in use for more than year • This seem a good opportunity to test the fault detection ability of M-mp testing, as the likelihood of finding defect was expected to be low

  19. M-mp Testing (continues) • Phase of the experiment

  20. M-mp Testing (continues) • Phase of the experiment • The experiment entailed a test design, a test execution and test evaluation phase as depicted • Test design phase comprised test data selection • Which includes activities related to test data generation, Model program design and model program implantation • The test execution phase includes activities relating to setting up the test environment and executing the program • The test evaluation phase includes the disagreement analysis procedures whereby disagreements between the program need to be arbitrated.

  21. Conclusion • M-mp approach could test a scheduling program more adequately than manually designed test and at lower cost • M-mp testing should be judged on a case by case basic • M-mp approach will tend to be better option than the manual one when the number of tests required to achieve a particular adequacy level become large

  22. Question • Finished • Please be easy on us.Thank you

More Related