1 / 16

Software Testing

Software Testing. Test Early, Test Often CS673 Fazle Khan Nov 1, 2011. What Is Software Testing?. Book: A validation process, the purpose of which is to detect as many defects of as high a level of seriousness as possible.

demont
Download Presentation

Software Testing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Software Testing Test Early, Test Often CS673 Fazle Khan Nov 1, 2011

  2. What Is Software Testing? • Book: A validation process, the purpose of which is to detect as many defects of as high a level of seriousness as possible. • Wikipedia:An investigation conducted to provide stakeholders with information about the quality of the product or service under test

  3. Planning • What type of testing is needed for your application? • When are the tests a going to be executed • Where are the tests going to be executed • How to structure an application to perform each of the selected test types? Here is where testing directly impacts software design • Metrics • Test Success/Failure Count . A zero failure policy is the best choice • Code Coverage • Defects Per Hour

  4. Scope • Important to define upfront how broad testing effort will be. • Defined from the Requirements Document • Web applications should define their supported browsers with versions • Multi-platform application need to have their platform specified • Failure to properly scope the testing effort will result in a mix of a missed deadlines, buggy code, and cost overruns

  5. Who Tests What?An example

  6. Black Box Testing • Wikipedia • A method of software testing that tests the functionality of an application as opposed to its internal structures or workings • Specific knowledge of the application's code/internal structure and programming knowledge in general is not required. • Test cases are built around specifications and requirements, i.e., what the application is supposed to do. • It uses external descriptions of the software, including specifications, requirements, and designs to derive test cases. • These tests can be functional or non-functional, though usually functional. • The test designer selects valid and invalid inputs and determines the correct output.

  7. White Box Testing • Wikipedia • A method of testing software that tests internal structures or workings of an application, as opposed to its functionality • In white-box testing an internal perspective of the system, as well as programming skills, are required and used to design test cases. • The tester chooses inputs to exercise paths through the code and determine the appropriate outputs.

  8. Unit Testing • Wikipedia: • a method by which individual units of source code are tested to determine if they are fit for use. A unit is the smallest testable part of an application • In procedural programming a unit may be an individual function or procedure. • In object-oriented programming a unit is usually an interface, such as a class (method) • Ideally, each test case is independent from the others: substitutes like method stubs, mock objects, fakes and test harnesses can be used to assist testing a module in isolation. • Java Tools and techniques • JUnit • EasyMock • Dependency Injection / Inversion of Control (IoC) design pattern • Critical for any non-compiled/typed languages to maintain API through iterations

  9. Sample JUnit codeException testing @Test  public void testMatchNullInstrumentTest() {MatcherThread x = new MatcherThread("hi", 1L);TransactionMakerHoldertmh = new TransactionMakerHolder();tmh.setTransactionMaker(new TransactionMaker() {           @Override           public Object transact(TransactionMakerWorkertransactionMakerWorker) {               return transactionMakerWorker.work();           }       });x.setTransactionMakerHolder(tmh);x.setInstrumentDAO(new InstrumentDAO() {           @Override           @Nullable           public Instrument exclusiveLockAndLoad(Long instrumentId) {               return null;           }       });        try {x.match();Assert.fail();       } catch (RuntimeException r) {Assert.assertNotNull(r);       }   }

  10. Module Testing • Exercise methods in combination • 2-5, typically • Test most common sequences first • Include sequences likely to cause defects • Focus test on each attribute • Initialize, then execute method sequences that affect it • Verify that class invariants are unchanged • Verify invariant true with initial values • Execute a method sequence • Verify invariant still true • Verify that objects transition among expected states • Plan the state/transition event • Set up the object in the initial state by setting variables • Execute event and check that transition occurred

  11. Sample EasyMock Code public void testAddAndChangeDocument() { Collaborator mock = EasyMock.createMock(Collaborator.class); ClassUnderTestclassUnderTest = new ClassUnderTest(); classUnderTest.addListener(mock); //Mock Dependency injected! mock.documentAdded("Document"); mock.documentChanged("Document"); EasyMock.expectLastCall().times(3); EasyMock.replay(mock); classUnderTest.addDocument("Document", new byte[0]); classUnderTest.addDocument("Document", new byte[0]); classUnderTest.addDocument("Document", new byte[0]); classUnderTest.addDocument("Document", new byte[0]); verify(mock); }

  12. Integration Testing • Big Bang Integration (so y2k) • Bring up all modules up at once for testing • Promotes systems with circular dependencies among modules • Only option for legacy systems with existing circular dependencies without a rewrite • Incremental Integration • Modules are layered on top of other modules, dependencies run in one direction • Bottom-Up Integration • lowest level module (for example DAO class) is written and tested first. • then modules dependant on the previous module are coded and tested with the previous module still in context • Top-Down Integration • Highest level module with the most valuable business logic is written first • The dependencies of the first module are stubbed out initially and are implemented later in the development cycle. • Usually done with GUIs • Continuous Integration • Small amounts of code are added to the baseline application on a very frequent basis • As long as code is unit-tested and does not introduce errors to the base line it is integrated into the base-line • Java Tools • Any continuous integration build server, for example Teamcityor CruiseControl

  13. System Testing • When possible performed on the required environment • Functional Tests focus on how the customer will use the product in real life • Acceptance Tests • Non-Functional Testing

  14. Non-Functional System Testing • Performance • The system’s through put • Load/Stress Testing • Under what load does the system start to break • Reliability and Availability • How long between failures • Recoverability • Can the system be restarted after an unexpected failure • Usability • Can users actually work with the system • Security • Can the system be hacked • Compatibility • Does the system work well with other systems • Installation • How easy is it to install the system • Serviceability • How easy is it to fix and/or enhance the system

  15. Regression Testing • Wikipedia • any type of software testing that seeks to uncover new errors, or regressions, in existing functionality after changes have been made to a system, such as functional enhancements, patches or configuration changes. • The intent of regression testing is to ensure that a change, such as a bugfix, did not introduce new faults. • One of the main reasons for regression testing is that it's often extremely difficult for a programmer to figure out how a change in one part of the software will echo in other parts of the software."

  16. Luddite vs. Machinedeath to the trained money • Manual testing is a relic of the past • Does not intrinsically provide a repeatable regression suite • Very expensive to repeat tests • Not reliable or reproducible in the long run • Automation makes all test suites regression tests allowing the code base to expand with minimal team effort. • Today all testing should be coded/programmed in repeatable tests suites that are automatically run by regression test server

More Related