Software Failure: Reasons. Incorrect, missing, impossible requirements * Requirement validation. Incorrect specification * Specification verification. Faulty system design * Design reviews. Faulty software code * Unit testing. Faulty hardware infrastructure. Human Factor.
Software Failure: Reasons
Incorrect, missing, impossible requirements* Requirement validation.
Incorrect specification* Specification verification.
Faulty system design* Design reviews.
Faulty software code* Unit testing.
Faulty hardware infrastructure
Developers, analysts and users make ‘simple’ mistakes. These mistakes can be uncovered and corrected with the help review meetings, verification, validation and testing process.
However, mistakes made due to miscommunication or misunderstanding are much harder to notice.
Incorrect process steps
Rounding errors, small data types
Incorrect interface/API docs leads to program errors
Load / Capacity
Timing / Synchronization
Fault Distribution (HP)
Test Process Organization
Unit testing: verifies that the component functions property (in a controlled environment) with the expected input.
Integration testing: ensures that the system components work together as they are supposed.
Function testing: determines if the functions described in the requirement are performed. Successful function test yields validated system.
Acceptance testing: performed by customers, system is checked against the requirements.
Deployment testing: the software is tested again once it is deployed in a new (clean) environment.
Views of the Test Objects
Black Box: internal structure is unknown
- Difficult to test comprehensively
White Box/Clear Box: internal structure is known
- Possible to devise specific & boundary tests
Code ReviewIndependent / external reviewers examine and critique the code
Code WalkthroughYou walk the reviewers through your code
Code Inspectionchecks the code against predetermined list of concerns
+ Can uncover up to 90% of faults!
! It is best to use well prepared reviewers that are very well familiar with the product.
- Does not uncover requirement or design faults.
Formal Proof Techniques
Write assertions that describes code’s input and output conditions. These should be logical statements.
Draw a flow chart diagram of the code.
Using the flow chart and assertions devise a series of theorems that trace logical transformations performed by the code for various paths through the flow chart.
Prove the theorems.
+ Develops understanding of the code
+ Ensures algorithm correctness
- Difficult to set up and carry out the proofs
- Impractical / impossible for complex algorithms
Choosing Test Cases
Ideally we want to test every possible input. When the inputs are limitless or impractical to enumerate we need to separate data into the equivalence classes (when the object is white box). Then it would be sufficient to test data drawn from the equivalence class.
Equivalence classes need to be picked out such that they cover all branches and switches in the code in order to maximize the code coverage.
If the code under investigation maintains state we must devise a sequence of tests such that all states of the finite state machine are tried.
We also need to test boundary conditions as well as invalid input.
Test Thoroughness Approaches
Statement testing: every statement has to be tested.
Branch testing: every brunch must be process in both directions.
Path testing: every distinct path through the code must be tested.
Definition-use testing: every path from every definition (variable) to every use of that definition must be tested.
Unit Testing Comparison
Chapter 8 from the white book.