Software Engineering: Software Quality Assurance. Romi Satria Wahon o email@example.com http://romisatriawahono.net +6281586220090. Romi Satria Wahono. SD Sompok Semarang (1987) SMPN 8 Semarang (1990) SMA Taruna Nusantara , Magelang (1993)
Romi Satria Wahonoromi@romisatriawahono.nethttp://romisatriawahono.net+6281586220090
Product Development Process
Mortensonvs. Timeberline Software (≈1993)
Mortenson used a TS application when creating a bid to build a hospital
The software created a bid that was $2M too low
TS knew about the bug, but had not sent an update to Mortenson
The State of Washington Supreme Court ruled in favor of TS
Article 2 of the Uniform Commercial Code
Uniform Computer Information Transaction Act (UCITA)allows software manufacturers to: (≈1999)
disclaim all liability for defects
prevent the transfer of software from person to person
remotely disable licensed software during a dispute
does not apply to embedded systems
DISCLAIMER OF WARRANTIES. TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, MICROSOFT AND ITS SUPPLIERS PROVIDE TO YOU THE SOFTWARE COMPONENT, AND ANY (IF ANY) SUPPORT SERVICES RELATED TO THE SOFTWARE COMPONENT ("SUPPORT SERVICES") AS IS AND WITH ALL FAULTS; AND MICROSOFT AND ITS SUPPLIERS HEREBY DISCLAIM WITH RESPECT TO THE SOFTWARE COMPONENT AND SUPPORT SERVICES ALL WARRANTIES AND CONDITIONS, WHETHER EXPRESS, IMPLIED OR STATUTORY, INCLUDING, BUT NOT LIMITED TO, ANY (IF ANY) WARRANTIES OR CONDITIONS OF OR RELATED TO: TITLE, NON-INFRINGEMENT, MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, LACK OF VIRUSES, ACCURACY OR COMPLETENESS OF RESPONSES, RESULTS, LACK OF NEGLIGENCE OR LACK OF WORKMANLIKE EFFORT, QUIET ENJOYMENT, QUIET POSSESSION, AND CORRESPONDENCE TO DESCRIPTION. THE ENTIRE RISK ARISING OUT OF USE OR PERFORMANCE OF THE SOFTWARE COMPONENT AND ANY SUPPORT SERVICES REMAINS WITH YOU.
Term coined by DoD years ago
Problem Today: complexity of problems addressed by software has outpaced improvements in software creation process
"We have repeatedly reported on cost rising by millions of dollars, schedule delays, of not months but years, and multi-billion-dollar systems that don't perform as envisioned.
The understanding of software as a product and of software development as a process is not keeping pace with the growing complexity and software dependence of existing and emerging mission-critical systems."
Government Accounting Office
"Few fields have so large a gap between best current practice and average current practice."
Department of Defense
Software errors are sections of the code that are partially or totally incorrect as a result of a grammatical, logical or other mistake made by a systems analyst, a programmer, or another member of the software development team
Software faults are software errors that cause the incorrect functioning of the software during a specific application
Software faults become software failures only when they are “activated”, that is, when a user tries to apply the specific software section that is faulty. Thus, the root of any software failure is a software error
"Software bugs, or errors, are so prevalent and so detrimental that they cost the U.S. economy an estimated $59.5 billion annually, or about 0.6 percent of the gross domestic product. …
Although all errors cannot be removed, more than a third of these costs, or an estimated $22.2 billion, could be eliminated by an improved testing infrastructure that enables earlier and more effective identification and removal of software defects. These are the savings associated with finding an increased percentage (but not 100 percent) of errors closer to the development stages in which they are introduced. Currently, over half of all errors are not found until "downstream" in the development process or during post-sale software use.“
(US Dept of Commerce, June 2002)
Patriot Missile System
ESA's Ariane 5 Launch System
When operating in soft X-ray mode, the machine was designed to rotate three components into the path of the electron beam, in order to shape and moderate the power of the beam. …
The accidents occurred when the high-energy electron-beam was activated without the target having been rotated into place; the machine's software did not detect that this had occurred, and did not therefore determine that the patient was receiving a potentially lethal dose of radiation, or prevent this from occurring.
The design did not have any hardware interlocks to prevent the electron-beam from operating in its high-energy mode without the target in place
The hardware provided no way for the software to verify that sensors were working correctly
The equipment control task did not properly synchronize with the operator interface task, so that race conditions occurred if the operator changed the setup too quickly. This was evidently missed during testing, since it took some practice before operators were able to work quickly enough for the problem to occur
The software set a flag variable by incrementing it. Occasionally an arithmetic overflow occurred, causing the software to bypass safety checks
On February 25, 1991, the Patriot missile battery at Dharan, Saudi Arabia had been in operation for 100 hours, by which time the system's internal clock had drifted by one third of a second. For a target moving as fast as an inbound TBM, this was equivalent to a position error of 600 meters
The radar system had successfully detected the Scud and predicted where to look for it next, but because of the time error, looked in the wrong part of the sky and found no missile. With no missile, the initial detection was assumed to be a spurious track and the missile was removed from the system. No interception was attempted, and the missile impacted on a barracks killing 28 soldiers
June 4, 1996 was the first test flight of the Ariane 5 launch system. The rocket tore itself apart 37 seconds after launch, making the fault one of the most expensive computer bugs in history.
The Ariane 5 software reused the specifications from the Ariane 4, but the Ariane 5's flight path was considerably different and beyond the range for which the reused code had been designed. Specifically, the Ariane 5's greater acceleration caused the back-up and primary inertial guidance computers to crash, after which the launcher's nozzles were directed by spurious data. Pre-flight tests had never been performed on the re-alignment code under simulated Ariane 5 flight conditions, so the error was not discovered before launch.
Because of the different flight path, a data conversion from a 64-bit floating point to 16-bit signed integer caused a hardware exception (more specifically, an arithmetic overflow, as the floating point number had a value too large to be represented by a 16-bit signed integer). Efficiency considerations had led to the disabling of the exception handler for this error. This led to a cascade of problems, culminating in destruction of the entire flight.
To know that quality has improved, it would be helpful to be able to measure quality.
How can we measure quality?
- Software game harusinteraktifdanresponsif
- Software phone switching harushandal
- Software ecommerce danperbankanharusaman (secure)
Fa = w1c1 + w2c2 + … + wncn
F= Factor, W= Weight, C=Criteria
*Source: MengukurKualitasPerangkatLunak, RomiSatriaWahono.Net
Capability Maturity Model (CMM), Software Engineering Institute
ISO 9000 seeks to set criteria which achieve a goal and is not prescriptive as to methods. The requirements come in Sections 4 to 8.
In each of these areas, ISO 9001: 2000 seeks to set out key requirements, which if met will ensure quality.
2. Normative references
3. Definitions, abbreviations, and conventions
4. V&V software integrity levels
5. V&V processes
5.1 Process: Management
5.2 Process: Acquisition
5.3 Process: Supply
5.4 Process: Development
5.4.2 Activity: Requirements V&V
5.4.3 Activity: Design V&V
5.4.4 Activity: Implementation V&V
5.4.5 Activity: Test V&V
5.4.6 Activity: Installation and Checkout V&V
5.5 Process: Operation
5.6 Process: Maintenance
6. Software V&V reporting, administrative, and documentation requirements
Annex A Mapping of ISO/IEC 12207 V&V requirements to IEEE Std 1012 V&V activities and tasks
So develop test plan early
Example, test completeness of CRC cards
Has as specific objective
Has specific test cases to examine
Uses test specifications
If the tested class requires methods that aren't ready
Use stubs (hard coded fake methods)Test Planning
3. Interaction testing
4. System interface testing