Chapter 3 quality assurance
1 / 24

Chapter 3 Quality Assurance - PowerPoint PPT Presentation

  • Uploaded on

Chapter 3 Quality Assurance. SE 471 - Software Testing and Quality Assurance. 1. Overview. Classification: QA as Dealing with Defects Defect Prevention Education and training Formal method Other defect prevention techniques Defect Reduction

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Chapter 3 Quality Assurance' - ocean

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Chapter 3 quality assurance

Chapter 3Quality Assurance

SE 471 - Software Testing and Quality Assurance



  • Classification: QA as Dealing with Defects

  • Defect Prevention

    • Education and training

    • Formal method

    • Other defect prevention techniques

  • Defect Reduction

    • Inspection: Direct fault detection and removal

    • Testing: Failure observation and fault removal

    • Other techniques and risk identification

  • Defect Containment

    • Software fault tolerance

    • Safety assurance and failure containment


Quality assurance from the correctness centered viewpoint ccv
Quality Assurance from the Correctness Centered Viewpoint (CCV)

  • According to CCV, Quality means:

    • few, if any, defects remain in the software system when delivered

    • the remaining defects cause minimal damage

  • With the correctness centered quality definition given above, we can view different QA activities as attempting to

    • prevent,

    • reduce, or

    • contain the damage

  • Briefly discussed here - detailed descriptions are presented in later chapters


Dealing with pre post release defects
Dealing with Pre-/Post-release Defects (CCV)

  • Focus is to remove as many defects as possible before product release - why?

    • Cost

    • Reputation

  • Pre-release QA activities

    • Defect prevention, detection, removal

    • Several approaches including development methodologies, programming techniques, defect removing tools, testing, beta tests, etc


Dealing with pre post release defects1
Dealing with Pre-/Post-release Defects (CCV)

  • Dormant defects: Faults left in the finished software

    • May stay dormant for sometime

    • Potential to cause problem

  • After release QA activities

    • Aim at minimizing the negative impact of the remaining faults during operational use after product release

    • Fixing defects reported by customers

    • Failure containment:

      • Most defect containment techniques involve many redundancies

      • Require significantly more development effort to design/implement

      • typically limited to the situations where failures are associated with substantial damage


Graphical depiction of the classification scheme
Graphical Depiction of the Classification Scheme (CCV)

  • QA activities act as Barriers - dotted lines

  • Each barrier

    • removes or blocks defect sources

    • prevents undesirable consequences

Defect Prevention Activities

Defect Removal Activities

Fault Tolerance Activities


Classification of qa approaches
Classification of QA Approaches (CCV)

  • We can classify these QA alternatives into the following three generic categories:

    • Defect prevention

    • Defect reduction

    • Defect containment


Defect prevention
Defect Prevention (CCV)

  • Defect prevention: can be done in two ways:

    • Eliminating certain error sources. Examples:

      • Eliminating ambiguities - SRS

      • Correcting human misconception – lack of knowledge

    • Fault prevention or blocking missing human actions through the use of certain tools or technologies. Examples:

      • Missing to put ‘{‘ or ‘(‘ can be avoided by using “Syntax-Directed Editors”. Automatically balances out parenthesis


Defect prevention1
Defect Prevention (CCV)

  • There are known error sources or missing/ incorrect actions that result in certain fault injections

  • So? What is needed?

    • Root cause analysis

    • Find root causes for injected or potential faults

  • Error source & Remedy

    • If human misconceptions are the error sources → education and training

    • If imprecise designs and implementations that deviate from product specifications → formal methods

    • If non-conformance to selected processes or standards is the problem → process conformance/standard enforcement

    • Certain tools or technologies may reduce fault injections


Defect prevention2
Defect Prevention (CCV)

  • Education & Training

    • Product and domain specific knowledge

      • developers unfamiliar with embedded software may design software without considering its environmental constraints - interface problems

    • General software development knowledge and expertise

      • lack of expertise with requirement analysis - rework in subsequent design, coding, and testing activities

    • Knowledge about the specific development methodologies and tools used by the organization

      • In Cleanroom technology if the developers are not familiar with the key components of formal verification or statistical testing - little chance for producing high-quality products

    • Knowledge about the software process used by the organization

      • incremental software development - uncoordinated development may lead to many interface or interaction problems


Defect prevention3
Defect Prevention (CCV)

  • Formal Method

    • Formal specification and formal verification

      • Formal specification is concerned with producing an unambiguous set of product specification

      • Formal verification checks the conformance of software design or code against these formal specifications

    • Examples

      • Axiomatic approach

      • Planguage, Z notation, VDM

    • Most influential formal method is the axiomatic approach

      • Meaning of program statements is abstracted as a set of axioms

      • Rules are used to compose axioms to build up proofs of entire program correctness

      • Program behavior is explained as a set of pre-conditions and post-conditions

    • Obstacle to formal methods: high cost (automated support needed), difficult task (human intensive activities)


Defect prevention4
Defect Prevention (CCV)

  • Process conformance/standard enforcement may reduce fault injections. Examples:

    • A better managed process can also eliminate many systematic problems e.g. no process for configuration management may lead to inconsistencies

    • Lean Software – avoid low quality ‘fat’ software

  • Certain Tools & Technologies may reduce fault injections. Examples:

    • Tools, e.g. Syntax directed editor automatically balances out each open parenthesis


Defect reduction
Defect Reduction (CCV)

  • Defect prevention activities to be 100% effective – an unrealistic expectation

  • Need effective techniques to remove as many of the injected faults as possible

  • Defect reduction: detect and remove faults once they have been injected.

  • Done in two ways:

    • Static Analysis: e.g. Inspection of software code, design etc

    • Dynamic Analysis: e.g. Testing of programs by executing


Defect reduction1
Defect Reduction (CCV)

  • Inspection

    • Critical reading and analysis of software code or other software artifacts, such as designs, product specifications, test plans, user manuals etc

    • Typically conducted by multiple human inspectors

    • Identified faults need to be removed & removal verified

    • Used throughout the development process

    • An economical QA alternative

    • Two ways to do it: Informal & Formal

    • Informal reviews:

      • self conducted reviews/inspections

    • Formal inspections:

      • Fagan inspection - most influential work in inspection

      • Gilb inspection

      • Inspection processes vary, typically include some planning & follow up activities


Defect reduction2
Defect Reduction (CCV)

  • Testing

    • most important parts of QA

    • the most commonly performed QA activity

    • Involves execution & observation

    • When to start

      • Logically testing can only take place after implementation

      • However, preparation can start earlier

      • In the incremental or iterative process, testing can usually get started much earlier

      • can be divided into various sub-phases starting from the coding phase up to post-release product support, including: unit testing, component testing, integration testing, system testing, acceptance testing, beta testing, etc


Defect reduction3
Defect Reduction (CCV)

  • Testing

    • What to test

      • Black-box (or functional) testing verifies the correct handling of the external functions

      • White-box (or structural) testing verifies the correct implementation of internal units, structures, and relations among them

      • White-box test examines source code with focus on:

        • Control flow

        • Data flow


Defect reduction4
Defect Reduction (CCV)

  • Other static & dynamic techniques for defect reduction

    • Static: formal model based analysis techniques

      • Algorithm analysis, boundary value analysis, finite state machine, control and data flow analysis, software fault trees etc.

    • Dynamic:

      • Simulation and prototyping

    • Timing and performance analysis for real-time systems

    • Accident analysis and reconstruction using software fault trees and event trees for safety critical systems

    • Some Risk identification techniques

      Will be covered later in detail


Defect containment
Defect Containment (CCV)

  • Defect reduction activities can only reduce the number of faults to a fairly low level, but not completely eliminate them

  • Still a problem for software systems where failure impact is substantial such as many real-time control software sub-systems used in medical, nuclear, transportation

  • Few remaining (dormant) faults may be triggered under rare conditions or unusual dynamic scenarios


Defect containment1
Defect Containment (CCV)

  • Unrealistic to expect planning of huge number of test cases

  • Some other means are needed to:

    • Either contain the damage to local areas so that there is no global failures observable to users

    • Or limit the damage caused by the software system failure


Defect containment2
Defect Containment (CCV)

  • Done in two ways:

    • Using Fault Tolerance Techniques:

      • Motivation

        • Traditional hardware: spare parts and backup units

      • Fault Tolerance Techniques - Examples

        • Recovery blocks: rollback and redo

        • NVP: N-version programming (inspired by hw FT)

    • Containment Measures:

      • Avoid catastrophic consequences e.g. death etc

      • Examples:

        • Failure containment for real time control software used in nuclear reactors may include concrete walls to encircle and contain radioactive materials in case of reactor melt-down due to software failure


Defect containment3
Defect Containment (CCV)

  • Fault Tolerance (FT) VS Exception Handling (EH):

    • EH deviates from the specification

    • FT attempts to provide services compliant with the specification after detecting a fault

    • Important to realize the difference between trying to construct robust software versus trying to construct reliable software

    • Reliable software by FT or EH?

  • High profile disaster due to uncaught exception:

    • On June 4, 1996, the maiden flight of the European Ariane 5 launcher crashed about 40 seconds after takeoff.

    • The loss was $500,000,000 uninsured

    • An uncaught exception caused entire software to crash


Defect containment4
Defect Containment (CCV)

  • Safety Assurance and Fault Containment

    • Prevent accidents from happening

      • accident: failure with severe consequences

    • In addition to the QA techniques discussed before, various specific techniques are also used for safety critical systems based on analysis of hazards

      • Hazard elimination:

        • eliminate hazard sources

      • Hazard reduction:

        • reduce the probability of hazard

      • Hazard control:

        • reduce the severity of failures

      • Damage control:

        • reduce the severity of damage


Self study assignment included in exam
Self-Study Assignment – (CCV)Included in Exam

  • Read article “Quantifying Quality Requirements Using Planguage”, understand it and practice it for simple examples. Its easy & interesting!

    • Planguage is derived from Planning & Language.

    • Designed to quantify qualitative statements in plans, specifications, and designs, Planguage is a keyword-driven language that allows measurable, testable quality requirements to be written.

    • Planguage has many benefits; it is easy to learn, flexible, compact, extensible, and prevents omissions by providing a consistent set of parameters for quality requirements.

    • In this article, Planguage keywords and syntax are introduced. Examples of quality requirements before and after using Planguage are given, and the experiences of introducing Planguage are discussed.

  • You may be given a question in exam/quiz!


Homework no 2
Homework No 2 (CCV)

  • Read article “Design by Contract: The Lessons of Ariane” from:

  • Then read another interesting article which is in fact the critique of the above one titled “Critique of ‘Put it in the contract: The lessons of Ariane’” from: