chapter 3 quality assurance
Download
Skip this Video
Download Presentation
Chapter 3 Quality Assurance

Loading in 2 Seconds...

play fullscreen
1 / 24

Chapter 3 Quality Assurance - PowerPoint PPT Presentation


  • 111 Views
  • Uploaded on

Chapter 3 Quality Assurance. SE 471 - Software Testing and Quality Assurance. 1. Overview. Classification: QA as Dealing with Defects Defect Prevention Education and training Formal method Other defect prevention techniques Defect Reduction

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Chapter 3 Quality Assurance' - ocean


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
chapter 3 quality assurance

Chapter 3Quality Assurance

SE 471 - Software Testing and Quality Assurance

1

overview
Overview
  • Classification: QA as Dealing with Defects
  • Defect Prevention
    • Education and training
    • Formal method
    • Other defect prevention techniques
  • Defect Reduction
    • Inspection: Direct fault detection and removal
    • Testing: Failure observation and fault removal
    • Other techniques and risk identification
  • Defect Containment
    • Software fault tolerance
    • Safety assurance and failure containment

2

quality assurance from the correctness centered viewpoint ccv
Quality Assurance from the Correctness Centered Viewpoint (CCV)
  • According to CCV, Quality means:
    • few, if any, defects remain in the software system when delivered
    • the remaining defects cause minimal damage
  • With the correctness centered quality definition given above, we can view different QA activities as attempting to
    • prevent,
    • reduce, or
    • contain the damage
  • Briefly discussed here - detailed descriptions are presented in later chapters

3

dealing with pre post release defects
Dealing with Pre-/Post-release Defects
  • Focus is to remove as many defects as possible before product release - why?
    • Cost
    • Reputation
  • Pre-release QA activities
    • Defect prevention, detection, removal
    • Several approaches including development methodologies, programming techniques, defect removing tools, testing, beta tests, etc

4

dealing with pre post release defects1
Dealing with Pre-/Post-release Defects
  • Dormant defects: Faults left in the finished software
    • May stay dormant for sometime
    • Potential to cause problem
  • After release QA activities
    • Aim at minimizing the negative impact of the remaining faults during operational use after product release
    • Fixing defects reported by customers
    • Failure containment:
      • Most defect containment techniques involve many redundancies
      • Require significantly more development effort to design/implement
      • typically limited to the situations where failures are associated with substantial damage

5

graphical depiction of the classification scheme
Graphical Depiction of the Classification Scheme
  • QA activities act as Barriers - dotted lines
  • Each barrier
    • removes or blocks defect sources
    • prevents undesirable consequences

Defect Prevention Activities

Defect Removal Activities

Fault Tolerance Activities

6

classification of qa approaches
Classification of QA Approaches
  • We can classify these QA alternatives into the following three generic categories:
    • Defect prevention
    • Defect reduction
    • Defect containment

7

defect prevention
Defect Prevention
  • Defect prevention: can be done in two ways:
    • Eliminating certain error sources. Examples:
      • Eliminating ambiguities - SRS
      • Correcting human misconception – lack of knowledge
    • Fault prevention or blocking missing human actions through the use of certain tools or technologies. Examples:
      • Missing to put ‘{‘ or ‘(‘ can be avoided by using “Syntax-Directed Editors”. Automatically balances out parenthesis

8

defect prevention1
Defect Prevention
  • There are known error sources or missing/ incorrect actions that result in certain fault injections
  • So? What is needed?
    • Root cause analysis
    • Find root causes for injected or potential faults
  • Error source & Remedy
    • If human misconceptions are the error sources → education and training
    • If imprecise designs and implementations that deviate from product specifications → formal methods
    • If non-conformance to selected processes or standards is the problem → process conformance/standard enforcement
    • Certain tools or technologies may reduce fault injections

9

defect prevention2
Defect Prevention
  • Education & Training
    • Product and domain specific knowledge
      • developers unfamiliar with embedded software may design software without considering its environmental constraints - interface problems
    • General software development knowledge and expertise
      • lack of expertise with requirement analysis - rework in subsequent design, coding, and testing activities
    • Knowledge about the specific development methodologies and tools used by the organization
      • In Cleanroom technology if the developers are not familiar with the key components of formal verification or statistical testing - little chance for producing high-quality products
    • Knowledge about the software process used by the organization
      • incremental software development - uncoordinated development may lead to many interface or interaction problems

10

defect prevention3
Defect Prevention
  • Formal Method
    • Formal specification and formal verification
      • Formal specification is concerned with producing an unambiguous set of product specification
      • Formal verification checks the conformance of software design or code against these formal specifications
    • Examples
      • Axiomatic approach
      • Planguage, Z notation, VDM
    • Most influential formal method is the axiomatic approach
      • Meaning of program statements is abstracted as a set of axioms
      • Rules are used to compose axioms to build up proofs of entire program correctness
      • Program behavior is explained as a set of pre-conditions and post-conditions
    • Obstacle to formal methods: high cost (automated support needed), difficult task (human intensive activities)

11

defect prevention4
Defect Prevention
  • Process conformance/standard enforcement may reduce fault injections. Examples:
    • A better managed process can also eliminate many systematic problems e.g. no process for configuration management may lead to inconsistencies
    • Lean Software – avoid low quality ‘fat’ software
  • Certain Tools & Technologies may reduce fault injections. Examples:
    • Tools, e.g. Syntax directed editor automatically balances out each open parenthesis

12

defect reduction
Defect Reduction
  • Defect prevention activities to be 100% effective – an unrealistic expectation
  • Need effective techniques to remove as many of the injected faults as possible
  • Defect reduction: detect and remove faults once they have been injected.
  • Done in two ways:
    • Static Analysis: e.g. Inspection of software code, design etc
    • Dynamic Analysis: e.g. Testing of programs by executing

13

defect reduction1
Defect Reduction
  • Inspection
    • Critical reading and analysis of software code or other software artifacts, such as designs, product specifications, test plans, user manuals etc
    • Typically conducted by multiple human inspectors
    • Identified faults need to be removed & removal verified
    • Used throughout the development process
    • An economical QA alternative
    • Two ways to do it: Informal & Formal
    • Informal reviews:
      • self conducted reviews/inspections
    • Formal inspections:
      • Fagan inspection - most influential work in inspection
      • Gilb inspection
      • Inspection processes vary, typically include some planning & follow up activities

14

defect reduction2
Defect Reduction
  • Testing
    • most important parts of QA
    • the most commonly performed QA activity
    • Involves execution & observation
    • When to start
      • Logically testing can only take place after implementation
      • However, preparation can start earlier
      • In the incremental or iterative process, testing can usually get started much earlier
      • can be divided into various sub-phases starting from the coding phase up to post-release product support, including: unit testing, component testing, integration testing, system testing, acceptance testing, beta testing, etc

15

defect reduction3
Defect Reduction
  • Testing
    • What to test
      • Black-box (or functional) testing verifies the correct handling of the external functions
      • White-box (or structural) testing verifies the correct implementation of internal units, structures, and relations among them
      • White-box test examines source code with focus on:
        • Control flow
        • Data flow

16

defect reduction4
Defect Reduction
  • Other static & dynamic techniques for defect reduction
    • Static: formal model based analysis techniques
      • Algorithm analysis, boundary value analysis, finite state machine, control and data flow analysis, software fault trees etc.
    • Dynamic:
      • Simulation and prototyping
    • Timing and performance analysis for real-time systems
    • Accident analysis and reconstruction using software fault trees and event trees for safety critical systems
    • Some Risk identification techniques

Will be covered later in detail

17

defect containment
Defect Containment
  • Defect reduction activities can only reduce the number of faults to a fairly low level, but not completely eliminate them
  • Still a problem for software systems where failure impact is substantial such as many real-time control software sub-systems used in medical, nuclear, transportation
  • Few remaining (dormant) faults may be triggered under rare conditions or unusual dynamic scenarios

18

defect containment1
Defect Containment
  • Unrealistic to expect planning of huge number of test cases
  • Some other means are needed to:
    • Either contain the damage to local areas so that there is no global failures observable to users
    • Or limit the damage caused by the software system failure

19

defect containment2
Defect Containment
  • Done in two ways:
    • Using Fault Tolerance Techniques:
      • Motivation
        • Traditional hardware: spare parts and backup units
      • Fault Tolerance Techniques - Examples
        • Recovery blocks: rollback and redo
        • NVP: N-version programming (inspired by hw FT)
    • Containment Measures:
      • Avoid catastrophic consequences e.g. death etc
      • Examples:
        • Failure containment for real time control software used in nuclear reactors may include concrete walls to encircle and contain radioactive materials in case of reactor melt-down due to software failure

20

defect containment3
Defect Containment
  • Fault Tolerance (FT) VS Exception Handling (EH):
    • EH deviates from the specification
    • FT attempts to provide services compliant with the specification after detecting a fault
    • Important to realize the difference between trying to construct robust software versus trying to construct reliable software
    • Reliable software by FT or EH?
  • High profile disaster due to uncaught exception:
    • On June 4, 1996, the maiden flight of the European Ariane 5 launcher crashed about 40 seconds after takeoff.
    • The loss was $500,000,000 uninsured
    • An uncaught exception caused entire software to crash

21

defect containment4
Defect Containment
  • Safety Assurance and Fault Containment
    • Prevent accidents from happening
      • accident: failure with severe consequences
    • In addition to the QA techniques discussed before, various specific techniques are also used for safety critical systems based on analysis of hazards
      • Hazard elimination:
        • eliminate hazard sources
      • Hazard reduction:
        • reduce the probability of hazard
      • Hazard control:
        • reduce the severity of failures
      • Damage control:
        • reduce the severity of damage

22

self study assignment included in exam
Self-Study Assignment – Included in Exam
  • Read article “Quantifying Quality Requirements Using Planguage”, understand it and practice it for simple examples. Its easy & interesting!
    • Planguage is derived from Planning & Language.
    • Designed to quantify qualitative statements in plans, specifications, and designs, Planguage is a keyword-driven language that allows measurable, testable quality requirements to be written.
    • Planguage has many benefits; it is easy to learn, flexible, compact, extensible, and prevents omissions by providing a consistent set of parameters for quality requirements.
    • In this article, Planguage keywords and syntax are introduced. Examples of quality requirements before and after using Planguage are given, and the experiences of introducing Planguage are discussed.
  • You may be given a question in exam/quiz!

23

homework no 2
Homework No 2
  • Read article “Design by Contract: The Lessons of Ariane” from: http://archive.eiffel.com/doc/manuals/technology/contract/ariane/
  • Then read another interesting article which is in fact the critique of the above one titled “Critique of ‘Put it in the contract: The lessons of Ariane’” from: http://home.flash.net/~kennieg/ariane.html

24

ad