Software quality assurance
This presentation is the property of its rightful owner.
Sponsored Links
1 / 41

Software Quality Assurance PowerPoint PPT Presentation


  • 131 Views
  • Uploaded on
  • Presentation posted in: General

Software Quality Assurance. Recap. What is testing? Who does testing? Why do we do testing? Software testing process? Software Testing Levels of testing Methods/techniques of testing Test cases Writing effective test cases. What is SQA?.

Download Presentation

Software Quality Assurance

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Software quality assurance

Software Quality Assurance


Recap

Recap

  • What is testing?

  • Who does testing?

  • Why do we do testing?

  • Software testing process?

  • Software Testing

    • Levels of testing

    • Methods/techniques of testing

  • Test cases

  • Writing effective test cases


What is sqa

What is SQA?

  • Software Quality Assurance is an umbrella activity that is applied throughout the software process...


What is quality

What is quality?

  • Quality refers to any measurable characteristics such as correctness, maintainability, portability, testability, usability, reliability, efficiency, integrity, reusability and interoperability.


Quality terminologies

Quality terminologies

  • Quality of Design refers to the characteristics that designer’s specify for an item.

  • Quality of Conformance is the degree to which the design specifications are followed during manufacturing.

  • Quality Control is the series of inspections, reviews and tests used throughout the development cycle to ensure that each work product meets the requirements placed upon it.

  • Quality policy refers to the basic aims and objectives of an organization regarding quality as stipulated by the management.

  • Quality assurance consists of the auditing and reporting functions of management.

  • Cost of Quality includes all costs incurred in the pursuit of quality or in performing quality related activities such as appraisal costs, failure costs and external failure costs.

  • Quality planning is the process of assessing the requirements of the procedure and of the product and the context in which these must be observed.

  • Quality testing is assessment of the extent to which a test object meets given requirements

  • Quality assurance plan is the central aid for planning and checking the quality assurance.

  • Quality assurance system is the organizational structure, responsibilities, procedures, processes and resources for implementing quality management.


Relative cost of correcting an error

Relative cost of correcting an error?


Elements of s w quality assurance

Elements of S/W Quality Assurance

  • Standards

  • Reviews and audits

  • Testing

  • Error/defect collection and analysis

  • Change management

  • Education

  • Vendor management

  • Security management

  • Safety

  • Risk management


Sqa tasks

SQA tasks

  • Prepares an SQA plan for a project

  • Participates in the development of the project’s software process description

  • Reviews software engineering activities to verify compliance with the defined software process

  • Audits designated software work products to verify compliance with those defined as part of the software process

  • Ensures that deviations in software work and work products are documented and handled according to a documented procedure

  • Records and noncompliance and reports to senior management


Sqa goals attributes and metrics

SQA Goals, Attributes and Metrics

  • Metric

  • Number of ambiguous modifiers (e.g., many, large, human-friendly)

  • Number of TBAs, TBDs

  • Number of sections/subsections

  • Number of changes per requirement

  • Time (by activity) when change is requested

  • Number of requirements not traceable to design/code

  • Number of UML models

  • Number of descriptive pages per model

  • Number of UML errors

  • Existence of architectural model

  • Number of components that trace to architectural model

  • Complexity of procedural design

  • Layout appropriateness

  • Number of patterns used

  • Attributes

  • Ambiguity

  • Completeness

  • Understandability

  • Volatility

  • Traceability

  • Model clarity

  • Architectural integrity

  • Component completeness

  • Interface complexity

  • Patterns

Goals

Requirement quality

Design quality


Sqa goals attributes and metrics1

SQA Goals, Attributes and Metrics

  • Metric

  • Cyclomatic complexity

  • Design factors

  • Percent internal comments

  • Variable naming conventions

  • Percent reused components

  • Readability index

  • Staff hour percentage per activity

  • Actual vs. budgeted completion time

  • Review metrics

  • Number of errors found and criticality

  • Effort required to correct an error

  • Origin of error

  • Attributes

  • Complexity

  • Maintainability

  • Understandability

  • Reusability

  • Documentation

  • Resource allocation

  • Completion rate

  • Review effectiveness

  • Testing effectiveness

Goals

Code quality

QC effectiveness


Sqa plan

SQA plan

  • Management section

    • describes the place of SQA in the structure of the organization

  • Documentation section

    • describes each work product produced as part of the software process

  • Standards, practices, and conventions section

    • lists all applicable standards/practices applied during the software process and any metrics to be collected as part of the software engineering work

  • Reviews and audits section

    • provides an overview of the approach used in the reviews and audits to be conducted during the project

  • Test section

    • references the test plan and procedure document and defines test record keeping requirements

  • Problem reporting and corrective action section

    • defines procedures for reporting, tracking, and resolving errors or defects, identifies organizational responsibilities for these activities

  • Other

    • tools, SQA methods, change control, record keeping, training, and risk management


Statistical sqa

Statistical SQA

  • Information about software defects is collected and categorized

  • An attempt is made to trace each defect to its underlying cause

  • Isolate the vital few causes of the major source of all errors

  • Then move to correct the problems that have caused the defects


Statistical sqa categories of errors

Statistical SQA – Categories of errors

  • Incomplete or erroneous specification (IES)

  • Misinterpretation of customer comm (MCC)

  • Intentional deviation from specification (IDS)

  • Violation of programming standards (VPS)

  • Error in data representation (EDR)

  • Inconsistent module interface (IMI)

  • Error in design logic (EDL)

  • Incomplete or erroneous testing (IET)

  • Inaccurate or incomplete documentation (IID)

  • Error in programming lang. Translation (PLT)

  • Ambiguous or inconsistent human-computer interface (HCI)

  • Miscellaneous (MIS)

  • Most often IES, MCC and EDR are the vital few causes for majority of errors.


Identifying the vital few

Identifying the vital few


Statistical sqa1

Statistical SQA


Example

Example


Statistical sqa six sigma

Statistical SQA – Six Sigma

  • Most widely used strategy for statistical SQA

  • Three core steps

    • Define customer requirements, deliverables and project goals via well-defined methods of customer communication

    • Measure the existing process and its output to determine quality

    • Analyze defect metrics and determine the vital few causes

  • If an existing software process is in place, but improvement is required six sigma suggests

    • Improve the process by eliminating the root causes of defects

    • Control the process to ensure that future work does not reintroduce the cases of defects

  • If an organization is developing a software process, the core steps are augmented

    • Design the process to (1) avoid the root causes of defects and (2) to meet customer requirements

    • Verify that the process model will, in fact, avoid defects and meet customer requirements


Reviews t o uncover errors defects

Reviews To uncover errors/defects

  • To uncover errors in function, logic, or implementation for any representation of the software

  • To verify that software meets its requirements

  • To ensure that software representation meets predefined standards

  • To achieve software development in a uniform manner

  • To make projects more manageable


Review roles

Review Roles

  • Presenter (designer/producer).

  • Coordinator (not person who hires/fires).

  • Recorder

    • records events of meeting

    • builds paper trail

  • Reviewers

    • maintenance oracle

    • standards bearer

    • user representative

    • others


Formal technical reviews

Formal Technical Reviews

  • Involves 3 to 5 people (including reviewers)

  • Advance preparation (no more than 2 hours per person) required

  • Duration of review meeting should be less than 2 hours

  • Focus of review is on a discrete work product

  • Review leader organizes the review meeting at the producer's request.

  • Reviewers ask questions that enable the producer to discover his or her own error (the product is under review not the producer)

  • Producer of the work product walks the reviewers through the product

  • Recorder writes down any significant issues raised during the review

  • Reviewers decide to accept or reject the work product and whether to require additional reviews of product or not.


Formality and timing

Formality and Timing

  • Formal review presentations

    • resemble conference presentations.

  • Informal presentations

    • less detailed, but equally correct.

  • Early

    • tend to be informal

    • may not have enough information

  • Later

    • tend to be more formal

    • Feedback may come too late to avoid rework


Formality and timing1

Formality and Timing

  • Analysis is complete.

  • Design is complete.

  • After first compilation.

  • After first test run.

  • After all test runs.

  • Any time you complete an activity that produce a complete work product.


Why do peer reviews

Why do peer reviews?

  • To improve quality.

  • Catches 80% of all errors if done properly.

  • Catches both coding errors and design errors.

  • Enforce the spirit of any organization standards.

  • Training and insurance.


Review guidelines

Review Guidelines..

  • Review the product, not producer

  • Set an agenda and maintain it

  • Limit the debate

  • Enunciate problem areas, not to solve every problem noted

  • Take written notes

  • Allocate resources and time schedule for FTR’s

  • Use standards to avoid style disagreements.

  • Let the coordinator run the meeting and maintain order.

  • Limit the number of participants and insist upon advance preparation

  • Develop a checklist for each work product to be reviewed

  • Training for all reviewer’s

  • Reviewing earlier reviews Keep it short (< 30 minutes).

  • Don’t schedule two in a row.

  • Don’t review product fragments.


Effectiveness of review defect amplification and removal

Development step

Defects

Detection

Errors passed through

Percent efficiency for error detection

Errors from previous steps

Errors passed to next step

Amplified errors 1:x

Newly identified errors

Effectiveness of review  Defect Amplification and Removal

  • Used to illustrate the generation and detection of errors during design and code generation


Effectiveness of review defect amplification and removal1

Effectiveness of review  Defect Amplification and Removal

No reviews

With reviews


Effectiveness of review defect amplification and removal2

Effectiveness of review  Defect Amplification and Removal


Example1

Example


Review metrics and their use

Review metrics and their use

  • Many metrics can be defined for technical reviews

  • The following can be calculated for each review conducted:

    • Preparation effort (Ep)

    • Assessment effort (Ea)

    • Rework effort (Er)

    • Work product size (WPS)

    • Minor errors found (Errminor)

    • Major errors found (Errmajor)


Analyzing review metrics

Analyzing review metrics

  • Total review effort (Ereview)

    • Ereview = Ep + Ea + Er

  • Total number of errors (Errtot)

    • Errtot = Errminor + Errmajor

  • Error density represents the errors found per unit of work product reviewed

    • Error density = Errtot / WPS

  • Cost effectiveness of reviews

    • Effort saved per error = Etesting – Ereviews


Effectiveness of review defect amplification and removal3

Effectiveness of review  Defect Amplification and Removal

No reviews

With reviews


Effectiveness of review defect amplification and removal4

Effectiveness of review  Defect Amplification and Removal


Software reliability

Software reliability

  • Defined as the probability of failure free operation of a computer program in a specified environment for a specified time.

  • Can be measured directly and estimated using historical and developmental data (unlike many other software quality factors)

  • Software reliability problems can usually be traced back to errors in design or implementation.

  • Reliability metrics are units of measure for system reliability

  • System reliability is measured by counting the number of operational failures and relating these to demands made on the system at the time of failure

  • A long-term measurement program is required to assess the reliability of critical systems


Measuring s w reliability

Measuring S/W reliability

  • A measure of software reliability is mean time between failures where

  • MTBF = MTTF + MTTR

  • MTTF = mean time to failure

  • MTTR = mean time to repair

  • Availability =MTTF/(MTTF + MTTR) * 100%

  • Software availability is the probability that a program is operating according to requirements at a given point in time


Example2

Example


Software reliability software safety

Software reliability -- Software safety

  • Processes that help reduce the probability that critical failures will occur due to SW

  • Hazard analyses

    • Identify hazards that could call failure

    • Develop fault tree

    • Identify all possible causes of the hazard

    • Formally review the remedy for each

  • Redundancy

  • Require a written software safety plan

  • Require independent verification & validation


Example fault tree thermal

Example Fault Tree -- Thermal

Loss of heat

...

Power failure

Computer failure

Incorrect

input

SW failed to throw switch

Computer failure

SW failed to throw switch

...

Logic reversed


Software safety

Software Safety

  • Redundancy

    • Replicated at the hardware level

    • Similar vs.. dis-similar redundancy

  • Verification

    • Assuring that the software specifications are met

  • Validation

    • Assuring that the product functions as desired

  • Independence


Iso 9000 quality standards

ISO 9000 Quality Standards

  • ISO 9000 describes QA elements in generic terms

    • Elements include organizational structure, procedures, processes and resources.

  • It treats an enterprise as a network of interconnected processes.

  • To be ISO-complaint processes should adhere to the standards described.

  • Ensures quality planning, quality control, quality assurance and quality improvement.

  • From S/W engineering view point: An international standard which provides broad guidance to software developers on how to Implement, maintain and improve a quality software system capable of ensuring high quality software

  • Consists of 20 requirements...

    • Differs from country to country..

23


Iso 9001 requirements

Management responsibility

Quality system

Contract review

Design Control

Document and data control

Purchasing

Control of customer supplied product

Product identification and traceability

Process control

Inspection and testing

Control of inspection, measuring and test equipment

Inspection and test status

Control of non-confirming product

Corrective and preventive action

Handling, storage, packaging, preservation and delivery

Control of quality records

Internal quality audits

Training

Servicing

Statistical techniques

ISO 9001 … requirements

25


Summary

Summary

  • SQA must be applied at each step

  • SQA might be complex

  • Software reviews are important SQA activities

  • Statistical SQA helps improve product quality and software process

  • Software Safety is essential for critical systems

  • ISO 9001 standardizes the SQA activities

27


  • Login