410245(B): Software Testing and Quality Assurance (Elective-II)

1. Teaching Scheme: TH: 03 Hours/Week Examination Scheme: In-Sem (Paper): 30 Marks End-Sem (Paper): 70 Marks 410245(B):Software Testing and Quality Assurance(Elective-II) B.E. Computer Engineering Sem-I

2. Unit-VI: Software Quality Tools • TQM – Total Quality Management • Software Quality Metrics • Ishikawa’s 7 Basic Tools

3. Total Quality Management (TQM).. • TQM is a management approach to improve the quality. • It is a method by which management and employees can become involved in the continuous improvement of the production of goods and services. • TQM is a management philosophy that seeks to integrate all organizational functions (marketing, finance, design, engineering and production, customer services etc.) to focus on meeting customer needs and organizational objectives.

4. Total Quality Management (TQM).. • Six fundamentals steps of quality improvement paradigm are: • Characterize the project and its environment • Set the goals • Choose the appropriate processes • Execute the processes • Analyze the data • Package the experience for reuse

7. Software Quality Metrics.. • Software metrics can be classified into three categories: • Product Metrics :Describes the characteristics of the product such as size, complexity, design features, performance, and quality level. • Process Metrics:These characteristics can be used to improve the development and maintenance activities of the software. • Project Metrics :This metrics describe the project characteristics and execution. Examples include the number of software developers, the staffing pattern over the life cycle of the software, cost, schedule, and productivity.

8. Software Quality Metrics.. • Software quality metrics are a subset of software metrics that focus on the quality aspects of the product, process, and project. These are more closely associated with process and product metrics than with project metrics. • Software quality metrics can be further divided into three categories: • Product quality metrics • In-process quality metrics • Maintenance quality metrics

9. Product Quality Metrics.. • Software quality consists of two levels: • Intrinsic product quality • customer satisfaction • The metrics that covers both levels are: • Mean Time to Failure • Defect Density • Customer Problems • Customer Satisfaction

10. Product Quality Metrics.. • Mean Time to Failure: It is the time between failures. This metric is mostly used with safety critical systems such as the airline traffic control systems, and weapons. • Defect Density: It measures the defects relative to the software size expressed as lines of code or function point, etc. i.e., it measures code quality per unit. This metric is used in many commercial software systems.

11. Product Quality Metrics.. 3. Customer Problems: It measures the problems that customers encounter when using the product. It contains the customer’s perspective towards the problem space of the software, which includes the non-defect oriented problems together with the defect problems. The problems metric is usually expressed in terms of Problems per User-Month (PUM).

12. Product Quality Metrics.. PUM = Total Problems that customers reported (true defect and non-defect oriented problems) for a time period + Total number of license months of the software during the period Where, Number of license-month of the software = Number of install license of the software × Number of months in the calculation period • PUM is usually calculated for each month after the software is released to the market, and also for monthly averages by year.

13. Product Quality Metrics 4. Customer Satisfaction: • Customer satisfaction is often measured by customer survey data through the five-point scale, • Very satisfied • Satisfied • Neutral • Dissatisfied • Very dissatisfied • Satisfaction with the overall quality of the product and its specific dimensions is usually obtained through various methods of customer surveys.

14. In-process Quality Metrics.. • In-process quality metrics deals with the tracking of defect arrival during formal machine testing for some organizations. This metric includes − • Defect density during machine testing • Defect arrival pattern during machine testing • Phase-based defect removal pattern • Defect removal effectiveness

15. In-process Quality Metrics.. • Defect density during machine testing • Defect rate during formal machine testing (testing after code is integrated into the system library) is correlated with the defect rate in the field. Higher defect rates found during testing is an indicator that the software has experienced higher error injection during its development process, unless the higher testing defect rate is due to an extraordinary testing effort. • This simple metric of defects per KLOC or function point is a good indicator of quality, while the software is still being tested.

16. In-process Quality Metrics.. • Defect arrival pattern during machine testing • The defect arrivals or defects reported during the testing phase by time interval (e.g., week). Here all of which will not be valid defects. • The pattern of defect backlog overtime. This metric is needed because development organizations cannot investigate and fix all the reported problems immediately. This is a workload statement as well as a quality statement.

17. In-process Quality Metrics.. • Phase-based defect removal pattern • This is an extension of the defect density metric during testing. In addition to testing, it tracks the defects at all phases of the development cycle, including the design reviews, code inspections, and formal verifications before testing. • The pattern of phase-based defect removal reflects the overall defect removal ability of the development process.

18. In-process Quality Metrics • Defect removal effectiveness • It can be defined as follows − • This metric can be calculated for the entire development process, for the front-end before code integration and for each phase. It is called early defect removal when used for the front-end and phase effectiveness for specific phases.

19. Maintenance Quality Metrics.. • Although much cannot be done to alter the quality of the product during this phase, following are the fixes that can be carried out to eliminate the defects as soon as possible with excellent fix quality. • Fix backlog and backlog management index • Fix response time and fix responsiveness • Percent delinquent fixes • Fix quality

20. Maintenance Quality Metrics.. • Fix backlog and backlog management index • Fix backlog is related to the rate of defect arrivals and the rate at which fixes for reported problems become available. It is a simple count of reported problems that remain at the end of each month or each week. • Backlog Management Index (BMI) is used to manage the backlog of open and unresolved problems. • If BMI is larger than 100, it means the backlog is reduced. If BMI is less than 100, then the backlog increased.

21. Maintenance Quality Metrics.. • Fix response time and fix responsiveness • The fix response time metric is usually calculated as the mean time of all problems from open to close. Short fix response time leads to customer satisfaction. • The important elements of fix responsiveness are customer expectations, the agreed-to fix time, and the ability to meet one's commitment to the customer.

22. Maintenance Quality Metrics.. • Percent delinquent fixes • It is calculated as follows-

23. Maintenance Quality Metrics • Fix Quality • A defective fix can be recorded in two ways: Record it in the month it was discovered or record it in the month the fix was delivered. The first is a customer measure; the second is a process measure. The difference between the two dates is the latent period of the defective fix. • The quality goal for the maintenance process, of course, is zero defective fixes without delinquency.

24. Ishikawa’s 7 Basic Tools.. • Ishikawa's 7 Basic Tools of Quality is a designation given to a fixed set of graphical techniques identified as being most helpful troubleshooting issues related to quality. • They are called basic because they are used easily by people with little formal training in statistics.

25. Ishikawa’s 7 Basic Tools.. • The seven tools are: • Check sheet (Checklist) • Histogram • Pareto chart • Control chart • Scatter diagram • Stratification (alternately, flow chart or run chart) • Cause-and-effect diagram (also known as the "fishbone" or Ishikawa diagram) • This set of tools was first emphasized by Kaoru Ishikawa, a professor of engineering at Tokyo University and “father” of Quality Circles

26. 1. Check Sheet (Checklist).. • A check sheet can be introduced as the most basic tool for quality. • A check sheet is basically used for gathering and organizing data. • When this is done with the help of software packages such as Microsoft Excel, you can derive further analysis graphs and automate through macros available. • The check sheets are three major types, such as Defect-Location, Tally and Defect Cause check sheets. • One can always use a paper-based check sheet when the information gathered is only used for backup or storing purposes other than further processing.

28. 2. Histogram.. • Histogram is used for illustrating the frequency and the extent in the context of two variables. • Histogram is a chart with columns. This represents the distribution by mean. If the histogram is normal, the graph takes the shape of a bell curve. • If it is not normal, it may take different shapes based on the condition of the distribution. Histogram can be used to measure something against another thing. Always, it should be two variables. • Consider the example: The following histogram shows morning attendance of a class. The X-axis is the time of the day and the Y-axis the number of students.

30. 3. Pareto Charts.. • Definition: Pareto Diagram is a bar graph used to arrange information in such way that priorities for process improvement can be established. • The principal was developed by Vilfredo Pareto, an Italian economist and sociologist. • The fundamental idea behind the use of Pareto Diagram is Quality Improvement. • This helps you to work on the propriety issues in order to get the condition under control.

32. 4. Control Charts.. • Control chart is the best tool for monitoring the performance of a process. These types of charts can be used for monitoring any processes related to function of the organization. • These charts allow you to identify the following conditions related to the process that has been monitored. • Stability of the process • Predictability of the process • Identification of common cause of variation • Special conditions where the monitoring party needs to react

34. 5. Scatter Diagram.. • The scatter diagram helps to identify the existence of a measurable relationship between two items by measuring them in pairs and plotting them on a graph. • It visually shows the correlation between the two sets of measurement. • Two sets of data are plotted on a graph. With the Y-axis being used for the variable to be predicted (effect) and the X-axis being used for the variable to make the prediction (Cause).

37. 6. Flow Charts • This is one of the basic quality tool that can be used for analyzing a sequence of events. The tool maps out a sequence of events that take place sequentially or in parallel. The flow chart can be used to understand a complex process in order to find the relationships and dependencies between events. • You can also get a brief idea about the critical path of the process and the events involved in the critical path. • Flow charts can be used for any field to illustrate complex processes in a simple way. There are specific software tools developed for drawing flow charts, such as MS Visio.

39. 7. Cause and Effect Diagram.. • Cause and effect diagrams (Ishikawa Diagram) are used for understanding organizational or business problem causes. • Organizations face problems everyday and it is required to understand the causes of these problems in order to solve them effectively. Cause and effect diagrams exercise is usually a teamwork. • A brainstorming session is required in order to come up with an effective cause and effect diagram. • All the main components of a problem area are listed and possible causes from each area is listed.