Week 8 - Quality Management Learning Objectives

1. Week 8 - Quality ManagementLearning Objectives You should be able to: List and explain common principles of quality management (QM) List, distinguish between, and describe the processes and tools of Quality Planning, Assurance, and Control Apply QM principles to Project Management Apply QM principles to software development project management Demonstrate how the CMM incorporates QM principles

2. Quality: is everyone�s job, comes from prevention not inspection, means meeting the needs of customers, demands teamwork, requires continuous improvement, involves strategic planning, means results, requires clear measures of success.

3. History of QM/QC/QA Deming: plan, do, check, act Juran: improvement, planning, control Crosby: zero defects, management commitment Ishikawa quality circles, root cause of problems Taguchi: prevention vs. inspection Feigenbaum: worker responsibility

4. Quality Management Organization-wide commitment: culture Results and measurement focus Tools and technical support needed Training and learning Continuous improvement of each process Is it necessary? Can it be done better?

5. 7 Malcolm Baldrige Award Categories Pre-production leadership information and analysis strategic quality planning Production human resource allocation quality assurance

6. ISO 9000 Standard:5 Elements (500 points) Quality Planning Performance Information Cost of Quality (economics) Continuous Improvement Customer Satisfaction

7. Quality in Project Management I ISO 9000, TQM, CQI principles Prevention over inspection lower cost, higher productivity, more cust. satisfaction Management responsibility and team participation Plan-do-check-act (re: Deming, etc.) - PDCA Applied successfully in environments that have well-defined processes and products More difficult in areas like software development

8. Quality in Project Management II Customer satisfaction validation: �the right job done� Conformance to specifications verification: �the job done right� Fitness for use can be used as intended Satisfaction of implied or stated needs All project stakeholders considered Project Management: making implicit needs explicit Project Processes and Product continuous improvement of both

10. Quality Planning (QP) Identifying relevant quality standards Determining how to meet them QP inputs: quality policy: adopted, disseminated scope and product description standards, regulations

11. Software Quality Planning Functionality features: required and optional Outputs Performance volume of data, number of users response time, growth rate Reliability: MTBF (mean time between failures) Maintainability

12. QP Outputs Quality management plan how team will implement quality policy structure, responsibilities, resources, processes (same as project plan?) Operational definitions metrics: what it is and how it�s measured Checklists industry-specific

13. Quality Assurance (QA) Evaluating project performance regularly to assure progress towards meeting standards Inputs: quality management plan operational definitions results of measurements Outputs: quality improvement actions Tools: QP tools, quality audits

14. QP/QA Tools Cost / benefit analysis and tradeoffs less rework = higher productivity, lower costs, stakeholder satisfaction Design of Experiments comparison of options, approaches Benchmarking comparison of project practices to best practices Cause and effect (fishbone, etc., diagrams)

15. Quality Control (QC) Monitoring project results Measuring compliance with standards Determining causes if not in compliance Identifying ways to eliminate causes Performed throughout project life cycle

16. QC Inputs and Outputs Inputs: Work results Quality Management Plan Operational Definitions Checklists Outputs: Quality Improvement Acceptance decisions Rework Process adjustments corrective or preventive actions Completed checklists project records

17. QC Tools Inspection: measuring, examining, testing products Control Charts: monitor output variables detect instability in process graphical display of results Pareto analysis 80 / 20 rule histogram: frequencies Statistical sampling acceptable deviation 6-sigma 7-run rule

18. Statistical Quality Control Prevention keeping errors out of the process Inspection keeping defects from the customer Sampling: attributes and variables Tolerances: acceptable ranges Control limits: acceptable levels

19. Testing (Software) During most phases of product development Unit tests Integration testing System testing User acceptance testing

20. Improving Software Quality Leadership top management and organization-wide commitment to quality Costs of quality cost of non-conformance costs: prevention, appraisal, failures, testing Work environment

21. PMI Maturity Model: 5 levels Ad-hoc: chaotic, chronic cost & schedule delays Abbreviated: processes in place, but not predictable Organized: documented, standards that are used Managed: measures are collected Adaptive: feedback enables continuous improvement project success is norm

22. Capability Maturity Model (CMM) - 5 levels 1. Initial: chaotic, heroic efforts, unpredictable 2. Repeatable: processes & standards established 3. Defined: documented standards, training, use 4. Managed: quantitative measures, predictable 5. Optimizing: defect-prevention, organization-wide continuous improvement

23. CMM and Quality(see Appendix A: goals for key process areas) Level 2: requirements management (customer focus) project planning (quality planning) project tracking and oversight (quality control) software quality assurance configuration management (prevention)

24. CMM Level 3 and Quality organization process focus (commitment) organization process definition (operational definitions) training program software product engineering (prevention) intergroup organization (teamwork) peer reviews (teamwork)

25. CMM Level 4 and Quality Quantitative process improvement Software quality management goals planned and measured

26. Achieving Software Quality Focus on critical requirements early Use metrics early and continuously Provide development tools supporting: configuration control, change control test automation, self-documentation abstraction, reliability, reuse Early and continuous demonstration-based evaluations Major milestone demonstrations assessed against critical requirements

27. Software Quality Measurement Software quality measured by ease of change Examples of data collected: Number and types of changes number of components / effort (FPs, SLOC, classes...) number of change orders (SCOs) number of defective and fixed components Baseline: total size (SLOC, FP, classes, etc.) Scrap: broken code, may or may not be fixed Rework: healthy early in project, should decrease

28. SCO: Software Change Order 1. rework a poor quality component (fix) 2. rework to improve quality (enhancement) 3. accommodate new customer requirement (scope change) Configured Baseline: the set of products subject to change control size of �completed� components

29. Software Quality Metrics Modularity: breakage localization: extent of change re: baseline size Adaptability cost of change (effort needed to resolve and retest) Maturity number of SCO�s over time = MTBF during testing Each of above 3 should decrease over time Maintainability productivity of rework / productivity of development

30. Operational Definitions Defects: measured by change orders SCOs Open rework (breakage) broken components measured by SCOs Closed rework (fixes) fixed SCOs Rework effort: effort expended fixing SCOs Usage time: baseline testing in normal use

31. Quantifying Quality Metrics Modularity: breakage / SCOs Adaptability rework effort / SCOs Maturity usage time / SCOs (mean time between defects) Maintainability (percent broken) / (percent rework vs. total effort) End-product and �over time� indicators

32. Peer inspections: pros and cons Pros Team development Accountability Determine causes of defects 20%: critical components Cons Superficial Not cost effective Other QA activities are more effective

34. Quality of IT Projects Many people joke about the poor quality of IT products (cars and computers joke) People seem to accept systems being down occasionally or needing to reboot their PCs There are many examples in the news about quality-related problems

35. What Went Wrong? In one of the biggest software errors in banking history, Chemical Bank mistakenly deducted about $15 million from more than 100,000 customer accounts one evening. The problem resulted from a single line of code in an updated computer program that caused the bank to process every withdrawal and transfer at its automated teller machines (ATMs) twice. For example, a person who withdrew $100 from an ATM had $200 deducted from his or her account, though the receipt only indicated a withdrawal of $100. The mistake affected 150,000 transactions from Tuesday night through Wednesday afternoon. In 1996 Apple Computer's PowerBook 5300 model had problems with lithium-ion battery packs catching fire, causing Apple to halt shipments and replace all the packs with nickel-metal-hydride batteries. Other quality problems also surfaced, such as cracks in the PowerBook's plastic casing and a faulty electric power adapter. Hundreds of newspapers and web sites ran stories about the "Melissa" virus in March of 1999. The rapidly spreading computer virus forced several large corporations to shut down their e-mail servers as it rode the Internet on a global rampage, according to several leading network security companies.

36. What Is Project Quality Management? The International Organization for Standardization (ISO) defines quality as the totality of characteristics of an entity that bear on its ability to satisfy stated or implied needs Other experts define quality based on conformance to requirements: meeting written specifications fitness for use: ensuring a product can be used as it was intended

37. Project Quality Management Processes Quality planning: identifying which quality standards are relevant to the project and how to satisfy them Quality assurance: evaluating overall project performance to ensure the project will satisfy the relevant quality standards Quality control: monitoring specific project results to ensure that they comply with the relevant quality standards while identifying ways to improve overall quality

38. Modern Quality Management Modern quality management requires customer satisfaction prefers prevention to inspection recognizes management responsibility for quality Noteworthy quality experts include Deming, Juran, Crosby, Ishikawa, Taguchi, and Feigenbaum

39. Quality Experts Deming was famous for his work in rebuilding Japan and his 14 points Juran wrote the Quality Control Handbook and 10 steps to quality improvement Crosby wrote Quality is Free and suggested that organizations strive for zero defects Ishikawa developed the concept of quality circles and using fishbone diagrams Taguchi developed methods for optimizing the process of engineering experimentation Feigenbaum developed the concept of total quality control

40. Sample Fishbone or Ishikawa Diagram

41. Malcolm Baldrige Award and ISO 9000 The Malcolm Baldrige Quality Award was started in 1987 to recognize companies with world-class quality ISO 9000 provides minimum requirements for an organization to meet their quality certification standards

42. Quality Planning It is important to design in quality and communicate important factors that directly contribute to meeting the customer�s requirements Design of experiments helps identify which variables have the most influence on the overall outcome of a process Many scope aspects of IT projects affect quality like functionality, features, system outputs, performance, reliability, and maintainability

43. Quality Assurance Quality assurance includes all the activities related to satisfying the relevant quality standards for a project Another goal of quality assurance is continuous quality improvement Benchmarking can be used to generate ideas for quality improvements Quality audits help identify lessons learned that can improve performance on current or future projects

44. Quality Control The main outputs of quality control are acceptance decisions rework process adjustments Some tools and techniques include pareto analysis statistical sampling quality control charts testing

45. Pareto Analysis Pareto analysis involves identifying the vital few contributors that account for the most quality problems in a system Also called the 80-20 rule, meaning that 80% of problems are often due to 20% of the causes Pareto diagrams are histograms that help identify and prioritize problem areas

46. Sample Pareto Diagram

47. Statistical Sampling and Standard Deviation Statistical sampling involves choosing part of a population of interest for inspection The size of a sample depends on how representative you want the sample to be Sample size formula: Sample size = .25 X (certainty Factor/acceptable error)

48. Commonly Used Certainty Factors

49. Standard Deviation Standard deviation measures how much variation exists in a distribution of data A small standard deviation means that data cluster closely around the middle of a distribution and there is little variability among the data A normal distribution is a bell-shaped curve that is symmetrical about the mean or average value of a population

50. Normal Distribution and Standard Deviation

51. Sigma and Defective Units

52. QC Charts, 6 Sigma, and the Seven Run Rule A control chart is a graphic display of data that illustrates the results of a process over time. It helps prevent defects and allows you to determine whether a process is in control or out of control Operating at a higher sigma value, like 6 sigma, means the product tolerance or control limits have less variability The seven run rule states that if seven data points in a row are all below the mean, above the mean, or increasing or decreasing, then the process needs to be examined for non-random problems

53. Sample Quality Control Chart

54. Reducing Defects with Six Sigma

55. Testing Many IT professionals think of testing as a stage that comes near the end of IT product development Testing should be done during almost every phase of the IT product development life cycle

56. Testing Tasks in the SDLC

57. Types of Tests A unit test is done to test each individual component (often a program) to ensure it is as defect free as possible Integration testing occurs between unit and system testing to test functionally grouped components System testing tests the entire system as one entity User acceptance testing is an independent test performed by the end user prior to accepting the delivered system

58. Building Testing into a Project Plan

59. Improving IT Project Quality Several suggestions for improving quality for IT projects include Leadership that promotes quality Understanding the cost of quality Focusing on organizational influences and workplace factors that affect quality Following maturity models to improve quality

60. Leadership �It is most important that top management be quality-minded. In the absence of sincere manifestation of interest at the top, little will happen below.� (Juran, 1945) A large percentage of quality problems are associated with management, not technical issues

61. The Cost of Quality The cost of quality is the cost of conformance or delivering products that meet requirements and fitness for use the cost of nonconformance or taking responsibility for failures or not meeting quality expectations

62. Costs of Downtime Caused by S/W Defects

63. Five Cost Categories Related to Quality Prevention cost: the cost of planning and executing a project so it is error-free or within an acceptable error range Appraisal cost: the cost of evaluating processes and their outputs to ensure quality Internal failure cost: cost incurred to correct an identified defect before the customer receives the product External failure cost: cost that relates to all errors not detected and corrected before delivery to the customer Measurement and test equipment costs: capital cost of equipment used to perform prevention and appraisal activities

64. Organization Influences & Workplace Factors Study by DeMarco and Lister showed that organizational issues had a much greater influence on programmer productivity than the technical environment or programming languages Programmer productivity varied by a factor of one to ten across organizations, but only by 21% within the same organization Study found no correlation between productivity and programming language, years of experience, or salary A dedicated workspace and a quiet work environment were key factors to improving programmer productivity

65. Maturity Models Maturity models are frameworks for helping organizations improve their processes and systems Software Quality Function Deployment Model focuses on defining user requirements and planning software projects The Software Engineering Institute�s Capability Maturity Model provides a generic path to process improvement for software development Several groups are working on project management maturity models

66. Project Management Maturity (PMM) Model 1. Ad-Hoc: The project management process is described as disorganized, and occasionally even chaotic. The organization has not defined systems and processes, and project success depends on individual effort. There are chronic cost and schedule problems. 2. Abbreviated: There are some project management processes and systems in place to track cost, schedule, and scope. Project success is largely unpredictable and cost and schedule problems are common. 3. Organized: There are standardized, documented project management processes and systems that are integrated into the rest of the organization. Project success is more predictable, and cost and schedule performance is improved.

67. Project Management Maturity Model (Cont.) 4. Managed: Management collects and uses detailed measures of the effectiveness of project management. Project success is more uniform, and cost and schedule performance conforms to plan. 5. Adaptive: Feedback from the project management process and from piloting innovative ideas and technologies enables continuous improvement. Project success is the norm, and cost and schedule performance is continuously improving.