Software Process Improvement: CMMI and Lean Six Sigma

1. Software Process Improvement:CMMI and Lean Six Sigma USC CSC1 510 2 Nov 2016 Rick Hefner, Ph.D. California Institute of Technology rhefner@caltech.edu Software Process Improvement

2. Agenda • Current Challenges Facing the Software Industry • Capability Maturity Model Integrated • Lean Six Sigma • Corporate Support Software Process Improvement

3. What are some of the challenges facing software companies? Software Process Improvement

4. People-Related Mistakes 1. Undermined motivation 2. Weak personnel 3. Uncontrolled problem employees 4. Heroics 5. Adding people to a late project 6. Noisy, crowded offices 7. Friction between developers and customers 8. Unrealistic expectations 9. Lack of effective project sponsorship 10. Lack of stakeholder buy-in 11. Lack of user input 12. Politics placed over substance 13. Wishful thinking Process-Related Mistakes Overly optimistic schedules Insufficient risk management Contractor failure Insufficientplanning Abandonment of planning under pressure Wasted time during the fuzzy front end Shortchanged upstream activities Inadequate design Shortchanged quality assurance Insufficient management controls Premature or too frequent convergence Omitting necessary tasks from estimates Planning to catch up later Code-like-hell programming Product-Related Mistakes 28. Requirements gold-plating 29. Feature creep 30. Developer gold-plating 31. Push me, pull me negotiation 32. Research-orienteddevelopment Technology-Related Mistakes 33. Silver-bullet syndrome 34. Overestimated savings from new tools or methods 35. Switching tools in the middle of a project 36. Lack of automated source-code control Software Projects Have Historically Suffered from Mistakes Reference: Steve McConnell, Rapid Development • Standish Group survey of 13,000 projects (2003) • 34% successes • 15% failures • 51% overruns Software Process Improvement

5. Top Software Engineering Issues in Defense Industry • Development of software and systems must occur more rapidly than current acquisition models and development processes allow in order to meet the increasing demands of the warfighter. • Inadequate/insufficient program planning, start-up, and management of baseline changes has a negative impact on the program’s SW success. • Current software development and management practices have not kept pace with emerging needs in areas such as complex systems, Systems of Systems, system assurance, software verification, and COTS/NDI integration. • There is insufficient attention given to the overall software life cycle activities including sustainment and changing threats. “Top Software Engineering Issues Within Department of Defense and Defense Industry”, NDIA Systems Engineering Division and Software Committee, Sep 2010 Software Process Improvement

6. Many Approaches to Solving the Problem • Which weaknesses are causing my problems? • Which strengths may mitigate my problems? • Which improvement investments offer the best return? People Business Environment Management Structure Tools Product Methods Technology Process Software Process Improvement

7. Two Complimentary Approaches to Process Improvement Model-Driven (e.g., CMMI) • Determine the industry best practice • Benchmarking, models • Compare your current practices to the model • Appraisal, education • Identify and prioritize improvement opportunities • Implementation • Institutionalization • Look for ways to optimize the processes Data-Driven (e.g., Lean Six Sigma) • Clarify what your customer wants (Voice of Customer) • Critical to Quality (CTQs) • Determine what your processes can do (Voice of Process) • Statistical Process Control • Identify and prioritize improvement opportunities • Causal analysis of data • Anticipate your customers/ competitors (Voice of Business) • Design for Six Sigma Software Process Improvement

8. Capability Maturity Model Integrated(CMMI)

9. How would you ensure best practices are consistently implemented in your software company?

10. What is the Capability Maturity Model Integrated? The CMMI is a collection of industry best-practices for engineering, services, acquisition, project management, support, and process management • Developed under the sponsorship of DoD • Consistent with DoD and commercial standards Three constellations sharing common components and structure CMMI for Development - used by engineering organizations CMMI for Acquisition - used by buyers (e.g., govt. agencies) CMMI for Services - used by service providers (e.g., help desk) Software Process Improvement

11. Basic Building Blocks – 22 Process Areas Implemented by each project Implemented by the organization • Process Management • Organizational Process Focus • Organizational Process Definition • Organizational Training • Organizational Process Performance • Organizational Performance Management • Engineering • Requirements Development • Requirements Management • Technical Solution • Product Integration • Verification • Validation • Support • Configuration Management • Process and Product Quality Assurance • Measurement and Analysis • Decision Analysis and Resolution • Causal Analysis and Resolution Software Process Improvement Project Management • Project Planning • Project Monitoring and Control • Supplier Agreement Management • Integrated Project Management) • Risk Management • Quantitative Project Management

12. SG 1 Establish Estimates SP 1.1 Estimate the Scope of the Project SP 1.2 Establish Estimates of Work Product and Task Attributes SP 1.3 Define Project Lifecycle Phases SP 1.4 Estimate Effort and Cost SG 2 Develop a Project Plan SP 2.1 Establish the Budget and Schedule SP 2.2 Identify Project Risks SP 2.3 Plan Data Management SP 2.4 Plan the Project’s Resources SP 2.5 Plan Needed Knowledge and Skills SP 2.6 Plan Stakeholder Involvement SP 2.7 Establish the Project Plan SG 3 Obtain Commitment to the Plan SP 3.1 Review Plans That Affect the Project SP 3.2 Reconcile Work and Resource Levels SP 3.3 Obtain Plan Commitment GG 2 Institutionalize a Managed Process GP 2.1 Establish an Organizational Policy GP 2.2 Plan the Process GP 2.3 Provide Resources GP 2.4 Assign Responsibility GP 2.5 Train People GP 2.6 Manage Configurations GP 2.7 Identify and Involve Relevant Stakeholders GP 2.8 Monitor and Control the Process GP 2.9 Objectively Evaluate Adherence GP 2.10 Review Status with Higher Level Management GG 3 Institutionalize a Defined Process GP 3.1 Establish a Defined Process GP 3.2 Collect Improvement Information Expected Practices Provide Guidancefor Implementation and Institutionalization Project Planning – Implementation Project Planning - Institutionalization Software Process Improvement

13. Practice Ratings for the Organization/Projects Software Process Improvement

14. How is the CMMI Used for Process Improvement? IDEAL Model www.sei.cmu.edu/ideal/ Software Process Improvement

15. Typical CMMI Benefits Cited in Literature • Reduced costs • 33% decrease in the average cost to fix a defect (Boeing) • 20% reduction in unit software costs (Lockheed Martin) • Faster Schedules • 50% reduction in release turnaround time (Boeing) • 60% reduction in re-work following test (Boeing) • Greater Productivity • 25-30% increase in productivity within 3 years (Lockheed Martin, Harris, Siemens) • Higher Quality • 50% reduction of software defects (Lockheed Martin) • Customer Satisfaction • 55% increase in award fees (Lockheed Martin) Software Process Improvement

16. Lean Six Sigma

17. What is Lean Six Sigma (LSS)? • Lean Six Sigma is a powerful approach to improving business • LSS improvement projects are performed by teams • Teams use a set of tools and techniques to understand problems and find solutions • Lean Six Sigma integrates tools and techniques from two proven process improvement methods + Software Process Improvement

18. DMAIC: Define-Measure-Analyze-Improve-Control A proven, structured problem-solving approach for improving existing processes • Define: Identify the issue causing the perceived problem or improvement opportunity • Measure: Collect data from the process • Analysis: Study the data for cause and effect • Improve: Eliminate cause(s) and determine resulting effects • Control: Sustain the gains by monitoring the system Software Process Improvement

19. DMAIC Toolkit Define Measure Analyze Improve Control • Charter • Kano Model • Voice of the Customer • Voice of the Business • Quality Function Deployment • Voice of the Process • Data Collection Methods • Measurement System Evaluation • Cause & Effect Diagrams/ Matrix • Failure Modes & Effects Analysis • Statistical Inference • Reliability Analysis • Root Cause Analysis, including 5 Whys • Hypothesis Test • Design of Experiments • Modeling • ANOVA • Tolerancing • Robust Design • Systems Thinking • Decision & Risk Analysis • Statistical Controls • Control Charts • Time Series methods • Non-Statistical Controls • Procedural adherence • Performance Management • Preventive measures Software Process Improvement

20. Lean - Definition • Lean is the philosophy of improving performance by reducing waste and eliminating bottlenecks • Used together, the two methodologies become a powerful set of tools for improving business processes "The 8 Wastes - DOWNTIME" by Source (WP:NFCC#4). Licensed under Fair use via Wikipedia. Software Process Improvement

21. Organizational Adoption: Roles & Responsibilities • Champions – Facilitate the leadership, implementation, and deployment • Sponsors – Provide resources • Process Owners – Responsible for the processes being improved • Master Black Belts – Serve as mentors for Black Belts • Black Belts – Lead major Six Sigma projects • Typically requires 4 weeks of training • Green Belts – Lead minor Six Sigma teams, or serve on improvement teams under a Black Belt • Typically requires 2 weeks of training Software Process Improvement

22. A Typical Lean Six Sigma Project in Aerospace • The organization notes that systems integration has been problematic on past projects (budget/schedule overruns) • A Six Sigma team is formed to scope the problem, collect data from past projects, and determine the root cause(s) • The team’s analysis of the historical data indicates that ineffective peer reviews are leaving significant errors to be found in test • Procedures and criteria for better peer reviews are written, using best practices from past projects • A pilot project uses the new peer review procedures and criteria, and collects data to verify they solve the problem • The organization’s standard process and training is modified to incorporate the procedures and criteria, to prevent similar problems on future projects Software Process Improvement

23. Corporate Support

24. Organizational Infrastructure Required Policies, Processes,Templates & Tools Process Group Training Program Process Improvement Measurement Repositories Predictive Modeling Best-Practice Libraries Audits & Appraisals Communications Developing and maintaining mature processes requires significant time and investment in infrastructure Software Process Improvement

25. Northrop Grumman Example Systematically analyze quality and process data and trends to determine how to improve our processes Improve process assets based on internal and external best practices Deploy to programs Disposition Analysis Information ISO/AS9100 Findings • Systems/ Software Engineering Process Group • Working Groups • Program Management Advisory Board Industry Standards Policy Configuration Control Board CMMI Appraisal Findings InternalBest Practices Process Customer Comments ExternalBest Practices Procedures Independent Audits Lean Six Sigma Projects Checklists and Guides Lessons Learned & Metrics Templates and Examples Tools msCAS PAL eToolkit StartIt! My MS Portal Workbench PCDB Software Process Improvement

26. Lessons Learned • Multiple improvement initiatives helps encourage a change in behavior as opposed to “achieving a goal” • Reinforces that change (improvement) is a way of life • Benefits result from institutionalizing local improvements across the wider organization • CMMI establishes the needed mechanisms • Training over half the staff as Lean Six Sigma Green Belts resulted in a change of language and culture • Voice of Customer, data-driven decisions, causal analysis, etc. • Better to understand/use tools in everyday work than to adopt the “religion” • CMMI and Lean Six Sigma compliment each other • CMMI can yield behaviors without benefits • Lean Six Sigma improvements based solely on data may miss innovative improvements (assumes a local optimum) Software Process Improvement