CMMI Level 5 Journey (A.K.A. Things I Wish Others Shared With Me) Linda Schultz February 2010

1. CMMI Level 5 Journey (A.K.A. Things I Wish Others Shared With Me) Linda Schultz February 2010

2. Agenda • Why is High Maturity CMMI Important to General Dynamics? • Our CMMI Journey • Why the Backslide? • Lessons Learned • Quality and Process-Performance Objectives (QPPOs) • Process Performance Models (PPMs) • Process Performance Baselines (PPBs) • Causal Analysis and Resolution Activities (CARs) • Organizational Innovations and Deployments (OIDs)

3. Importance of CMMI to General Dynamics • Organization has a strong affinity for process • CMM for software since 1980s • IEEE, MIL- and DoD- standards • CMMI Level 3 is a stated management expectation • We have contract requirements for certain CMMI levels • Strong belief that higher levels of CMMI maturity will make our developed products better, cheaper, and ready faster

4. CMMI Staged Representation

5. CMMI Journey

6. CMMI Journey We are here!

7. Why the Backslide?

8. CMMI Journey • We were appraised at CMMI Level 5 in 2005 • We just wanted to “renew” our credentials before they expired in 2008 • Started down the appraisal path in 2007 • What we did in the past didn’t work anymore • Big changes for the Level 4 and 5 Process Areas: OPP, QPM, OID and CAR • Artifacts that were acceptable in 2005 were no longer acceptable in 2007 • Needed to listen and learn . . . fast!

9. Changed Expectations QPPO Lessons Learned • A Quality and Process-Performance Objective (QPPO) is not just any well-stated goal or objective • QPPOs relate back to business objectives • QPPOs must be quantified and time-based • Reduce the escape of software defects by 15% prior to Integration Testing by December 31, 2010 • QPPOs are impacted by 1 or more well-documented sub-processes • Sub-processes must be composed of clearly defined steps • Sub-processes must be under statistical process control

10. Changed ExpectationsQPPO Lessons Learned (continued) • Organizational QPPOs flow down to the programs • Programs need to do some data analysis to assure that org QPPOs are achievable • If programs modify org QPPOs, they must provide rationale based on data analysis • Programs document their QPPOs in their Program QMP • For each program QPPO, there should be a strategy as to how it will be measured during the execution of the project and documented in the QMP • During program execution, if a program can’t meet a QPPO, they implement process changes to try to achieve it • If it’s not possible to achieve the QPPO, programs negotiate a change to their QMP and obtain management approval using the CAR process

11. Changed ExpectationsPPM Lessons Learned • A Process Performance Model (PPM) is not a model [at least as we knew it] • It’s not an application model – like CoCoMo • It is . . . • An equation that relates controllable and uncontrollable factors of a sub-process • Generated using modeling techniques (e.g., regression analysis) with tools such as Minitab • Used throughout a program’s lifecycle to evaluate current status and predict progress towards meeting QPPOs

12. Changed ExpectationsPPB Lessons Learned • Be careful about selecting the data for Process Performance Baselines (PPBs) • PPBs need to be based on: • “Clean” data vs. “Dirty” data • Data representative of current process • Sufficient # of data points on which to draw conclusions • Dis-aggregated data • Don’t combine all Systems Data in a single PPB; Break separately into Requirements, Integration & Test, etc.

13. Changed ExpectationsPPB Lessons Learned (continued) • Generated using a tool like Minitab • Initially, a program may use organizational PPBs until they have enough data points to calculate their own PPBs • Organizational PPBs represent a rollup of many programs’ data; programs should realize the risk of using Org PPBs • A program may use PPBs from a similar program that may be “closer” to their own (instead of using the org PPBs) until they have enough data points to calculate their own PPBs

14. Changed ExpectationsCAR Lessons Learned • A Causal Analysis and Resolution (CAR) activity isn’t just root cause analysis done on any problem • It is . . . • A program process area • Based on Common Cause (proactive) vs. Special Cause Variation (reactive) • Must be related to a quantitative measure on the program -- even better if related to a QPPO • Requires a measure of effectiveness • Effectiveness must be shown in terms of a process performance change in the program’s data • Not an ROI calculation

15. Changed ExpectationsOID Lessons Learned • An Organizational Innovation and Deployment (OID) isn’t just any process improvement • even if it’s well-planned, been piloted and been monitored • It is . . . • An organizational process area - the org version of CAR • Change affects a stable process • Process change requires a phased implementation (Pilot, Deployment, etc.) • Based on Common Cause • Requires a measure of effectiveness in terms of a process performance change in the program’s data - not an ROI calculation • Apply process changes across the organization – not on 1 program or function

16. Benefits From Our CMMI Journey • Lots of new training, tools, and templates being used • Managing more with objective data rather than subjective gut feel • Data being collected is of higher quality; allows better decisions to be made • More rigorous statistical analysis done for the organization and the programs as a matter of course • More dramatic process improvements on the programs – can see the difference in their charts • Organizational process improvements have broader applicability • Tighter coupling between business objectives, process improvement goals and program QPPOs

17. Questions?Comments?Contact Information:Linda SchultzGeneral Dynamics Advanced Information SystemsSenior Process and Quality Manager, andCMMI Program Manager952-921-6338linda.schultz@gd-ais.com