Object-Oriented Analysis and Design

1. Object-Oriented Analysis and Design Lecture 9 The Capability Maturity Model

2. Background • Above all, this is about management, not technology. • Technology promises productivity gains, but they will likely be unfulfilled in a “maelstrom of an undisciplined, chaotic project.” • 1986, the SEI and Mitre decide to do something: a process maturity framework to help organizations improve their software process.

3. Background (cont.) • The methods are “software process assessment'' and “software capability evaluation.'' • Note the ideas, methods and the language are DoD-centric. • Once again, “guidance.”

4. Background (cont.) • The result of the SEI effort is the Capability Maturity Model (CMM), which is • “...sets of recommended practices in a number of key process areas that have been shown to enhance software process capability.” • A better mousetrap will come from a better mousetrap process.

5. The Immature Organization • Software processes are generally improvised during the course of a project. • Managers are reactionary, solving immediate crises. • Functionality and quality are compromised when hard deadlines approach. “The death march to Egghead.” • No objective basis for judging product quality; thus quality is difficult to predict.

6. The Mature Organization • The software process is accurately communicated to existing and new staff. • Work activities are actually carried out according to the planned process. • The mandated processes are usable and consistent with the way work actually gets done. • Roles and responsibilities are clear, schedules and budgets are based on historical performance.

7. Fundamental Concepts • Software process: methods, activities, practices used to develop and maintain software • Software process capability: the range of expected results achieved by following a software process. • Software process performance: the actual results achieved by following a software process.

8. More Concepts • Software process maturity: the extent to which a specific process is explicitly defined, managed, measured, controlled, and effective. • As an organization gains in process maturity, it institutionalizes its process by policies, standards and organizational structures.

9. The Five Levels • The key idea is to identify an evolutionary path from immature to mature. • There are a number of well-defined plateaus along the way. • Each plateau (level) is a set of process goals which, when satisfied, stabilize some component of the software process. • The process goals are prioritized, so organizations understand what to do next.

10. The CMM Levels

11. Level 1: The Initial Level • Unstable environment • Makes commitments that can't be meet with an orderly process. • Plans are scrapped during crises, jumping to coding and testing. • Can success happen? Yes, but only through heroic efforts.

12. Level 1 (cont.) • Such efforts will not likely be repeatable. • Capability is a characteristic of individuals, not the organization. • Would you buy from such an organization?

13. Level 2: The Repeatable Level • Management policies are in place. • Planning and managing are based on experience. • Policies are enhanced on a project by project basis. • Commitments are realistic.

14. Level 2 (cont.) • Managers track software costs, schedules, functionality. • Project standards are defined and followed. • The process is disciplined because planning and tracking are stable. • Earlier successes can be repeated.

15. Level 3: The Defined Level • A standard process for software engineering and management across the organization is documented. • There is a group responsible for this standard software process. • There are organization-wide training programs.

16. Level 3 (cont.) • Projects tailor the standard process to account for their unique features. • Such a tailored process contains readiness criteria, inputs, verification mechanisms, outputs and completion criteria. • This process capability is based on a common, organization-wide understanding of activities, roles and responsibilities.

17. Level 4: The Managed Level • The organization sets quantitative quality goals for software products and processes. • A process database is used to collect and analyze data from projects' processes. • Variation in process performance is narrowed to acceptable levels. • Software processes are instrumented.

18. Level 4 (cont.) • Risks in new product development are well understood. • The process capability is quantifiable and predictable. • Software products are of predictably high quality.

19. Level 5: The Optimizing Level • The entire organization is focused on continuous process improvement. • Weaknesses can be identified and remedies found. • Data on software process allows cost-benefit analyses on new technologies.

20. Management View of “Visibility” Into a Project • Moving through the levels, managers are better informed, and have greater control to do good…

22. Management View of “Visibility” Into a Project • Level 1: • The project is a black box • Requirements flow into the project haphazardly • The staging of activities is hidden • Managers can’t establish project status

23. Management View of “Visibility” Into a Project • Level 2: • Requirements are controlled • Visibility at defined occasions • The project is a sequence of black boxes • Management reacts to problems as they occur

24. Management View of “Visibility” Into a Project • Level 3: • Tasks inside each black box are known • Management and engineers understand their “roles and responsibilities” within each box • Management is proactive in dealing with problems, because of rapid status updates

25. Management View of “Visibility” Into a Project • Level 4: • Processes are “instrumented” (details of estimates and outcomes are recorded) • Decisions are based on hard facts • Outcomes can be predicted more accurately • Variability in outcomes gets smaller

26. Management View of “Visibility” Into a Project • Level 5: • Improved methods of software development are regularly tried • Defect-prone activities are replaced • Managers can predict the impact of process changes

27. Process Capability and Prediction of Performance • Three improvements are expected as the software process matures: • The difference between targeted results and actual results decreases. • The variability of actual results around targeted results decreases. • Targeted results improve as maturity increases.

29. Operational Definitions • These are supposed to be specific recommendations for an organization to increase its maturity. • There are at least four uses for them:

30. Operational Definitions (cont.) • Assessment teams use CMM to identify strengths and weaknesses. • Evaluation teams use CMM to identify risks in selecting contractors. • Upper management will use CMM to figure out how to launch a process improvement program. • Staff and process improvement groups will be guided by CMM.

31. Key Process Areas: Level 2 • Requirements management • Software project planning • Software project tracking and oversight • Software subcontract management • Software quality assurance • Software configuration management

32. Key Process Areas: Level 3 • Organization process focus • Organization process definition • Training program • Integrated software management • Software product engineering • Intergroup coordination • Peer reviews

33. Key Process Areas: Level 4 • Quantitative process management • Software quality management

34. Key Process Areas: Level 5 • Process change management • Technology change management • Defect prevention

35. Evaluation • Process maturity is evaluated by SEI teams (and those trained by them). • For each Key Process Area, an organization is rated according to: • Commitment to perform • Ability to perform • Activities performed • Measurement and analysis • Verifying implementation

36. Comments From Ed Yourdon • This is not the only model (cf. Software Productivity Research, the Capers Jones outfit) • What about small, innovative firms? According to the CMM, “Apple Computer should not exist.” • Small organizations probably cannot afford the overhead required by CMM.

37. Other Implications • You can't skip levels (these are cultural changes). • It takes time to go from one level to the next (10 years total? What about regression?). • New technology should be avoided at lower levels (these are management issues). • New software organizations won't start at level 3 (the all-star team). • Evaluations important for winning contracts.

38. A Serious Rebuttal • From James Bach: • “The CMM is a broad and increasingly deep set of assertions about good software development practice.” • Where do these assertions come from? Are they complete and correct?

39. Bach (cont.) • “My thesis...is that the CMM is a particular mythology about software process evolution that cannot legitimately claim to be a natural or essential representation of software processes.”

40. More Bach • The CMM is • “at best a consensus among a particular group of software theorists and practitioners” • “at worst a whitewash that obscures the true dynamics of software engineering [and] suppresses alternative models” • “If an organization follows it for its own sake, it may lead to the collapse of that company's competitive potential.”

41. Other Comments • The CMM was originally designed to evaluate the ability of government contractors to develop a software project. Will/does it work? • Is it useful outside of this context? • The biggest (popularly-known) names in software (Borland, Symantec, Apple, Microsoft) are likely Level 1. (Note in Jim McCarthy's book, CMM isn't even mentioned.)

42. Shrink-Wrap Perspective • No theoretical basis; fashioned by “very knowledgeable people,” so the de facto principle is that experts know what they're doing. • Any other model from “very knowledgeable people” has equal veracity. • Vague empirical support; this support doesn't exclude other models; there is no comparison of methods.

43. Shrink-Wrap (cont.) • It doesn't account for the success of shrink-wrap companies. • CMM reveres process but ignores people; process makes up for mediocrity; where is the justification for this?

44. Shrink-Wrap (cont.) • Imagine a process definition for playing chess. • Institutionalization of process guarantees nothing; such efforts lead to a simplified public process and an undercover private process. • Process dynamics are lost; why isn't training at level 1, for example? Level 1 is nothing; just get to level 2!

45. Shrink-Wrap (cont.) • Most of the Key Practices could be introduced at level 1, depending on the organization. • The CMM replaces the true mission of better process to the artificial mission of higher maturity level; “level envy.”

46. Other Arguments • The most powerful argument against CMM is the many successful companies that should not exist. Microsoft is level 1, IBM is mature. • Process should support innovation; systematic problem-solving leadership to enable innovation rather than cookie-cutter solutions.

47. Still More • CMM is profoundly ignorant of the dynamics of innovation. CMM pushes authority up in the organization. • For CMM, heroism is an unsustainable sacrifice on the part of individuals. Personal mastery is left out of the equation. • An alternative: well, Bach obviously has his own story...