benefits of cmmi within the defense industry l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Benefits of CMMI Within the Defense Industry PowerPoint Presentation
Download Presentation
Benefits of CMMI Within the Defense Industry

Loading in 2 Seconds...

play fullscreen
1 / 55

Benefits of CMMI Within the Defense Industry - PowerPoint PPT Presentation


  • 139 Views
  • Uploaded on

Benefits of CMMI Within the Defense Industry . Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 May 2010 . Outline. Introduction Benefits of CMMI Implementation Quantitative Qualitative Looking Ahead Summary .

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Benefits of CMMI Within the Defense Industry' - najwa


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
benefits of cmmi within the defense industry

Benefits of CMMI Within the Defense Industry

Software Engineering Institute

Carnegie Mellon University

Pittsburgh, PA 15213

May 2010

outline
Outline
  • Introduction
  • Benefits of CMMI Implementation
          • Quantitative
          • Qualitative
  • Looking Ahead
  • Summary

This report was created with the cooperation of the Systems Engineering Division (SED) of the National Defense Industrial Association (NDIA) and their member companies and DoD organizations.

purpose of presentation
Purpose of Presentation

Present new evidence about effective implementations of CMMI

  • Examples are provided by the defense industrial base and DoD organizations.
  • New examples are based upon the measures that practicing organizations use to track value to their businesses.
  • Examples are provided by organizations that have tracked and measured performance improvements from using CMMI over many years.
  • Many of the organizations emphasize high maturity results and show that they enabled superior performance.
  • Their data indicate why CMMI is important to the DoD & its suppliers.

The new data presented in this report demonstrates that effective implementation of good practices aided by use of CMMI can improve cost, schedule, and quality performance.

cmmi major benefits to dod
CMMI: Major Benefits to DoD

“Does CMMI work?” We asked our nation’s defense contractors, as well as government agencies, to share results from their performance improvement efforts using CMMI. The results spoke for themselves: “Yes, CMMI works!”

The following slides include information from six defense organizations that responded.*

*Results reported in this presentation are not attributed to protect confidentiality.

background on the data for this presentation
Background on the Data for this Presentation

Organizational and project leaders decided which measures were most useful to them when tracking the results of CMMI-based improvements.

A common thread was their interest in measuring the effect CMMI had on schedule, effort and cost, and quality.

The summarized results demonstrate the wide scope of business values and goals of the participating organizations.

The source studies in this presentation used current data as follows:

  • 2010: Organizations 1, 2A, 3, & 6
  • 2009: Organizations 5 & 7
  • 2008: Organization 2B
selected results high maturity reduces costs for repair organization 1
Selected Results: High Maturity ReducesCosts for Repair (Organization 1)
  • High Maturity Projects
  • Discover defects earlier
  • Early detection and repair lowers Costs
  • 57.7% fewer hours for ML5 projects expended to repair defects versus ML3
  • 105.3 fewer hours per defect
    • 88.6 fewer hours during Testing alone
    • When largest risk to schedule occurs
slide9

Selected Results: Effort to Repair Defectsby Phase (Organization 1)

Hours to Repair Defects

(by Phase)

(233 KESLOC Avg Project)

45,000

42,519

40,000

Maturity Level 3

57.7% fewer hours (24,527)

Maturity Level 5

expended for ML 5

35,000

30,000

6.35 times(20,641 hrs) less

24,496

25,000

risk of Cost or Schedule

impact late in program

20,000

17,992

15,000

11,022

10,000

8,309

5,155

3,651

3,855

5,000

2,177

1,846

-

Req & Design

Code & UT

Sys & Acpt Test

Post Delivery

Total Hours

Potential Cost Savings From $ 1.9 M to $2.3 M per average sized program

selected results software productivity organization 1
Selected Results: Software Productivity(Organization 1)

Average project size was 233 KESLOC

Largest = 1,360 KESLOC

Smallest = 29 KESLOC

  • Average customer project savings due to increased productivity
    • Equivalent of 406 work months per project (33.8 work years)
selected results award fee organization 6

Percent

SW CMM L2

SW CMM L3

SW CMM L4

SW CMM L5

CMMI L5

CMMI L5

Selected Results: Award Fee (Organization 6)

50% of Potential

Additional Award Fee

Achieved

Potential Additional Award Fee Available

Customer Satisfaction Continues to Improve

quantitative result return on investment organization 2a
Quantitative Result: Return on Investment (Organization 2a)

Organization 2a reported their quantified ROI from CMMI Maturity Level 5 activity to be 24 : 1.

  • Using the data in Performance Results of CMMI ® -Based Process Improvement (CMU/SEI-2006-TR-004) they were able to compare their ROI performance to others in industry:
      • Median ROI 4 : 1
      • Lowest ROI 1.7 : 1
      • Organization 2a24 : 1
      • Highest ROI 27.7 : 1
  • These results are a consequence of meaningful process improvement aligned with the business and engineering objectives.
cmmi provides many qualitative benefits as well
CMMI Provides Many QualitativeBenefits as Well*
    • Reduced overtime and less intense pressure
  • Clear roles and responsibilities for business execution
  • Common language (i.e., defined processes, measures) across business units
  • Decrease in replanning
  • Products with lower levels of defects and lower risk; one organization offers a lifetime warranty on products

Organizations also gathered various qualitative measures to compliment their quantitative measurements. They found qualitative benefits such as:

  • Improved program insight,control, and tracking
  • Reducedtraining: process documentation enables knowledge transfer to new generation of workers
  • Process transformation (via consistency, integration, coordination)
  • Personnel retention and job satisfaction
  • *based on published benefits from a wide variety of organizations
the bottom line
The Bottom Line

Why improve processes? - Because processes are the foundation for all other business improvements, and critical for

    • lasting improvements
    • successful technology insertion

If a performance management system is not in use, leadership is unaware of what is and is not working.

CMMI is a proven approach to performance management – with more than a decade of results showing it does work.

Organizations have provided data that shows CMMI

  • enables the delivery of lower-defect products, with predictable cost, schedule, and quality
  • improves business performance
  • serves as competitive discriminator
results depend on implementation
Results Depend on Implementation

Simply deciding to “do CMMI” is not enough to achieve benefits.

Defining good processes, using them, measuring the results, and making improvements based on what you learn are all key to reaping the benefits described in this presentation.

The CMMI models are a foundational part of a comprehensive approach to process improvement that helps organizations understand

  • why they should improve
  • what frameworks and tools would best fit their needs
  • how to implement them
cmmi research references
CMMI Research - References

Bibliographic information cited in this presentation:

looking ahead
Looking Ahead

The road ahead for CMMI implementation

  • A continued focus on high maturity

More and more organizations are striving for and achieving high maturity – and are collecting data demonstrating the benefits. Once at ML 4 or 5, organizations must maintain their focus on good implementation practices for continuous improvement.

  • Implementation of CMMI for Services (CMMI-SVC)

CMMI-SVC extends the benefits of CMMI to a new audience. Service providers can use the model concept that has proven useful in the development community to specifically address their interests and concerns.

  • Implementation of CMMI for Acquisition (CMMI-ACQ)

CMMI-ACQ helps organizations improve relationships with their suppliers and improve acquisition processes. The model can enable increased control of projects, better management of global sourcing of products and services, and more successful acquisition solutions.

  • Integration with other improvement paradigms (e.g., TSP, ISO, Lean Six Sigma)

Organizations are finding that integrated improvement initiatives can produce outstanding results. Choosing CMMI doesn’t mean discontinuing improvement efforts already in place or avoiding new ones that show promise.

summary
Summary

Many stakeholders are involved in the development and maintenance of CMMI models, with participants from commercial industry, government, and the DoD. Broad adoption has occurred worldwide. Adopters range from small and midsize organizations (these are the majority) to large and very large organizations.

Organizations that provide products and services to the DoD use CMMI to improve programs, systems, product and service management, systems and software engineering, work processes, and training solutions.

Quantitative and qualitative results have been documented by defense contractors and others, as shown in this report. There is a great deal of additional data showing the benefits of CMMI from a broad range of industries, including banking and finance, manufacturing, medical, and others.

CMMI enables performance improvement focused on business objectives, but the level of success depends on the implementation.

background the achievement of excellence
Background: The Achievement of Excellence

CMMI leads the way to high performance through improved processes.

The management of the development and delivery of software systems must be guided by quantitatively managed processes.

Performance comes from processes that are

predictable, repeatable, and continuously improving

in terms of product quality, cost and schedule performance,

process performance, and customer satisfaction.

results overview quantitative measures
Results Overview – Quantitative Measures
  • We received data/information showing performance improvements in the following categories:
  • Schedule
  • Effort/cost
  • Quality
  • Customer satisfaction
  • Business growth
background leadership stewardship and evolution of maturity models
Background: Leadership, Stewardship, and Evolution of Maturity Models
  • Many stakeholders have been involved in the development and evolution of the maturity models published by the SEI, with hundreds of people contributing their time and expertise over the years.

Industry participants

Government participants

41

cmmi adoption knows no borders
CMMI Adoption Knows No Borders

There are 33 countries with more than ten appraisals as of March 2010:

USA 1582

China 1229

India 524

Japan 306

Spain 180

France 168

Korea (ROK) 165

Brazil 144

Taiwan 134

U.K. 113

Mexico 86

Argentina 77

Germany 76

Malaysia 71

Canada 59

Egypt 43

Italy 43

Thailand 38

Chile 37

Australia 36

Also: Colombia, Pakistan, Philippines, Singapore, Israel, Hong Kong,

Vietnam, Turkey, Netherlands, Portugal, Sri Lanka, Ireland and Russia

An estimated 1.8 million people work in organizations that have had at least one SCAMPI A appraisal since April 2002.

cmmi works for organizations of all sizes
CMMI Works for Organizations of All Sizes

1 to 10058.2%

201 to 2000+23.8%

Source for these statistical analyses: http://www.sei.cmu.edu/cmmi/casestudies/profiles/cmmi.cfm

cmmi adoption is multi sector
CMMI Adoption Is Multi-Sector

Services71.1%

Manufacturing16.3%

Source for these statistical analyses: http://www.sei.cmu.edu/cmmi/casestudies/profiles/cmmi.cfm

why care about improving software engineering performance
Why Care about Improving Software Engineering Performance?
  • To improve software engineering cost, schedule, and quality performance
  • To improve competitive economic and military advantage
cmmi a strong partner for dod and the defense industrial base
CMMI: A Strong Partner for DoD and theDefense Industrial Base

Large or small, organizations that provide products and services to the DoD share common challenges, from meeting defense software specifications and requirements, to securing networks, to developing and retaining a talented workforce.

CMMI helps the defense industrial base create better systems management, improved software engineering, more efficient processes, and tailored training solutions.

CMMI’s worldwide growth even in tough economic times indicates the value of the framework.

the cmmi mission vision at the sei
The CMMI Mission & Vision at the SEI

Mission

  • Improve the development and acquisition of software through research, and the transition to practice, of new, breakthrough, but proven engineering management methods. [by proven we mean having hard data and evidence]

Vision

  • Systems and software engineering management are guided by facts, models, and data that are shown to predictably improve performance and results well beyond the limits of current practice.
  • The practice of managing engineering work is recognized to be not just the responsibility of management, but of professionals at all levels and in every related activity.
  • Professionals that are developing or acquiring systems think and manage quantitatively.
improving performance requires knowledge and expertise
Improving Performance Requires Knowledgeand Expertise

The “How” – Appraisal Methods, Operational Practices, Improvement Techniques, Measurement and Analysis Tools

The “What” – Quality Principles

slide37

Causal Analysis and Resolution

Organizational Innovation and Deployment

What processes characterize

high maturity organizations?

Quantitative Project Management

Organizational Process Performance

quality and process performance objectives
Quality and Process Performance Objectives

The engine that drives project performance

The engine that drives business performance

CPI

Productivity

Defect Density

SPI

Defect Containment

Requirements Volatility

The engine that drives high maturity

organizational process performance opp
Organizational Process Performance (OPP)

The purpose of Organizational Process Performance (OPP) is to establish and maintain a quantitative understanding of the performance of selected processes within the organization’s set of standard processes in support of achieving quality and process-performance objectives, and to provide process-performance data, baselines, and models to quantitatively manage the organization’s projects.

quantitative project management qpm
Quantitative Project Management (QPM)

The purpose of Quantitative Project Management (QPM) is to quantitatively manage the project’s defined process to achieve the project’s established quality and process-performance objectives.

causal analysis and resolution car
Causal Analysis and Resolution (CAR)

The purpose of Causal Analysis and Resolution (CAR) is to identify causes of selected outcomes and take action to improve process performance.

organizational innovation and deployment oid
Organizational Innovation and Deployment (OID)

The purpose of Organizational Innovation and Deployment (OID) is toproactively seek, identify, select and deploy incremental and innovative improvements that measurably improve the organization’s processes and technologies. The improvements support the organization’s quality and process-performance objectives as derived from the organization’s business objectives.

why do we need improved performance and better processes
Why do we need improved performanceand better processes?

- Because there is STILL a management crisis in software!

A recent Standish report confirms that the number of troubled projects rises each year 

The result?

  • Losses in the millions for the government agencies and companies affected 
  • Leadership can be unaware of what is and is not working
  • Without a robust performance management system, management is operating without needed data to make quality decisions

Source: http://www.projectsmart.co.uk/docs/chaos-report.pdf

measurement challenges
Measurement Challenges

There are challenges in measuring return on investment in a traditional sense when correlating the CMMI framework to predictable performance improvement in a given organization

  • Companies that wanted to adopt CMMI had little data related to their conditions before starting and generally took no data to record their investments in applying the framework
  • They rarely had any data on their organizations performance until after they were well along the path to adherence
  • As a result, the ROI data published was generally notional and unsupported
cost to implement cmm
“Cost” to Implement CMM

Company A

Company J

Company F

Company C

Company M

Company I

recurring costs for pi
Recurring Costs for PI

Company H

Company B

Company M

Company D

Company I

Company C

Company O

cmm cost details
CMM Cost Details
  • Company A: $4.5M, 18 Months, 350 People, Level 2
    • Complete Outsourcing of CMM Support
  • Company C: NTE 5% of Budget; 1 Year, 30 People, Level 3 (SW)
    • Extensive Capture of Cost to Implement
  • Company B: 2% of Budget, 1 Year, 560 People, Level 2, Then 3
    • Extensive Outsourcing of CMM Support
  • Company O: 1.5-2.5%, 18 Months, 180 People, Level 2 (SW)
  • Company J: $900K, 2 Year, 400 People, Level 3 (SW)
  • Company D: 2% of Annual Budget, 150 People, No Assessment
    • 5 Years to Best Productivity; All Costs Not Captured
  • Company F: $1M, 150 People, Level 2, Then 3
  • Company M: Staffing at 3-5%; Up to 5% for Metrics Expense
  • Company H: 2% of Budget, 60% of Company at Level 4
  • Company I: Implement costs: $2.5M, 2.5 years, Level 3 (SW/SE), not all costs included, 15% workforce initial pilot. Sustainment costs: 15.25 work-years government, 4 full and part-time contractors. 3600 employees.

50% of Costs Devoted to Training

slide55

This material is considered SEI-Proprietary and is distributed by the Software Engineering Institute (SEI) to SEI Staff ONLY.

This material SHALL NOT be reproduced or used for any other purpose without requesting formal permission from the SEI at permission@sei.cmu.edu.

THE MATERIAL IS PROVIDED ON AN “AS IS” BASIS, AND CARNEGIE MELLON DISCLAIMS ANY AND ALL WARRANTIES, IMPLIED OR OTHERWISE (INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR A PARTICULAR PURPOSE, RESULTS OBTAINED FROM USE OF THE MATERIAL, MERCHANTABILITY, AND/OR NON-INFRINGEMENT).