How to implement a measurement program
Download
1 / 36

How to Implement a Measurement Program - PowerPoint PPT Presentation


  • 100 Views
  • Uploaded on

2002. 2001. 2000. 1999. 1998. 1997. How to Implement a Measurement Program. Why is Measurement so Important?. To improve decision-making To monitor improvement Allow the synergy of the people. Two Elements of a Measurement Program. Static perspective - the outcome

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' How to Implement a Measurement Program' - nicki


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
How to implement a measurement program

2002

2001

2000

1999

1998

1997

How to Implement a Measurement Program


Why is measurement so important
Why is Measurement so Important?

  • To improve decision-making

  • To monitor improvement

  • Allow the synergy of the people


Two elements of a measurement program
Two Elements of a Measurement Program

  • Static perspective - the outcome

    A structured set of measures related to goals and quantifiable objectives

  • Dynamic perspective - how the outcome is achieved

    Activities that must be planned, scheduled and controlled, and managed by experts with clearly defined responsibilities


Application
Application

The measurement program is also:

  • An application

  • In constant evolution

  • To be maintained by competent personnel


How to cope with roadblocks
How to Cope with Roadblocks

  • Measurement programs not related to objectives are useless and not worth the data collection and data storage expenses involved. A measurement program is a tool, and not itself an objective;

  • The measurement program should also take in consideration the organizational and individual dimensions, and discriminate between public and private data. One of the major obstacles to data collection is the confidential nature of the data and of their use (e.g. effort management);

  • Measurement programs must be nurtured in a favorable environment that permits timely, valid and complete measures. The measurement results must also be delivered in a timely manner, otherwise they are useless for decision-making; furthermore, an incomplete or invalid measure may lead to incorrect conclusions and decisions.


Parts of a measurement program

1

Maturity Level - Key Processes

2

6

4

Goals and Objectives

Reports

Standards

3

3

5

Resource

Process

Product

Measures

Key Ratios

7

Parts of a Measurement Program

feedback

feedback

Model

Model

Tools used to

gather measurements

Analysis sequence


Software engineering institute s s e i capability maturity model c m m
Software Engineering Institute’s (S E I)Capability Maturity Model (C M M)

5

Optimizing

4

Managed

3

Defined

2

ISO 9000

Repeatable

1

Short term Target

Initial /Ad Hoc

  • Our organization


1 selection of processes select processes at your level of maturity

Levels

Level characteristics

Problems to work

Improving process

Automation

Optimized

Process change management

Technology innovation

Defect prevention

Quantitative : measured

processes

Managed

Qualitative : depends on

individuals

Process measurement and analysis

Quality management

Defined

Peer reviews

Intergroup coordination

Software product engineering

Integrated software management

Training program

Organization process definition

Organization process focus

Intuitive : depends on

individuals

Repeatable

Configuration management

Quality assurance

Subcontract management

Project tracking and oversight

Project planning

Requirements management

Initial

Ad hoc: control and plans non effective

1 - Selection of ProcessesSelect Processes at Your Level of Maturity


2 define goals and objectives example 1
2 - Define Goals and Objectives (Example 1)

Key process: Application Maintenance

Goal: Improve the value delivered by application maintenance

Objective: The percent of total dollars spent on Application Corrections compared to percent of total dollars spent on Preventive /Perfective/ Adaptive Maintenance will be reduced by 75% by the end of 1998.


2 define goals and objectives example 2
2 - Define Goals and Objectives (Example 2)

Key process: Configuration management

Goal: Reduce failures due to configuration errors

Objective: The number of failures attributable to configuration management shall be less than 1 in 100 by the end of 1998.

Objectives must be specific in terms of measurement / key ratio to be achieved and the timescale for achievement.


3 4 5 define measures

  • Define what to measure

  • Define how to measure

  • Define when to measure

3, 4, & 5 - Define Measures

Use “standards” (define your own, if necessary)

Need a mechanism or procedure


Questions
Questions

  • General term

    • what is a project ?

    • what is an application ?

    • what is maintenance ?

    • etc..

  • Do we have standard ?

  • Do we want to use tools ?

    • which one ?

    • why ?

    • when ?


Measures examples

Product

Resources

Process

Effort

Duration

# of defects

Maintenance Category

LOC

Function Point

Unit Process

Tools

Methods

Experience

Measures (Examples)


6 7 design the measurement program

Key process, goals

Entities

Attributes

Metrics

Objectives

Data

model

Questions

(how)

Definitions

Documentation

References

Process

Reports

Tools

6 & 7 - Design the Measurement Program


Data validation
Data Validation

  • There are two ways to see the data validation:

    • a priori

    • a posteriori


A priori
A Priori

  • Do we have a written procedure ?

  • Is it practical for the organization ?

  • Do we know the procedure ?

  • Do we have clear responsibilities ?

  • Can we follow and control the data ?


A posteriori
A Posteriori

  • What are the validation criteria ?

  • Can we explain marginal cases ?

  • How do we know we have the correct data ?

  • Do we have a lot on non elucidated cases ?


Success factors
Success Factors

  • For IS Group it is related to:

    • the level of maturity

    • the capacity to set objectives

    • the acceptance of standards by all the group

  • For the measurement group, it is related to:

    • the capacity to manage the measurement    program as a project

    • the capacity to deliver the measurement program as an application


Measures relating to maintenance objective

unit of

measure

frequency of

collection

recorded

by

stored

on

data item

number of Faults Reports raised

effort expended on corrective maintenance

effort expended on adaptive maintenance

elapsed time to clear Fault Reports

number of faults fixed

number of faults outstanding

size of system at start of maintenance stage

size of system at end of maintenance stage

number of Faults Reports that were not faults

integer

monthly

MT

FRL

hours

stage end

MT

PMW

hours

stage end

MT

PMW

hours

stage end

MT

PMW

integer

monthly

MT

FRL

integer

monthly

MT

FRL

FPC

repres.

FPC

database

FPC

start of stage

FPC

repres.

FPC

database

FPC

stage end

integer

stage end

MT

FRL

Key: MT maintenance team

FPC function point count

FRL fault report log

PMW project management workbench

Measures Relating to Maintenance Objective



Implementation of a measurement program
Implementation of a Measurement Program

  • Management commitment build-up

  • Staff commitment build-up

  • Selection of the key processes to be improved

  • Identification of the goals and objectives related to the key processes

Define

  • Design of the measurement program

  • Description of the Information System to be put in place

  • Deployment of the measurement program

Realize


Step 1 management commitment build up
Step 1 - Management Commitment Build-up

This first step consists of:

Finding the necessary information that will help management make a decision on the relevance of implementing a measurement program within the organization.


Step 1 issues
Step 1 - Issues

Middle managers do not readily see the relevance of initiating measurement programs in software engineering since :

  • They perceive them to be expensive and bureaucratic

  • They mention significant delays prior obtaining expected results

  • The impact of these measurements is limited

  • They often get contradictory advice from experts


Step 2 staff commitment build up
Step 2 - Staff Commitment Build-up

This step consists of:

Finding necessary arguments that will lead the staff involved in data collection process:

  • To accept the measurement program

  • To commit to support the measurement program


Step 2 issues
Step 2 - Issues

It appears that there is almost always major staff reluctance to accept a measurement program.

  • Main objections are related to:

    • Measures relevance

    • Validity of measure units

    • Effort required to complete measures

    • Effective use of measures

  • Solutions are related to the:

    • Choice of a pilot project

    • Documentation

    • Responsibilities

    • Training

    • Data availability


Step 3 selection of the key processes to be improved
Step 3 - Selection of the Key Processes to be Improved

This step consists in:

  • Evaluating the maturity level of the software development organisation

  • Selecting the priorities to be given to the key processes targeted for improvement programs.


Step 3 issues
Step 3 - Issues

  • It is possible that management sees this step as:

    • Not significant

    • Too expensive

    • Not useful and/or too long

  • Solutions are related to the:

    • Stick on the maturity level (part 1 of a measurement program)

    • Options confirmation or information


Step 4 identification of the goals and objectives
Step 4 - Identification of the Goals and Objectives

The purpose of this step is:

  • To determine goals and objectives of the measurement program

  • Related to the key processes

    (eg: improve estimating for development project)


Step 4 issues
Step 4 - Issues

  • The goals are sometimes:

    • Ambiguous

    • Redundant

    • In terms of action instead of intention

    • Not realistic

  • We must pay attention to:

    • The capability to achieve them

    • Their priority

Goals

Goals

Goals


Step 5 design of the measurement program
Step 5 - Design of the Measurement Program

This step consist of :

  • Designing a measurement program to

    • See if the objectives have been reached

    • Understand why they have not been reached


Step 5 issues
Step 5 - Issues

The issues are related to :

  • The organisations expectations about tools

  • The use of ill-defined concepts

  • The definition of objectives only related to the available measures

  • Weak or non-existent documentation


Roles and responsibilities
Roles and Responsibilities

Level of responsibilities to be considered:

  • Strategic level

    • Identify who are responsible to deliver or define objectives

  • Tactical level

    • Responsibilities related to the measurement support

    • The choice of methods, definitions, tools and standards

  • Operational level

    • Who provides the database on the standards with or without specific tools

    • It is also at this level that we can find the database and reports


Type of personnel
Type of Personnel

  • Sponsor

  • Champion

  • Expert

  • Consultant

  • Delivery personnel


Step 6 description of the information system to be put in place
Step 6 - Description of the Information System to be Put in Place

  • This step is related to :

    • The modeling of all the measures to be collected to meet the objectives

  • This data is based directly on the unit of measure and indirectly on the standards


Step 7 deployment of the measurement program
Step 7 - Deployment of the Measurement Program Place

This step consists of deploying the MP through:

  • Selection of a pilot site

  • Personnel training

  • Assigning responsibilities and tasks

  • Setting-up the measurement group


Step 7 issues
Step 7 - Issues Place

  • The main problems are :

    • Having to resolve to many problems in a short period of time

    • Training is not really addressed

    • Roles and responsibilities are not clear

    • The measurement group is in a situation of conflict

    • The delivery people are too busy

    • The measurement personnel is inexperienced

  • Possible solutions are related to :

    • The selection of a pilot site

    • The training

    • The roles and responsibilities

    • The choice of the personnel for the measurement group


ad