to evaluate benefits realization to spi using cbd architecture n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
To Evaluate Benefits Realization to SPI Using CBD Architecture PowerPoint Presentation
Download Presentation
To Evaluate Benefits Realization to SPI Using CBD Architecture

Loading in 2 Seconds...

play fullscreen
1 / 29

To Evaluate Benefits Realization to SPI Using CBD Architecture - PowerPoint PPT Presentation


  • 100 Views
  • Uploaded on

To Evaluate Benefits Realization to SPI Using CBD Architecture. Nov. 5. 2003. Kyung whan Lee, professor Computer Science and Engineering Division, Chung-Ang University (Kwlee@object.cau.ac.kr). Table of Contents. Introduction Performance Measures and GQM Approach

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

To Evaluate Benefits Realization to SPI Using CBD Architecture


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
to evaluate benefits realization to spi using cbd architecture

To Evaluate Benefits Realization to SPI Using CBD Architecture

Nov. 5. 2003

Kyung whan Lee, professor

Computer Science and Engineering Division,

Chung-Ang University

(Kwlee@object.cau.ac.kr)

table of contents
Table of Contents
  • Introduction
  • Performance Measures and GQM Approach
  • Data Collection and calibration for the process performance
  • GQM-based Questionnaire for the process capability
  • Process Capability Model to the Earned Value
  • Benefits Realization Model and Conclusions
  • References
introduction 1
Introduction (1)

Lynch and Cross(1) suggested that customer satisfaction, flexibility, and productivity are the driving process upon which company objectives are based. Business goals and strategies, together with factual data about attributes of product quality and process performance, are the key drivers that lead to actions that improve a software process. We must also look at the benefit that justify the costs associated with changing the process.

introduction 2

Goals and Strategies

Measure

Vision

Bussiness Unit

Financial

Market

Bussiness Operating Systems

Customer

Flexibility

Productivity

satisfaction

Cycle time

Departments and Work Centers

Quality

Delivery

Waste

External Effectiveness

Internal Efficiency

Introduction (2)

Figure1. Performance Pyramid

2 performance measures and gqm approach 1
2. Performance Measures and GQM Approach (1)

The new measurement is formed under a simple assumption: fast cycle time and elimination of waste improve productivity and enable lower prices and increased sales, which in turn improve financial performance.

On the figure 1, Performance Pyramid, we can make the relationships between the local operating measures and the financial results. For example, companies that achieve fast cycle times are reporting triple the revenue growth and double the profits over industry-average competitors. Fast cycle times and the elimination of waste will equal better financial performance.

The GQM approach (3) is a systematic way to tailor and integrate an organization’s objectives into measurement goals and refine them into measurable values. The approach assumes that software development is still an immature engineering discipline and thus much is unknown about its characteristics and performance.

2 performance measures and gqm approach 2
2. Performance Measures and GQM Approach (2)

The GQM-based measurement approach has been implemented as part of an organization-wide process improvement program. As a direct result of the measurement program, interest in software reuse has increased and several other projects have renewed their efforts to develop reusable models. The reuse package described earlier is still frequently used within the organization to show the benefits of reuse.

2 performance measures and gqm approach 3

Reduced price

Increased revenues

Improved benefit realization

Increased earned value

Improved return on investment (ROI)

Increased sales

More customers

Lower General and Administrative expense

Lower operating expense

Improved

Quality

Faster

cycle

time

Less waste

(e.g. carrying reuse)

On-time

delivery

2. Performance Measures and GQM Approach (3)
  • The new measurement and the bottom line
3 data collection and calibration for the process performance 1
3. Data Collection and calibration for the process performance (1)

The operational activities of measurement begin collecting data. The procedures that we define for collecting and refining data need to be integrated into our software processes and made operation.

Collecting data is more than just making the measurements. It consists of the implementing our plans, ensuring the work, and sustaining for the measurement activities of the results. The documentations of our procedure include follows:

  • Identifying the responsible persons and organizations
  • Specifying where, when, and how measurements will be done
  • Defining the procedure to be used for recording and reporting the results
3 data collection and calibration for the process performance 2
3. Data Collection and calibration for the process performance (2)

We collected the data by performing the assessments with SPICE and marked GQM-based process maturity questionnaire. The requirements help to ensure that the assessment output is self-consistent and provide the evidence for the substantiation of markings. The assessment has to be conducted in accordant to the documented process that in capable of meeting the assessment process. The assessment process shall contain minimally the following activities such as planning, data collection, data validation, process rating, and reporting. The qualified assessors are educated and certified with ISO Guide62 and TR15504, and I identify the responsible persons to be rating on the maturity questionnaire.

For the calibration of the isolate data, need to verify the calibration model. These problems are caused by two sources. First, the capability of respondent to the questionnaire is low. Second, it is the case of confusion for the respondent by the item of questionnaire itself.

3 data collection and calibration for the process performance 3
3. Data Collection and calibration for the process performance (3)

Now, we make a decision to exclude the first source because the reviewer is the certified assessor. If, however, there is a wild deviation between assessors and is too large, there will be some problem in the product of work on the process. And there are some ambiguities in the process definition and its questionnaire. Therefore, we want to verify the confidence of the calibrated model with the hypothesis and assumptions as shown in <Table 1>

3 data collection and calibration for the process performance 4
3. Data Collection and calibration for the process performance (4)

<Table 1> Assumption and hypothesis to verify the calibrated model

Assumption 1 : Current maturing questionnaire is not trusted

Assumption 2 : If population follows the normal distribution (Z) by the science of statistics theory, it will be in the administration section (believability) within more than 98%.

Assumption 3 : The same (n) of population, which follows the normal distribution by the science of statistics theory, forms T-distribution.

Assumption 4 : In case of the distribution sample group (N) comes to population and is to follow the normal distribution (Z), if the sample is followed to the T-distribution by the science of statistics theory.

Null hypothesis (H0) : if the sample group (N) is the significant leve (P) = 0.2, that is in reliable if it showed within the confidence interval 99.8%.

Alternative hypothesis (H1) : Cannot be trusted sample group (N) is not existed within the confidence interval 99.8%.

3 data collection and calibration for the process performance 5

Population Data

Sample space=>(N)

(Inference)

n1

n2

n3

n4

Defect Data

3. Data Collection and calibration for the process performance (5)

Let us <Table 1> put into a diagram as follows;

<Figure 3> Process data verification

3 data collection and calibration for the process performance 7
3. Data Collection and calibration for the process performance (7)

Therefore, the calibration model established as presented in <Table 2>

<Table 2> calibration model of process data

4 gqm based questionnaire for the process capability 1
4. GQM-based Questionnaire for the process capability (1)
  • Project practical experiences
    • Size of work product
    • Defect opportunity
    • Capability of defect operation
    • Defect introduction by Software quality assurance activity
    • Development maturity capability using CBD Architecture
  • Capability metric subscript with the GQM-based approach

I can get the meta data using by the Goal-Question-Metrics approach as follows:

    • Quality:
      • Defect density
      • Impact of requirement change rate
      • Effectiveness using CBD architecture
4 gqm based questionnaire for the process capability 2
4. GQM-based Questionnaire for the process capability (2)
  • Delivery:
    • Delivery time
    • Reduced time-to-market using CBD architecture
  • Cycle time:
    • Classification of process, activity, and task time
    • Man-Months used(effort distribution)
    • Size estimation
    • Efficiency using CBD architecture
  • Waste:
    • Rework
    • Productivity
    • Reuse
    • Efficiency using CBD architecture
5 process capability model to the earned value 1
5. Process Capability Model to the Earned Value (1)

Earned value is measure of budgetary performance that relates actual expenditure to technical achievements as determined by milestone completions. Trend analysis use in conjunction with earned value. In business case, trends are heavily used in forecasting. For examples, managers can consider future raises when estimating labor costs by inflating salaries based on trends in economic index data. They can plot buying trends as a function of their sales forecast.

In software organizations during testing, software engineers can examine defect find-and-fix rates to determine whether they are removing defects more quickly than they are finding them. Software managers can plot their rate of progress in achieving milestones as a means to assess their programmatic performance to supplement concept like earned value.

5 process capability model to the earned value 2
5. Process Capability Model to the Earned Value (2)

6 Sigma typically monitors the process using control charts, which compare control limits with specification limits to determine process capability. We measure process capability in terms of capability indices which are simplified measures that quickly describe the relationship between process variability and the specification limits’ spread.

Process capability is the number of standard deviations that fit between the mean and specification limit; this unit corresponds to defect probability. The essential value of process capability analysis is its ability to predict the percentage of process characteristics or products that the process will produce within specification limits.

When analyzing process performance data, we must be constantly concerned that we have identified all sources of variation in the process. When data are aggregated, our process will be particularly susceptible to overlooked or hidden sources of variation.

5 process capability model to the earned value 3
5. Process Capability Model to the Earned Value (3)

The example that follows shows some of the advantages to calculate the process capability.

1) On the quality of performance pyramid;

- work product defect density

- technical documentation defect density

- code defect density

- defect management density

- requirement change rates

- typical costs : appraisal costs

5 process capability model to the earned value 4
5. Process Capability Model to the Earned Value (4)
  • On the delivery of performance pyramid;
    • differences between the budget and the actual progress
    • differences between the plan and the actual progress of planning and analysis activity
    • differences between the plan and the actual progress of design activity
    • differences between the plan and the actual progress of construction activity
    • differences between the plan and the actual progress of testing activity
    • typical costs : document, reports, records, memos, indoctrination costs of custom process
5 process capability model to the earned value 5
5. Process Capability Model to the Earned Value (5)
  • On the cycle time of performance pyramid;
    • resources allocation for the planning and analysis activity
    • resources allocation for the design activity
    • resources allocation for the construction activity
    • resources allocation for the testing activity
    • differences between the budget and the actual human resources
    • typical costs : policies, procedures, processes, activity
  • On the waste of performance pyramid;
    • productivity
    • documentation size to SLOC
    • work product size to a man-month
    • reuse rates of code
    • accuracy rates of size estimation
    • typical costs : training, labor hours, travel costs, project costs, administration
5 process capability model to the earned value 6

yes

yes

BCWP >=

BCWS?

BCWP >=

Cost?

Develop/update

Plans, BCWS

Performance

to plans

yes

no

no

Determine Corrective Action

5. Process Capability Model to the Earned Value (6)

Now, we can design the earned value system monitors benefis realized;

- financial benefits realized vs. cost (ROI)

- benefits realized vs. schedule

- actual cost vs. schedule

and propose classical earned value process as follows (4);

BCWS : Budgeted Cost of Work Scheduled,

BCWP : Budgeted Cost of Work Performed, Cost : Actual Cost of Work Performed

5 process capability model to the earned value 7

Assumption

INTER MEDIATE OUTCOME

•reduce defect • time to market • cycle efficiency • reduce rework

∂F

OUTCOME

= 0

∂Rq * ∂Rd

• productivity

• increased customer

satisfaction

• flexibility

∂F

= 0

∂Rc * ∂Rw

5. Process Capability Model to the Earned Value (7)
  • Benefit Realization Results Chain to Calculate Earned Value

: ① elaboration of results chain assumption

- base of estimation

② value management

-assumption monitoring

- progress/risk monitoring

: earned value extension

- change management

contribution

contribution

: reduce time to deliver product-delivery effectiveness

: reduce time to process

6 benefits realization model and conclusions 1
6. Benefits Realization Model and Conclusions (1)

On GQM-Based questionnaire we defined quality and delivery for the external effectiveness, and cycle time and waste for the internal efficiency to performance pyramid as follows;

quality ::= Rq (defect management, process capability, SPI cost,

component asset),

delivery ::= Rd (schedule, progress, marketing, time-to-market)

and

cycle ::= Rc (process capability, process improvement,

resource management)

waste ::= Rw (productivity, reduce rework, reuse capability,

expenditure rate)

6 benefits realization model and conclusions 2
6. Benefits Realization Model and Conclusions (2)

Now we propose the benefit maximize function as follows;

F (Rq, Rd, Rc, Rw) = 0.

To maximize benefit function for the external effectiveness,

∂F

= 0,

∂Rq* ∂Rd

and to maximize benefit function for the internal efficiency,

∂F

= 0,

∂Rc* ∂Rw

6 benefits realization model and conclusions 3
6. Benefits Realization Model and Conclusions (3)

These condition links to assumptions, which condition the realization of outcomes on the benefit-realization approach chain.

For the assumptions we can propose the procedures for benefit realization as follows;

6 benefits realization model and conclusions 4

n

aijA1j

i,j =1

6. Benefits Realization Model and Conclusions (4)

I. To evaluate process capability and estimate project size,

II. To calculate investment cost

A0 : the existence project cost

A1 : cost for SPI using CBD Architecture

A1j : j = 1, n (SPI items), aij : i, j = 1, n (weight of each item)

And then

A1 = Σ

  • To make cause-effect graph for the weighted cost
  • To make cause-effect graph for the weighted external effectiveness and internal efficiency
  • Now calculate the following benefits;
6 benefits realization model and conclusions 5

External Effectiveness + Internal efficiency

A0+A1

project benefits

project benefits – project cost

project costs

project cost

6. Benefits Realization Model and Conclusions (5)

benefit realization = (%)

Benefit / cost ratio =

ROI = * 100(%)

GQM-Based questionnaire gives us potential indicators of customer satisfaction, process flexibility, and productivity. Using them we can evaluate the benefit realization to software process improvement with CBD architecture which identify process measure to the business indicators and relate the costs and benefits of SPI also to the business indicators.

slide28
요약문

GQM 기반 설문서는 고객에 대한 만족도, 프로세스의 유연성, 및 제품의 생산성을 측정하고 이를 위해서 투자된 비용의 효율을 평가 할 수 있는 데이터를 수집해 준다.

CBD 아키텍쳐에 의해서 GQM기반 설문서에 의해서 수집할 데이터를 조정하여 신뢰성 구간을 설정하고 SPI에 대한 현실적인 이익을 계산하는 것은 CBD 아키텍쳐를 사용한 소프트웨어 시스템의 효율을 평가하는 기초가 된다

비즈니스 버전에 맞추어서 CBD 아키텍쳐를 사용하여 개발한 소프트웨어 시스템의 Benefit Realization, Benefit/Cost Ratio, 및 ROI를 계산할 수 있는 모델은 CBD 아키텍쳐의 재사용을 권장할 수 있는 기준과 지침이 될수 있다.

references
References
  • William A. Florac, Anita D. Carleton, Measuring the software process, 1999. SEI Series, Addison Wesley.
  • Richard L. Lynch, Kelvin F. Cross, Measure up!, 1995, Blackwell.
  • Frank Van Latum, Rini Van Soligen, Adopting GQM-Based Measurement in an Industrial Environment, 1998, IEEE software.
  • Value-Based software Engineering : A Case Study, Barry Boehm, Computer, March 2003, IEEE