Loading in 2 Seconds...
Loading in 2 Seconds...
To Evaluate Benefits Realization to SPI Using CBD Architecture. Nov. 5. 2003. Kyung whan Lee, professor Computer Science and Engineering Division, Chung-Ang University (Kwlee@object.cau.ac.kr). Table of Contents. Introduction Performance Measures and GQM Approach
Nov. 5. 2003
Kyung whan Lee, professor
Computer Science and Engineering Division,
Lynch and Cross(1) suggested that customer satisfaction, flexibility, and productivity are the driving process upon which company objectives are based. Business goals and strategies, together with factual data about attributes of product quality and process performance, are the key drivers that lead to actions that improve a software process. We must also look at the benefit that justify the costs associated with changing the process.
The new measurement is formed under a simple assumption: fast cycle time and elimination of waste improve productivity and enable lower prices and increased sales, which in turn improve financial performance.
On the figure 1, Performance Pyramid, we can make the relationships between the local operating measures and the financial results. For example, companies that achieve fast cycle times are reporting triple the revenue growth and double the profits over industry-average competitors. Fast cycle times and the elimination of waste will equal better financial performance.
The GQM approach (3) is a systematic way to tailor and integrate an organization’s objectives into measurement goals and refine them into measurable values. The approach assumes that software development is still an immature engineering discipline and thus much is unknown about its characteristics and performance.
The GQM-based measurement approach has been implemented as part of an organization-wide process improvement program. As a direct result of the measurement program, interest in software reuse has increased and several other projects have renewed their efforts to develop reusable models. The reuse package described earlier is still frequently used within the organization to show the benefits of reuse.
Improved benefit realization
Increased earned value
Improved return on investment (ROI)
Lower General and Administrative expense
Lower operating expense
(e.g. carrying reuse)
delivery2. Performance Measures and GQM Approach (3)
The operational activities of measurement begin collecting data. The procedures that we define for collecting and refining data need to be integrated into our software processes and made operation.
Collecting data is more than just making the measurements. It consists of the implementing our plans, ensuring the work, and sustaining for the measurement activities of the results. The documentations of our procedure include follows:
We collected the data by performing the assessments with SPICE and marked GQM-based process maturity questionnaire. The requirements help to ensure that the assessment output is self-consistent and provide the evidence for the substantiation of markings. The assessment has to be conducted in accordant to the documented process that in capable of meeting the assessment process. The assessment process shall contain minimally the following activities such as planning, data collection, data validation, process rating, and reporting. The qualified assessors are educated and certified with ISO Guide62 and TR15504, and I identify the responsible persons to be rating on the maturity questionnaire.
For the calibration of the isolate data, need to verify the calibration model. These problems are caused by two sources. First, the capability of respondent to the questionnaire is low. Second, it is the case of confusion for the respondent by the item of questionnaire itself.
Now, we make a decision to exclude the first source because the reviewer is the certified assessor. If, however, there is a wild deviation between assessors and is too large, there will be some problem in the product of work on the process. And there are some ambiguities in the process definition and its questionnaire. Therefore, we want to verify the confidence of the calibrated model with the hypothesis and assumptions as shown in <Table 1>
<Table 1> Assumption and hypothesis to verify the calibrated model
Assumption 1 : Current maturing questionnaire is not trusted
Assumption 2 : If population follows the normal distribution (Z) by the science of statistics theory, it will be in the administration section (believability) within more than 98%.
Assumption 3 : The same (n) of population, which follows the normal distribution by the science of statistics theory, forms T-distribution.
Assumption 4 : In case of the distribution sample group (N) comes to population and is to follow the normal distribution (Z), if the sample is followed to the T-distribution by the science of statistics theory.
Null hypothesis (H0) : if the sample group (N) is the significant leve (P) = 0.2, that is in reliable if it showed within the confidence interval 99.8%.
Alternative hypothesis (H1) : Cannot be trusted sample group (N) is not existed within the confidence interval 99.8%.
Defect Data3. Data Collection and calibration for the process performance (5)
Let us <Table 1> put into a diagram as follows;
<Figure 3> Process data verification
Therefore, the calibration model established as presented in <Table 2>
<Table 2> calibration model of process data
I can get the meta data using by the Goal-Question-Metrics approach as follows:
Earned value is measure of budgetary performance that relates actual expenditure to technical achievements as determined by milestone completions. Trend analysis use in conjunction with earned value. In business case, trends are heavily used in forecasting. For examples, managers can consider future raises when estimating labor costs by inflating salaries based on trends in economic index data. They can plot buying trends as a function of their sales forecast.
In software organizations during testing, software engineers can examine defect find-and-fix rates to determine whether they are removing defects more quickly than they are finding them. Software managers can plot their rate of progress in achieving milestones as a means to assess their programmatic performance to supplement concept like earned value.
6 Sigma typically monitors the process using control charts, which compare control limits with specification limits to determine process capability. We measure process capability in terms of capability indices which are simplified measures that quickly describe the relationship between process variability and the specification limits’ spread.
Process capability is the number of standard deviations that fit between the mean and specification limit; this unit corresponds to defect probability. The essential value of process capability analysis is its ability to predict the percentage of process characteristics or products that the process will produce within specification limits.
When analyzing process performance data, we must be constantly concerned that we have identified all sources of variation in the process. When data are aggregated, our process will be particularly susceptible to overlooked or hidden sources of variation.
The example that follows shows some of the advantages to calculate the process capability.
1) On the quality of performance pyramid;
- work product defect density
- technical documentation defect density
- code defect density
- defect management density
- requirement change rates
- typical costs : appraisal costs
Determine Corrective Action5. Process Capability Model to the Earned Value (6)
Now, we can design the earned value system monitors benefis realized;
- financial benefits realized vs. cost (ROI)
- benefits realized vs. schedule
- actual cost vs. schedule
and propose classical earned value process as follows (4);
BCWS : Budgeted Cost of Work Scheduled,
BCWP : Budgeted Cost of Work Performed, Cost : Actual Cost of Work Performed
INTER MEDIATE OUTCOME
•reduce defect • time to market • cycle efficiency • reduce rework
∂Rq * ∂Rd
• increased customer
∂Rc * ∂Rw5. Process Capability Model to the Earned Value (7)
: ① elaboration of results chain assumption
- base of estimation
② value management
- progress/risk monitoring
: earned value extension
- change management
: reduce time to deliver product-delivery effectiveness
: reduce time to process
On GQM-Based questionnaire we defined quality and delivery for the external effectiveness, and cycle time and waste for the internal efficiency to performance pyramid as follows;
quality ::= Rq (defect management, process capability, SPI cost,
delivery ::= Rd (schedule, progress, marketing, time-to-market)
cycle ::= Rc (process capability, process improvement,
waste ::= Rw (productivity, reduce rework, reuse capability,
Now we propose the benefit maximize function as follows;
F (Rq, Rd, Rc, Rw) = 0.
To maximize benefit function for the external effectiveness,
and to maximize benefit function for the internal efficiency,
These condition links to assumptions, which condition the realization of outcomes on the benefit-realization approach chain.
For the assumptions we can propose the procedures for benefit realization as follows;
i,j =16. Benefits Realization Model and Conclusions (4)
I. To evaluate process capability and estimate project size,
II. To calculate investment cost
A0 : the existence project cost
A1 : cost for SPI using CBD Architecture
A1j : j = 1, n (SPI items), aij : i, j = 1, n (weight of each item)
A1 = Σ
project benefits – project cost
project cost6. Benefits Realization Model and Conclusions (5)
benefit realization = (%)
Benefit / cost ratio =
ROI = * 100(%)
GQM-Based questionnaire gives us potential indicators of customer satisfaction, process flexibility, and productivity. Using them we can evaluate the benefit realization to software process improvement with CBD architecture which identify process measure to the business indicators and relate the costs and benefits of SPI also to the business indicators.
GQM 기반 설문서는 고객에 대한 만족도, 프로세스의 유연성, 및 제품의 생산성을 측정하고 이를 위해서 투자된 비용의 효율을 평가 할 수 있는 데이터를 수집해 준다.
CBD 아키텍쳐에 의해서 GQM기반 설문서에 의해서 수집할 데이터를 조정하여 신뢰성 구간을 설정하고 SPI에 대한 현실적인 이익을 계산하는 것은 CBD 아키텍쳐를 사용한 소프트웨어 시스템의 효율을 평가하는 기초가 된다
비즈니스 버전에 맞추어서 CBD 아키텍쳐를 사용하여 개발한 소프트웨어 시스템의 Benefit Realization, Benefit/Cost Ratio, 및 ROI를 계산할 수 있는 모델은 CBD 아키텍쳐의 재사용을 권장할 수 있는 기준과 지침이 될수 있다.