An Empirical Investigation of the Key Factors for Success in Software Process Improvement

1. An Empirical Investigation of the Key Factors for Success in Software Process Improvement Written by Tore Dyba Presented by Ayşegül Tüysüz, Meltem Yıldırım CmpE-550 / Advanced Topics in Software Engineering 14/12/2005

2. Outline • Motivation • Conceptual Research Model • Questionnaire Experiment • Results • Bivariate Correlational Analysis • Multiple Regression Analysis • Discussion • Critical Evaluation

3. Motivation • Present Models for succeeding with SPI (Quality Improvement Paradigm (QIP), IDEAL Model, ami method, SPICE) • Successful Companies (Alcatel, HP, Motorola, NASA, Philips, Siemens) • SPI research efforts are limited and lack adequate theoretical and psychometric justification • The paper • extends and integrates models from prior research • helps us to understand the influence of organizational issues on SPI success

4. Independent Variables • Business Orientation • Involved Leadership • Employee Participation • Concern for Measurement • Exploitation of Existing Knowledge • Exploration of New Knowledge • Dependent Variable • SPI Success • Perceived Level of Success • Organizational Performance Learning Strategy • Moderating Variables • Organizational Size • Environmental Conditions Conceptual Research Model Extent to which SPI goals and actions are aligned with explicit and implicit business goals and strategies Extent to which leaders at all levels in the organization are genuinely committed to and actively participate in SPI Extent to which employees use their knowledge and experience to dedicate, act, and take responsibility for SPI Extent to which SW organization collects and utilizes quality data to guide and assess the effects of SPI activities Cost reduction, cycle-time reduction, customer satisfaction • SPI success is positively associated with each individual independent variable • The joint contribution of the six independent variables explain a large amount of the variance in SPI success. Contextual factors

5. Questionnaire Experiment • 120 software and quality managers from 55 companies • Independent and Dependent Variables: • multi-item, 5-point, bipolar scales • Moderating Variables: • Organizational size = number of SW developers • Environmental conditions: 7-point, bipolar scale Strongly Disagree □1 □2 □3 □4 □5 Strongly Agree Stable □1 □2 □3 □4 □5 □6 □7 Unstable Predictable □1 □2 □3 □4 □5 □6 □7 Unpredictable

6. Questionnaire • 36 questions to measure 6 factors of SPI success • 5 questions to measure performance construct 1. Business Orientation There is a braod understanding of SPI goals and policy within our organization 4. Concern for Measurement We regularly collect quality data (e.g. defects, timeliness) from our projects 2. Leadership Involvement Management is actively participating in SPI activities 3. Employee Participation Software developers have responsibility related to the organization’s SPI activities 5. Exploitation of Existing Knowledge We collect and classify experience from prior projects 6. Exploration of New Knowledge We often carry out trials with new software engineering methods and tools • Perceived level of success • Our SPI work has • increased our software engineering competence • improved our overall performance • Organizational Performance • Over the past three years, we have greatly reduced • the cost of software development • the cycle time of software development

7. Results of Bivariate Correlational Analysis • Correlations among independent variables

8. Contextual factors are held constant Results of Bivariate Correlational Analysis • Testing Individual Relationships

9. Results of Multiple Regression Analysis • Testing Joint Contribution of Independent Variables • Strong support for effect of variables 1, 3, 4 and 5 from both analyses • Partial support for effect of variables 2 and 6

10. Discussion • Business Orientation, strongest influence on SPI success • Involved Leadership • insignificant in predicting SPI success • many SPI initiatives do not require management commitment beyond obtaining the resources needed • Employee Participation • strongest influence • people tend to support what they have participated in creating • Exploration of New Knowledge • insignificant in predicting SPI success • balance between exploitation and exploration • The results hold regardless of the contextual factors

11. Our Critical Evaluation • Too many repetitions and unnecessarily long • Several levels of analysis are possible (individual, group, process, project, organization) • Contextual factors • Guerrero, F., Eterovic, Y., “Adopting the SW-CMM in a Small IT Organization”, 2004 • Not enough factors are considered (project size, budget, educational level of developers) • The importance of environmental conditions is neglected • Training, employee participation, frequency of process assessment depend on environmental conditions

12. Questions