a study of the causes of requirements volatility and its impact on systems engineering effort n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
A study of the Causes of Requirements Volatility and its Impact on Systems Engineering Effort PowerPoint Presentation
Download Presentation
A study of the Causes of Requirements Volatility and its Impact on Systems Engineering Effort

Loading in 2 Seconds...

play fullscreen
1 / 17

A study of the Causes of Requirements Volatility and its Impact on Systems Engineering Effort - PowerPoint PPT Presentation


  • 344 Views
  • Uploaded on

A study of the Causes of Requirements Volatility and its Impact on Systems Engineering Effort. COSYSMO Workshop Center for Software and Systems Engineering, Annual Research Review March 11, 2010 Mauricio E. Peña – Ph.D. Student, USC Industrial and Systems Engineering Department. Motivation.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'A study of the Causes of Requirements Volatility and its Impact on Systems Engineering Effort' - billie


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
a study of the causes of requirements volatility and its impact on systems engineering effort

A study of the Causes of Requirements Volatility and its Impact on Systems Engineering Effort

COSYSMO WorkshopCenter for Software and Systems Engineering, Annual Research ReviewMarch 11, 2010

Mauricio E. Peña – Ph.D. Student, USC Industrial and SystemsEngineering Department

motivation
Motivation

“Requirements are the foundation of the project. They form the basis for design, manufacture, test and operations….changes in requirements later in the development cycle can have a significant cost impact, possibly resulting in cancellation.”

INCOSE Systems Engineering Handbook (2006)

objectives
Objectives
  • Provide and overview of requirements volatility and its importance to engineering projects
  • Introduce the research questions & research plan
  • Discuss the state of the art and implications to COSYSMO
  • Solicit feedback:
    • On a requirements volatility causal model
    • On a requirements volatility survey
requirements volatility overview
Requirements Volatility Overview

Requirements volatility is the change in requirements (added, deleted, and modified) over a given time interval [MIL-STD-498, 1994, Ferreira, 2002]

Also known as:

Requirements creep: An increase in scope and number of system requirements

Requirements churn: Instability in the requirements set – requirements are modified or re-worked without necessarily resulting in an increase in the total number of requirements

CDR

[Hammer et al, 1998]

importance of understanding requirements volatility
Importance of Understanding Requirements Volatility
  • Requirements volatility has been identified by numerous research studies as a risk factor and cost-driver of systems engineering projects [Ferreira 2002, Boehm 1991]
  • Requirements changes are costly, particularly in the later stages of the lifecycle process because the change may require rework of the design, verification and deployment plans [Kotonya and Sommerville, 1995]
  • Requirements volatility trends are considered leading indicators of project performance as defined by the Lean Advancement Initiative Systems Engineering Leading Indicators Guide (2007)
  • The Government Accountability Office (GAO) concluded in a 2004 report on the DoD’s acquisition of software-intensive weapons systems that missing, vague, or changing requirements are a major cause of project failure
research questions
Research Questions
  • Research Question 1: What causes requirements volatility?
  • Research Question 2: How does requirements volatility impact systems engineering effort?
research plan
Research Plan

Completed

In-progress

Planned

Literature

Review

Tested Hypothesis

Observations

Statistical

Analysis

Pilot RV survey

Causal Model

Industry Data

COSYSMO Workshop Discussion

Updated Causal Model (hypothesis)

Delphi

Survey

Delphi

Survey

RV survey

Considerations for requirements volatility

LAI Knowledge

Exchange

Survey

Results

literature background
Literature Background
  • Most of the requirements volatility research to date has been focused on software systems
  • Various research methods have been utilized to investigate the causes and effects of requirement volatility – a methodological breakdown of the studies reviewed to date is below
  • However, there still a lack of empirical data to determine the quantitative impact of requirements volatility on systems engineering effort for a broader base of engineering projects
implications to cosysmo
Implications to COSYSMO
  • Volatility was identified as a size driver adjustment factor in COSYSMO [Valerdi, 2005]
  • It is currently a placeholder in the model

Reuse

Categories

Volatility Factor

# Requirements

# Interfaces

# Scenarios

# Algorithms

Size

Drivers

COSYSMO

Effort

Effort

Multipliers

  • Application factors
    • 8 factors
  • Team factors
    • 6 factors

Calibration

[Fortune, 2009]

impact of volatility after the requirements have been baselined
Impact of volatility after the requirements have been baselined

PROJECT WITH VOLATILITY

PROJECT WITH LOW VOLATILITY

# Req.

# Req.

Planned # Req

Actual # Req

Planned # Req

Actual # Req

SRR

SRR

SE Hours

SE Hours

Delta SE effort

Delta SE effort

SRR

SRR

observations
Observations
  • Requirements volatility is correlated with an increase in the size drivers of systems engineering effort [Valerdi, 2005; Houston,2000]
  • Requirements Volatility is expected during the early stages of a project (conceptualize / requirements phase). It becomes a concern when it ocurrs after the requirements phase is complete because it is likely to result in re-work of engineering products [Kotonya and Sommerville (1995); Rosenberg and Hyatt (1996); Ferreira, (2002)]
  • Requirements volatility is correlated to increases in cost, schedule duration and re-work [Ferreira et al, 2009; Zowghi and Nurmuliani, 2002]
  • Requirements volatility may have a compounding effect on systems engineering effort because the changes may not only cause rework but also trigger increases in the number of lower-level requirements [Kulk and Verhoef, 2008]
  • The use of volatility metrics and thresholds can help determine the adequacy of SE processes and staff and trigger corrective actions to mitigate further volatility (Roedler and Rhodes, 2007)
  • Requirements volatility is caused by a number of factors that are both external (e.g. environmental, customer priority changes) and internal (process capability, organizational policies)

Based on the review of the literature a causal model was developed to investigate both research questions

slide12

Causal Model (normative)

Case 1 research question: what causes requirements volatility?

Contextual /

Environmental

changes

Changes in org. structure and process

Technology Maturity

Requirements. Re-use

SE Process Maturity

Experienced staff

-

(Ferreira, 2002)

(GAO, 2004)

(GAO, 2004)

-

-

(Jones 1994)

(Curtis et al. 1988)

Zowghi et al. (2000)

-

(Ferreira 2002)

(Ferreira 2002)

+

+

(Kotonya and Sommerville 1998)

Customer-Driven

Changes

+

(Jones 1994)

(Curtis et al. 1988)

(Kotonya and Sommerville 1998)

(Valerdi 2005)

Requirements volatility

Number of Sys Requirements

+/-

SE Productivity

+/-

(Houston, 2000)

Zowghi et al. (2000)

(Ferreira, et al 2009)

+

(Ferreira, 2002)

(Ferreira, et al 2009)

(Valerdi 2005)

(Houston, 2000)

(Ferreira 2002)

(Charette et al,2003)

+

+/-

+/-

(Valerdi 2005)

(Charette et al,2003)

SE Project Effort

Project Schedule

+

Quality

Re-work

-

+/-

(Ferreira, et al 2009)

(Houston, 2000)

(Ferreira 2002)

(Charette et al,2003)

+/-

+

(Valerdi 2005)

+

+/-

Project Cost

SE Cost

(Valerdi 2005)

-

-

Customer Satisfaction

Dependent Variable

Independent Variable

slide13

Causal Model (normative)

Case 2 research question: how does requirements volatility impact systems engineering effort?

Contextual /

Environmental

changes

Changes in org. structure and process

Technology Maturity

Requirements. Re-use

SE Process Maturity

Experienced staff

-

(Ferreira, 2002)

(GAO, 2004)

(GAO, 2004)

-

-

(Jones 1994)

(Curtis et al. 1988)

Zowghi et al. (2000)

-

(Ferreira 2002)

(Ferreira 2002)

+

+

(Kotonya and Sommerville 1998)

Customer-Driven

Changes

+

(Jones 1994)

(Curtis et al. 1988)

(Kotonya and Sommerville 1998)

(Valerdi 2005)

Requirements volatility

Number of Sys Requirements

+/-

SE Productivity

+/-

(Houston, 2000)

Zowghi et al. (2000)

(Ferreira, et al 2009)

+

(Ferreira, 2002)

(Ferreira, et al 2009)

(Valerdi 2005)

(Houston, 2000)

(Ferreira 2002)

(Charette et al,2003)

+

+/-

+/-

(Valerdi 2005)

(Charette et al,2003)

SE Project Effort

Project Schedule

+

Quality

Re-work

-

+/-

(Ferreira, et al 2009)

(Houston, 2000)

(Ferreira 2002)

(Charette et al,2003)

+/-

+

(Valerdi 2005)

+

+/-

Project Cost

SE Cost

(Valerdi 2005)

-

-

Customer Satisfaction

Dependent Variable

Independent Variable

slide14

Moderating impact of expected volatility / thresholds

Org Improvement Actions

+

+

+/-

SE Process Maturity

Experienced staff

(Ferreira, 2002)

(GAO, 2004)

-

(Ferreira 2002)

-

Volatility Metrics /

Thresholds

Requirements volatility

+/-

(Houston, 2000)

Zowghi et al. (2000)

(Ferreira, et al 2009)

+/-

(Kulk and Verhoef 2008

Zowghi and Nurmuliani (1998)

(Kulk and Verhoef 2008)

Number of Requirements

+/-

+/-

+/-

(Valerdi 2005)

(Ferreira, et al 2009)

SE Project Effort

Project Schedule

+/-

+/-

(Valerdi 2005)

SE Cost

Dependent Variable

Independent Variable

Moderating Variable

questions for discussion
Questions for discussion
  • Are there other important causes of volatility missing in the model?
  • In what cases is the relationship between requirements volatility and # of systems requirements a positive one, and in what cases is it a negative one?
  • Should the impact of requirements volatility on the # of system requirements be adjusted based on the criticality/coupling of the requirements that are added/modified/deleted?
  • Does volatility have a direct impact on productivity?
  • Should volatility thresholds vary depending on the size and duration of a project?
  • How does the lifecycle phase affect the expected level of volatility?
  • Feedback on the survey -
next steps
Next Steps
  • Utilize recommendations/suggestions from this workshop to update the causal model and survey
  • Administer the survey to LAI workshop participants
  • Need to develop an approach to generate cost driver weight factors based on requirements volatility
  • Start the industry outreach to reach agreement on data-sharing
references
References
  • Boehm, B. (1991). Software Risk Management: Principles and Practices. IEEE Software 8, 1. Pp 32-41.
  • Ferreira, S., Collofello, J., Shunk, D., and Mackulak, G. (2009). “Understanding the effects of requirements volatility in software engineering by using analytical modeling and software process simulation.” The Journal of Systems and Software. Vol. 82, pp 1568-1577.
  • Fortune, J. (2009). Estimating systems engineering reuse with the constructive systems engineering cost model (COSYSMO 2.0). Doctoral Dissertation. University of Southern California, Industrial and Systems Engineering Department.
  • GAO-04-393 (2004). Report to the Committee on Armed
  • Services, U.S. Senate. Defense Acquisitions. Stronger Management Practices Are Needed to Improve DOD’s Software-Intensive Weapon Acquisitions
  • Hammer, T., Huffman, L., and Rosenberg, L. (1998). “Doing requirements right the first time.” Crosstalk, the Journal of Defense Software Engineering. Pp 20-25.
  • Houston, Dan X. (2000). A Software Project Simulation Model for Risk Management, Ph.D. Dissertation, Arizona State University
  • INCOSE Systems Engineering Handbook, Version 3, INCOSE, June 2006
  • ISO/IEC (2008). ISO/IEC 15288:2008 (E) Systems Engineering - System Life Cycle Processes.
  • Kotonya, G., Sommerville, I., (1998). Requirements Engineering: Processes and Techniques. John Wiley and Sons, Ltd.
  • Kulk, G, and Verhoef, C. (2008). “Quantifying requirements volatility effects”. Science of Computer Programming. Vol 72. pp 136–175
  • MIL-STD-498. 1994. Software Development and Documentation. U.S. Department of Defense.
  • Roedler, G. and Rhodes, D. (2007). Systems engineering leading indicators guide. Version 1. Massachusetts Institute of Technology, INCOSE, and PSM.
  • Rosenberg, L. and Hyatt L., (1996). A Software Quality Model and Metrics for Identifying Project Risks and Assessing Software Quality. (retrieved from http://satc.gsfc.nasa.gov/support/STC_APR96/qualtiy/stc_qual.html)
  • Schunn, C. (2008) Engineering Educational Design. Educational Designer, 1(1)
  • Valerdi, R. (2005). The constructive systems engineering cost model (COSYSMO). Doctoral Dissertation. University of Southern California, Industrial and Systems Engineering Department.
  • Zowghi, D. and Nurmuliani, N. (2002). A Study of the Impact of Requirements Volatility on Software Project Performance. Proceedings of the Ninth Asia-Pacific Software Engineering Conference