Requirements Volatility Workshop

1. Rick Selby USC Center for Systems & Software Engineering http://csse.usc.eduFebruary 15, 2007 Requirements Volatility Workshop

2. Attendee List

3. Attendee List

4. Workshop Guidelines Product: briefing, preferably with notes Topics should include: Most critical success factors in each area Current best practices for addressing them Areas for further research Rated 0-10 on value and difficulty of research

5. Requirements Critical Success Factors Technical (tech)Investigate req interdependencies and dependencies (tech)Demistify the notion that requirements are independent (tech)Requirements gathering-what are the important attributes that should be captured (tech)Want virtual model and understand attributes (tech and people)Better ways to prioritize requirements, considering risk, cost factors, stakeholder utility function and how to reconcile stakeholders interest (tech and people)Same as above with greater emphasis on change (tech)Specify requirements/not implementation; users often want to specify implementation (tech)What are the critical/driving requirements

6. Requirements Critical Success Factors (continued) • Technical • (tech)Relate requirements work back to COSOSIMO and COSYSMO • (tech)Missing rich enough vocabulary to model “this whole thing” (wants, needs, problems, issues, business drivers that are referred to as requirements) • (tech)Need modeling approach other than English • (tech/tool)How do we automatically measure requirements, requirement volatility, progress, scope • (tech)Accurately assess/predict impact of changes • (tech)Standard classification method to capture types of requirements and level of requirements • (tech)Classify requirements volatility types and define metrics to measure volatility • (tech)Address differences in req vol across market segments

7. Requirements Critical Success Factors (continued) • Technical • (tech)Function points necessary but not sufficient—miss technical performance and constraints • (tech)Capabilities versus requirements • (tools/tech)Tools: relate estimation inputs to the outputs (for example, inputs req lead to sloc output instead of sloc to effort) • (tech)Global collaboration—state requirements in a way that they can be understood globally and implemented locally • (tech)Specify the behavior for off-nominal conditions and events • (tech)Detect the confliction between requirements

8. Requirements Critical Success Factors (continued) • Tool • (tech/tool)How do we automatically measure requirements, requirement volatility, progress, scope • (process/tool)Requirements traceability from the beginning (who initiated and why, which application, which test) • (tools)Investigate tools available for requirements process (measurement, tracking) • (tools/tech)Tools: relate estimation inputs to the outputs (for example, inputs req lead to sloc output instead of sloc to effort) • (process/tools)Need a good way to move back and forth between reqs and models

9. Requirements Critical Success Factors (continued) • Product • (prod)Define requirements in a sufficient enough way to cost them, but high level enough to provide flexibility (R. Turner comment)

10. Requirements Critical Success Factors (continued) • Process • (process)Capture requirement up front using elicitation in a way that avoids future change • (process)Plan for requirements to change (evolutionary/incremental development) • (process)Late binding of requirements-need to specify in a way to promote late binding • (process)Involve all key stakeholders and ensure their accountability for the requirements • (people/process)Treat requirements process as a negotiation and learning process • (management/process)Minimize requirements volatility by managing decision volatility • (management/process)Link decisions and requirements • (process)Need to identify assumptions as well as requirements

11. Requirements Critical Success Factors (continued) • Process • (process)Be clear what is an assumption and what is a requirement • (people/process)Treat requirements process as a negotiation and learning process • (process)Standard on requirements volatility-how to measure, what to measure • (process/methodology)Look at process used to development requirements, especially across enterprises • Modeling and sim • Environment (process, tools, people) • Instrumentation • Increase propensity to accept changes in the requirements process • (process)Steal from agile, tests are one of the better forms for capturing requirements

12. Requirements Critical Success Factors (continued) • Process • (process/management)Get immediate feedback from development and test teams that requirements can be implemented and tested • (process/management)Write requirements in alignment/context with business drivers • (process/tool)Requirements traceability from the beginning (who initiated and why, which application, which test) • (process/management)Keeping the multiple stakeholders engaged across the life cycle • (process/management)Integrate measurement and modeling of requirements changes

13. Requirements Critical Success Factors (continued) • Process • (management/process)Improve/investigate methodology for requirements development by incorporation of meta models, semantics, and ontology • (process)Using value based systems engineering we need to identify the 10-20 key driving requirements for large scale systems • (people/process)Collaboration with customer to prevent later volatility (using analog of collaborative design in the requirements space)

14. Requirements Critical Success Factors (continued) • Process • (process)Moving from requirements to models too early leads to premature codification of solution(s) • (process/tools)Need a good way to move back and forth between reqs and models • (process)25% req change results in 100% change in the solution space • (process)Requirements completeness for context at various points in time

15. Requirements Critical Success Factors (continued) • People • (tech and people)Same as above with greater emphasis on change • (people) Committed, representative, available, collaborative, knowledeable (CRACK) stakeholders • (people/process)Treat requirements process as a negotiation and learning process • (people/process)Treat requirements process as a negotiation and learning process • (people)Make stakeholder incentives explicit • (people)Function point modeling to provide another way to look at requirements

16. Requirements Critical Success Factors (continued) • People • (people)Data modeling (part of FP modeling) to analyze requirements • (people/management)Change management and expectation management • (people/org)How do we convince customers to think in smaller steps • (people/org)Customers tend to think 5-10 yrs in the future, others tend to think 9 months ahead • (people/process)Collaboration with customer to prevent later volatility (using analog of collaborative design in the requirements space) • (people)Experience of team

17. People • (people)Diverse representation in the systems engineering organization: role, function, people, cultures • (people/org)Managing stakeholder continuity/volatility/high variance of req interpretation • (people)Ensure that you understand your customer (familiar with customer) • (people)Impact of cognitive makeup of req candidates using standard psych tests (Toyota evaluates critical thinking, problem solving, decision making—40 hours of testing prior to interviewing) • (people)UML never represent customer space, they could, but they don’t—the first one you see is taken as a design model (any time you make an abstract model, it is assumed by others that it is how you plan to design the solution)—how do you convince people that there are abstract models that are useful to define the problem without defining the solution and to treat them as such

18. Requirements Critical Success Factors (continued) • People • (people)People involved in eliciting and analyzing requirements • Well versed in domain and the trade space for solutions leads to more effectiveness in eliciting and analyzing requirements • (people)Having domain knowledge and experience in extracting requirements leads to

19. Requirements Critical Success Factors (continued) Management (management/process)Minimize requirements volatility by managing decision volatility (management/process)Link decisions and requirements (management)Use models to understand effect of requirements change on the process (people/management)Change management and expectation management (process/management)Get immediate feedback from development and test teams that requirements can be implemented and tested

20. Requirements Critical Success Factors (continued) • Management • (process/management)Write requirements in alignment/context with business drivers • (process/management)Keeping the multiple stakeholders engaged across the life cycle • (process/management)Integrate measurement and modeling of requirements changes • (management/process)Improve/investigate methodology for requirements development by incorporation of meta models, semantics, and ontology

21. Requirements Critical Success Factors (continued) Methodology (process/methodology)Look at process used to development requirements, especially across enterprises Modeling and sim Environment (process, tools, people) Instrumentation Increase propensity to accept changes in the requirements process

22. Requirements Critical Success Factors (continued) Organizational (people/org)How do we convince customers to think in smaller steps (people/org)Customers tend to think 5-10 yrs in the future, others tend to think 9 months ahead (people/org)Managing stakeholder continuity/volatility/high variance of req interpretation

23. Requirements Critical Success Factors (continued) • (citation)Software requirements: the Requirements Set (Ferdinandi) (about categories of requirements)

24. Research ProjectsVc • V: 1,7,2 D:2,7,2 Establish relationships between capabilities and requirements 3 • V: 5,6,2 D: 0,4,5 How to support client/customer in requirements elicitation 4 • for naïve/non-expert clients • when the elicitor does not have domain knowledge • V: 2,5,5 D: 0,10,0 Define and evaluation a taxonomy of knowledge needed in req elicitation 2 • A) V: 11,3,0 D: 0,8,6 Develop improved/standardized methods for measuring and evaluating reqs/reqs changes and evaluate the benefits on experimental projects 7, 22/14=1.57 • V: 7,7,0 D: 1,8,4 Investigate use of boundary objects for modeling requirements in collaborative teams 5 • V: 5,6,4 D: 0,5,9 Build tools to visualize and investigate the relationships between decisions and requirements, with emphasis on managing changes that are resulting from decision change 5

25. Research Projects (continued) • B)Define set of rules/metrics that can accurately predict/reflect 13 ; D: 0,4,13 21/17=1.24 • requirements quality • decision quality • Req traceability with code and design (bi-directional) • Requirements completion • Requirements omissions • Develop checklists/tools help one identify missing reqs 4 • Translate English requirements set into a virtual representation 4 • C)Develop knowledge management tool to categorize requirements and based on this, develop cost estimate/risk predictions 6; D: 0,4,11 19/15=1.27 • Make function point models into a useful form of boundary objects 1

26. Research Projects (continued) • D) Define a representation and accompanying new tools for reqs that enables automated measurement and analysis 6; D: 0,3,11 17/14=1.21 • Investigate team org and roles that are effective at reqs elicitation, analysis, pruning, and prioritization 5 • E) Extend USC COCOMO research to relate reqs as inputs to SLOC as output and verify on real projects 6; D:0,7,8 22/15=1.47 • Investigate levels of requirements and develop factors to estimate number of reqs at various levels (requirements normalization to get consistent count of reqs at a given level) 4 • Investigate the effects of communicating reqs across cultures, especially globally 3

27. Research Projects (continued) • Small to medium projects: does a very structured process have an impact on 5 • Req quality and req volatility • quality and cost of end product • Define and develop helper for ensuring that behavior for undesirable events and off-nominal conditions are captured in the reqs 1 • Tool to identify potential emergent behaviors resulting from requirements 3 • F) Develop an approach for costing by capabilities instead of requirements 7; D:0,6,8 20/14=1.42 • Want to build a system “like that one” – reqs def by analogy • Investigate feasibility of function points as an alternative to COSYSMO reqs input 2 • Define/develop Tools to identify “similar requirements” to enable detect redundancy, conflicts, and opportunities for reuse 5

28. Research Projects (continued) • Compare contribution of people vs. process to the quality of the developed requirements 3 • Define and develop a way to measure the quality of developed requirements 3 • G) Develop meta model, semantics (categorization of terms, naming conventions, nomenclature) and ontologies of requirements 7; D:0,8,6 22/14=1.57 • Investigate the efficiencies of various requirements derivation techniques for deriving high quality/complete requirements 3 • H) Define the minimum required steps for lean requirements-related processes 7; D:4,9,2 32/15=2.13 • Small projects • Large projects

29. Research Projects (continued) • Develop tool to capture requirements associated with selected/reused/COTS components 1 • Identify preferred methodology for incorporation of legacy requirements into new projects 2 • Investigate the extent to which we can simulate the user’s problem space (virtual model)—executable model 3 • I) Define a mechanism for capturing reqs using test procedures (test-centric requirements development) 6; D: 0,12,1 25/13=1.92 • Define more specific completeness criteria for passing LCO/LCA milestones 4 • Investigate continuous milestones with continuous/tightened criteria • Identify and do good write-ups of case studies on outstanding requirements and processes in industry. 5 • Identify best practices for reqs development 4 • Identify preferred approach for seamless incorporation of se and sw eng life cycles, with emphasis on software life cycle within se life cycles 3

30. Research Projects (continued) • Define and develop an incremental sys eng life cycle model 3 • J) Develop a model to predict reqs volatility 14; D: 0,3,13 19/16=1.19 • Based on a given set of reqs, problem domain, and stakeholders • With emphasis on the velocity with which reqs were developed, changes made, defects injected and defected • Capture the types and provide predictors for latent req defects 1

31. Top 10 Research Project • Develop a model to predict reqs volatility 14 • Based on a given set of reqs, problem domain, and stakeholders • With emphasis on the velocity with which reqs were developed, changes made, defects injected and defected • Define set of rules/metrics that can accurately predict/reflect 13 • requirements quality • decision quality • Req traceability with code and design (bi-directional) • Requirements completion • Requirements omissions • V: 11,3,0 D: 0,8,6 Develop improved/standardized methods for measuring and evaluating reqs/reqs changes and evaluate the benefits on experimental projects 7 • Develop knowledge management tool to categorize requirements and based on this, develop cost estimate/risk predictions 6

32. Best Practices: • What are the current best practices for addressing risk related to requirements volatility • Postponement of requirements • Iterative requirements development • Clarity of requirements • Prioritize requirements • Prioritize by feasibility • More agile process • Capture justification and history of requirements • Architect product to withstand change • Better standards for requirements documentation • Agile prototyping • Involve many stakeholders, especially downstream customers • Hierarchical classifications (Layered—577 MBASE)

33. Best Practices (continued) • Prioritize early and at high levels on the requirements hierarchy • Change propagation and traceability • Requirements traceability • Over-engineer (just in case) and create a resilient architecture that anticipates change • Visible and understandable reqs • Measure requirements and volatility • Assess and communicate impact to all stakeholders • Training • Processes • Domain • Checklists • Make explicit win conditions and incentives of stakeholders • Triage requirements—if some reqs change, it does not matter, if others change, it matters a lot

34. Best Practices • ROI analysis for alternative requirements • Use tools • Rely on standards to a larger extent • Conduct sensitivity analysis up front • Categorize requirements, capabilities, project, performance, interface, evolution, future

35. Ease / Value

36. Ease / Value Chart