Information Systems Overview

1. Information Systems Overview Organizing Complex Projects Around Risks Arising From System Dynamic Behavior Neil Siegel Sector Vice-President & Chief Engineer Northrop Grumman 9 March 2011

2. Agenda • Scope • Hypothesis • The design-based technique • Research results • Summary

3. Scope of proposed study This study aims to assess the efficacy of one particular method of improving the outcomes on complicated, large-scale, software-intensive system development projects

4. “Bottom-line up front”

5. Systems are important to society,but many system development efforts fail • What do we mean by “fail”: • Do not produce a product that meets the original specifications • Produce such a product only after taking significantly more time and/or money than originally expected. • In the extreme, many such projects are cancelled before completion • Failure is apparently common: • (Glass 2001) cites data indicating that only about 16% of the system development projects that he surveyed were listed as successful by their own developers • (Johnson 2006) cites data from the Standish Group, which describes results from a 2004 survey: just 29% of development projects succeeded

6. Scope of the systems of interest • Complex emergent behavior, as described by (Rechtin 1991) • Interactions with physical devices (physically-moving mechanisms, other time-sensitive mechanical devices, etc.) • Stressing asynchronous stimuli (such as extraordinary high data-ingest rates, or highly-stressed communications structures) • Extraordinarily high availability / reliability requirements • Development efforts of large size • Systems that need to display much early progress through prototyping and re-use

7. Hypotheses During the development phase of a large-scale, complex computer-based system, the use of a specific design-based technique that centralizes the control of the dynamic behavior of a system will lower the density of those defects that are attributable to unplanned adverse dynamic system behavior

8. Partitioning the work • I propose using the design process to partition the work into different “skill bins”, so as to provide better ways to assign people to tasks. • This allows a particular difficult task (control and management of the system’s dynamic behavior) to be partitioned away from most of the development team. • Under current methods, the “hard” parts of the work can be disseminated into a large portion of the tasks

9. How I accomplish that partitioningThe System Architecture Skeleton

10. Results

11. All 6 periods / all 4 cases

12. Defect density“Use of the design-based technique will reduce the density of defects attributable to errors in managing system dynamic behavior” Project YYYY, periods I-III contractor tests, attributable problem reports by month.

13. Cost performance also tracksthe dependent variable

14. Summary & interpretations

15. Summary & interpretations • The data indicate that the proposed design-based technique may in fact lead to better outcomes. • Indicated by a materially-lower density of defects (on the order of four times lower) that were attributable to errors in controlling system dynamic behavior

16. Summary & interpretations • The design-based technique considered herein only applies to projects where such centralization is possible • The study did demonstrate that this set of projects is not vanishingly small • This specific technique, however, is only one possible technique for creating a partitioning of a project into easy and difficult portions • Future studies could propose and assess other such techniques.

17. Implications for practice • Control structures may be more important than generally recognized • New goal for the design process

18. Questions?

19. NGIS clearance approval number: ISHQ-2011-0010