“Learn as you go” Systems Engineering A Methodology for System Assembly

1. “Learn as you go” Systems Engineering A Methodology for System Assembly University of Virginia Barry M. Horowitz James H. Lambert

2. Component Design & Development Technology Assessment Business Value Analysis Component Integration Elements for Orchestration on aDevelopment Intensive Project Dominates Cost & Schedule

3. OTS Component Integration COTS Product Assessments Business Experimentation Elements for Orchestration on a System Assembly Project No Dominant Part

4. Comparison of Factors for System Engineering Methodology

5. Drivers of System Assembly • IT integration standards and the Internet • Low cost COTS • Product-based integration (using EAI technology) with legacy systems • Already trained user population • Low-cost initial system demo

6. Current Methodology • Create System Architecture based on legacy, standards, desired applications, and available products, with emphasis on integration and middleware • Order of integration is driven by functional needs • Assemble sub-systems from available components • Evaluate through testing • Learn and improve as you go

7. Gartner Group Prediction(September 2000) Over the next four years, 1/3 of the complex assembly projects will suffer cost/schedule overruns of 30-50%.

8. “Learn as you go” System Engineering Methodology

9. Business Requirements Integration problems Capacity limits Performance limits Out-of norm responses Security shortfalls System Assembly Requires Learning

10. Organized Learning • Requirements uncertainty • Precedented vs. unprecedented integration tasks • Managing risk via scheduled learning

11. Delayed Business Value Early Learning Latent Cost/Schedule Risks Early Business Value Balancing Learning Rate& Early System Value

12. An Approach for Managing OTS Integration Learning Rate • Enumerate product adaptation and integration tasks • Based on complexity and precedence, set cost/time risk levels for each task • Establish potential sequences (Plans) of integration/test/roll-out • Functional capability • Expected business value • Expected cost and schedule • Determine semi-std deviation of cost/schedule to calibrate risk • Select best combination • Replan as you learn

13. Metrics for Comparing Plans • Cost($), Schedule(Mo) • Remaining Cost & Schedule Risk vs. Time • Ratio of Remaining Cost Risk to Sunk Cost vs Time • Ratio of Remaining Schedule Risk to Sunk Cost vs Time • % of Final Operational Value Achieved vs Time • Uncertainty of % of Final Operational Value Achieved vs Time

14. Required Inputs for Comparing Plans • Components to be Assembled • Sequence and Schedule for Assembly • Adaptation & Preliminary Integration • Final Integration and Test • Operation • Expected Cost for each Operational Milestone • Overall Project Cost and Schedule Risk • Uncertainty in Cost/Schedule to Complete the Project after each Operational Milestone • Uncertainty in % of Final Operational Value for Future Milestones (Use 3 values to represent range-low,middle, high)

15. Gathering Inputs • Based on judgements of experts • Inputs gathered from a variety of sources • Technical • Operational • Value Managers • Project Managers • Systems Engineers • Inputs adjusted based on precedence considerations and external information on related efforts • Redone after each milestone is achieved to account for learning • New plans, components and sequences are considered at each milestone to account for actual results, learning, new technology, budget changes,etc.

16. Plan Evaluation • Requires skilled people in all relevant area • Enterprise Value • Operations • Technical • Project Management • Requires people who know the precedence of activities as well as the substance of the required effort • Requires a presentation of metrics that is compact and understandable to all

17. Wireless Internet Integration Wireless Device Assessments m-Commerce Services Elements of a Specific System Assembly Example

18. An Example m-Commerce Shopper’s Helper

19. Shopper’s Wireless Helper Web Server WAP Gateway Internet DBMS • FUNCTIONS • Price Comparison • Brand Comparison • Purchase (Secure) • Locate (GPS) OLAP

20. Constrained Bandwidth Performance Display New Standards WAP WTLS WML Very New Security Technology Entrust Verisign RSA Certicom Wireless Internet Integration & Security

21. Two Competing Plans • To simplify the example, assume three components for assembly: • Comparison Shopping (A) • GPS Add-on (B) • On-line Purchasing (C) • Two alternative sequences are of interest • A, AB, ABC • C, AC, ABC

22. Plan 1 Build on Current Customer Base Move to On-line Shopping after initial experience GPS an Accommodation to Customers Plan 2 Move to New Customer Base Go for The Early Revenue Gather Existing Customers later Comparison of Two Plans Higher Risk, Faster Payoff Go Slow and Manage Risk

37. Summary • System Engineering efforts for assembling a system requires a different emphasis than for developing a system • Learning by doing is fundamental and learning can be managed • “Learn as you go” is an approach for : • Selecting and Evaluating competing sequences of system assembly • Using specific metrics and compact forms for information and analysis presentation when comparing plans • Accounting for Precedence • Adjusting plans based on learning