Strategic Engineering

1. Version 2 Designing Systems for an Uncertain Future Strategic Engineering Change Propagation Analysis in Complex Systems Olivier L. de Weck, Ph.D. deweck@mit.edu Associate Professor of Aeronautics and Astronautics and Engineering Systems October 7, 2008

2. Strategic Engineering – “the big picture” technology http://strategic.mit.edu markets regulations flexibility real options commonality platforms System Architecture concept Design for Changeability Design for Commonality Integrated Modeling and Simulation Spatial Dimension Temporal Dimension performance, cost, risk changes standardization Multidisciplinary Design Optimization uncertainty variety “optimal” design x* at t=to more than one variant of the system is needed: x1*, x2, … xn at t=to+Dt requirements change and x* is no longer optimal

3. F/A-18 Center Barrel Section Y488 Y470.5 Wing Attachment Y453 74A324001

4. Manufacturing Processes Changed Original Change Fuselage Stiffened Flight Control Software Changed Center of Gravity Shifted Gross Takeoff Weight Increased F/A-18 Complex System Change F/A-18 System Level Drawing

5. Change Propagation Analysis in Complex Systems Giffin M., de Weck O., Bounova G., Keller R., Eckert C., Clarkson J., “Change Propagation Analysis in Complex Technical Systems”, DETC2007-34652, ASME 2007 Design Engineering Technical Conferences, DETC2007-34871, Las Vegas, NV, September 4-7, 2007 In Press: ASME Journal of Mechanical Design Sponsor: Raytheon Integrated Defense Systems

6. Complex Sensor System Complex sensor system, complex hardware, software, human operators Derivative of earlier system 9 Year development 46 Areas (“Subsystems”) Hardware Software Program Documentation System Map (graph) Interconnections between areas System Description

7. Change Request Database technical, managerial, procedural track parent, child, siblings by areas with unique ID number chronologically numbered IDs Data Mining Procedure Export from DBMS to text file Written into MySQL database with Perl scripts Equivalent to a MS Word document with 120,000 pages Sorting, Filtering, Anonymizing Write simplified change request format (see right side) Data Set Typical Change Request

8. Apply Graph Theory to extract networks of connected changes parent-child changes sibling changes Most changes are only loosely connected 2-10 related changes Some large networks emerged Question: do these networks emerge from a single initial change? Change Networks

9. Change Propagation Network Network plot of largest change network in the dataset, with 2579 associated change requests.

10. Mapping Changes to affected subsystem areas 30143 27585 28213 28187 28007 30344 28166 28122 27027 28153 28695 28567 28788 28790 29538 30614 29547 23942 29399 28846 27627 30465 28878 23945 28531 26333 30148 28528 27656 28428 26331 23992 29711 28009 28186 27169 24980 23729 28067 23922 27023 30771 23024 32289 28821 28529 30126 29826 23821 29226 29353 30548 29731 System Network Map 30466 30501 31471 23925 24781 30503 29227 28601 28162 29744 28696 23831 8000 22850 25481 31973 25053 24659 31972 27952 24927 25476 25515 32645 31966 12156 22946 24926 31235 27592 25463 31967 26117 13320 Change Propagation Network

11. Change Propagation Index (CPI) change propagation probability • Classify each area • Absorber, Carrier, Multiplier total completed changes in Area j instigating area DDSM Change Propagation Frequency receiving area A change in Area 1 caused changes in Area 6 with a frequency of 4.17%. -1 <= CPI <= +1

12. System Area Classification CPI Spectrum • Areas found to be strong multipliers • 16: hardware performance evaluation • 25: hardware functional evaluation • 5: core data processing logic • 32: system evaluation tools • 19: common software services • 3: graphical user interface (GUI) • Areas found to be perfect reflectors • 27, 41: look like perfect absorbers • but actually zero changes implemented • despite numerous changes proposed • = perfect reflectors

13. Discovered new change pattern: “inverted ripple” system integration and test bug fixes [Eckert, Clarkson 2004] subsystem design major milestones or management changes component design Change Request Generation Change Requests Written per Month 1500 1200 900 Number Written 600 300 0 1 5 9 77 81 85 89 93 73 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 Month

14. Insights • Inverse relationship between change magnitude and frequency of occurrence • Large changes are infrequent, small ones are ubiquitous • Many change requests are never implemented • Some are rejected, others are ignored (~ 50%) • Changes may form complex networks over time. • Most are small (<10 changes), a few large ones exist (beware of these !) • Change networks form through coalescence and not necessarily through multi-step causal change propagation • Changes can propagate between areas that are not direct neighbors in the system DSM (not shown here, but we found this is so) • Subsystems can be classified as: • Multipliers CPI > ~0.3 • Carriers -0.1<CPI<1.0 • Absorbers CPI<-0.3 • Reflectors of Change CRI>CAI • Acceptors of Change CAI>CRI • Analysis of change database revealed that • Real world change processes more complex than expected • Industry data tends to be “noisy” • Potential for deriving change impact and likelihood for future projects

15. Future Work • Change Prediction: • How good are our predictions regarding actual versus planned effort? • How can change propagation patterns observed on past projects be leveraged for future design decisions (e.g. modularity, flexibility) • Data Processing: • Standardize methods for recording and processing data, tracing large change networks in greater depth- attempt to reconstruct logic • Staffing and Organization: • Analyze effects of staffing on changes and components • Patterns based on which personnel/organization work on the changes? • Contractual: • Can change propagation be used to write better prime and sub-contracts? • Statistical: • Are there critical numbers for change propagation? Limits on the number of propagation steps? . CMI-Sponsored Workshop on Engineering Change MIT Endicott House, October 30-31, 2008 ~ 12 firms from various industries (aerospace, auto, printing, construction)

16. Cambridge-MIT-Institute (CMI)Engineering Change Twin Workshops Trinity Hall College, UK University of Cambridge April 7-8, 2008 MIT Endicott House, USA October 29-31, 2008

17. Reasons for Change • Problems discovered during production and operations in the field such as retrofits, recalls ….(melioration) • Customization of product variants for different customers and market segments (globalization) • Infusion of new technologies during product refreshes or major “block” upgrades (innovation) • Cost reduction Initiatives, response to new features introduced by other firms (competition) • New government regulations (e.g. fuel economy standards, no lead in electronics …(compliance) • Others ….?

18. Workshop Goals • Obtain multi-faceted industry perspective on state-of-the art in engineering change practice • Present academic perspective and recent research advances to industry • Establish a research agenda for the next 5 years • Put in place basis for Special Issue of RED* • Stimulate interest in follow-up collaboration • Establish user community for advanced engineering change methods and tools * Research in Engineering Design (RED) Journal

19. UK Rolls Royce (A/C Engines)* Perkins (Diesel)* Volvo (Trucks, Engines)* BAE Systems (Defense)* Bosch (Auto Supplier)* BMW (Cars)* BP (Oil & Gas)* MAN Roland (Printing Systems) Arup (Construction) US Xerox (Printing Systems) Ford, GM (Cars and Trucks) Agusta Westland (Helicopters) Boeing (Aircraft) General Mills (Food) Fluor (Construction) Mack (Highway Trucks) Gerber (Textile Machines) NASA (Spacecraft) Raytheon (Defense Systems) Ventana Systems (S/W) Aberdeen Group United Technologies Corp. Invited Companies *attended April 2008

20. Strategic Engineering • Strategic Engineering is the process of designing systems and products in a way that deliberately accounts for customization and future uncertainties such that their lifecycle value is maximized.