Challenges for Systems Architects: Technical and Otherwise

1. Challenges for Systems Architects: Technical and Otherwise Mark W. Maier, Ph.D. Distinguished Engineer The Aerospace Corporation 703-324-8879 mark.w.maier@aero.org

2. Guide to the Challenges • Origin • Systems architecture is a core element of the Systems Design and Management Program • So, is what we know, and teach, sufficient to make you all successful systems architects? • It isn’t, Lets consider the challenges • Aligning technical processes and market realities • How much requirements analysis is the “right” amount? • How much process rigor is the “right” amount? • Modeling beyond what our current methods really support • Creating multi-view methods and supporting semi-formalized transformations • Modeling abstractions for layered systems • Aligning “Strategic Identity” and architecture, especially when one or the other is unclear • Flexibility, is it worth anything if we don’t use it?

3. Development Practices: Origin • A medium size company was concerned about the cost of Engineering Change Orders (ECO’s) • The company is medium size, manufactures telecom devices, production runs of thousands, large customer base, heavy international competition • ECO’s issued to fix problems found in field use • Basic Question • Should they be fixing their process, and where should the process be fixed? • Reflexive Answer: • ECO’s are bad. The cost of defect removal rises by 2-10x for each project phase that passes after its insertion. Field-discovered defects could be removed at far less cost by getting the requirements right and having more rigorous engineering processes • But… • Is this always true? • More general question, how does the value of time effect archtectures?

4. Background And Analysis • Products are developed very rapidly because of large price premiums in being first to market • Many ECO’s are due to incompatibility with other vendors equipment • Company attitude is “Any incompatibility is our problem” • Complexity of field environment is very high, many products are not precisely standards compliant • Hard to comprehensively duplicate in a laboratory • Fixes often are in ROM, go into new production, effected customers get new production units

5. The Observations • Most of the problems were from interactions with the external world • Internal quality control is pretty good • Known interaction problems usually allowed for in design (though imperfectly) • Better requirements could have prevented a significant fraction of ECO’s, if they captured the “real-world” interoperability requirements • But, in the real-world this is hard, and expensive. How do you capture the real interoperability requirements for the corpus of deployed international telecommunications equipment? • Most fixes were a change to ROM in ongoing production • A few recalls, but mostly just shipping a replacement unit to the individual effected customer

6. A Larger Interpretation • Understanding the implications requires thinking of all the quality and requirements costs • Discovery cost • The cost of finding out that the defect is present • Removal cost • The cost of removing the defect from the product, including retrofits and recovering from impacts • Opportunity cost • What you lose because of time and effort you spend in discovery and removal (including for things not found) • Maximizing profit is not the same as minimizing cost • Minimizing removal cost is not the same as minimizing total cost

7. Opportunity Cost Version One • Suppose you are in a market where your product price follows the curve p=2-t/t, where t is the time from first introduction • For simplicity, let sales rate be constant (which will only understate the effect to come) • Then, you realize 1/2 of your total revenue potential in the first t days from introduction, 1/4 in the next t, and so on t may be 100 days in some markets Photo courtesy of http://gallery.hd.org “We are under constant pressure to deliver new process equipment, even when buggy, because the benefits of being first to market are so large” -Semi-conductor equipment person in one of my classes

8. Opportunity Cost Version Two • In other markets (e.g. space) failures are often catastrophic, and the opportunity cost is the entire program • Put another way, the defect removal cost for a field discovered defect is nearly the entire program cost • Under these conditions very strict attention to early defect removal is called for, and is economically justified • Note similar case for commercial products where field defect discovery results in a large recall and/or very bad public relations Photograph courtesy of the United States Navy

9. Architecture and Strategy, Part 1 • Strategic decisions are technical, technical decisions are strategic • Consider the loop • Strategic Identity is “We take end-to-end responsibility” • Dominant error types are hard to find, except in the field • Drives toward designs that provide good diagnostics, easy fixes • Enables strategy of “We take end-to-end responsibility” • Another alignment (or failure to) • Do you know the trade revenue/cost for getting it right versus getting it sooner? • How do discovery costs and recovery costs compare? • Is your technical architecture supportive of the trade? Do you know how to adapt to the trade in your business/organization?

10. Strategic to Technical, Part 2 • The graph shows some key results from a study of spacecraft failure by D. Bearden • Imagine your program is predicted to be at the indicated point • What should you do? • What are alternative interpretations, and how do they have rippling technical and strategic consequences? Data from D. Bearden, Fourth IAA International Conference on Low-Cost Planetary Missions

11. Interpretation One: No-Go Zone Interpretation #1 Get more time (money)

12. Interpretation Two: KISS Interpretation #2 Simplify the system KISS=Keep It Simple, Stupid

13. Interpretation Three: Process Choice The world of big space programs Classic, expensive processes essential “Rough-and-ready,” FBC processes fine Interpretation #3 Change the process. Build it differently.

14. Architecture and Strategic Identity • At the simplest level, architecting is about bringing problem and solution into alignment • It is about finding satisfactory and feasible solutions to ill-structured problems • At another level, architecting is about aligning tensions between problem, solution, program, and organization • Program: How we go about implementing a solution • Organization: The human grouping that does the solving • The “Art” of Systems Architecting has, at least, two components • The art of synthesis of creative solutions • The art of balance in multiple, disparate dimensions of problem, organization, and program

15. Three Systems Paradigms Ill-structured problem: A problem where the statement of the problem depends on the statement of the solution Maier and Rechtin, The Art of Systems Architecting, second edition, CRC Press, 2000

16. A More “Academic” Digression • As the previous highlights, a major challenge in architecting (in systems engineering in general) is developing and reconciling multiple, disparate views of our system-of-interest • So, how good are our methods for doing it? • Unfortunately, not nearly so good as we would like • What we really want is the ability to relate, or transform, models in one perspective or viewpoint to those in another

17. Model Transformation • Engineers, managers, Generals, and users don’t expect or want to see the same kinds of models Direction of control Queued Flow Hard Coupled-Flow Soft Coupled-Flow

18. Views and Viewpoints • We can think of this as views and viewpoints • Definitions: • A view is a description of the entire system from the perspective of a set of related concerns. A view is composed of one or more models. • A viewpoint is a standard or template for constructing a view • A model is an abstraction or representation of some aspect of a thing (here of a system) • If I have one view how can I evaluate its consistency with another view of the same system? • What modeling formalisms will automate this? The view is what you see The viewpoint is where you look from

19. 1471 Architecture Description Standard Has • An Architecture Description is the document (paper or otherwise) that describes an architecture • Larger scale solution is to make use of multi-view or multi-dimensional architecture descriptions System Architecture 1 Described by 1..* Architecture Description 1..* 1..* 1..* 1..* Covers Conforms to 1..* Stakeholder Concern Viewpoint View 1..* 1 1 1..* 1..* 1..* Covers One can more readily specify architecture description methods than process methods What is an appropriate set of views in specific cases? Defines Method 1..* Model Viewpoint Library

20. The Framework in Words • A system has 1 architecture • An architecture is described by 1 or more architecture descriptions • An architecture description is composed of 1 or more of stakeholders, concerns, viewpoints, views, and models • A stakeholder has 1 or more concerns • A concern has 1 or more stakeholders • A viewpoint covers 1 or more concerns and stakeholders • A view conforms to 1 viewpoint • A viewpoint defines the method of a model • A view has 1 or more models and a model is part of 1 or more views • A viewpoint library is composed of viewpoints

21. The Notion of View Consistency Consistency of a front, top, and perspective view can be grounded in geometry What is the equivalent for consistency between a physical block diagram, a functional model, and a performance model?

22. Trying to be Mathematical • A nice extension of the metaphor would be: • A system is an object in a vector space • A viewpoint is a subspace of the vector space • A view is the projection of the object onto the subspace • Leads to nice ideas, like dimensionality and minimal representation, but it doesn’t seem to work • Instead, we need a stronger notion of what it means to model, in a generalized mathematical sense (not just behavior). Maybe “Algebraic Semiotics” (Goguen)? • Viewpoints are modeling languages are “Sign Systems” • Sign Systems are a special type of “category” built around how “sentences” are constructed • Views are instances of objects in a sign system • Sign systems are related by morphisms, known as “Semiotic Morphisms” • Viewpoints plus morphisms form a larger category • Consistency and completeness become membership functions • The executive’s model is a “forgetting functor” applied to mine (plus nice icons)? • This is a research topic

23. Strategy and Architecture, Again • A great laboratory for architecture versus strategy is uncertainty, measures of success in, and our responses to it • If we build the system the customer wants, and the customer gets hammered by a competitor who chose differently, was the system we architected successful? • If we design for flexibility, and the customer needs it, and doesn’t use it, was the system successful? • Who is responsible for big bets (versus no-regrets or real option) alternatives? • Where in the architecting process should consideration of uncertainty at organizational, programmatic, and technical levels be made?

24. The DC-3 Story • The DC-3 is often regarded as the most successful airplane every built • Well identified architects • Jack Northrop and Arthur Raymond • Noted for important technical innovation • Monocoque fuselage with open, flat floor from front to rear • Two engines, but retaining single engine out survival over major US mountain ranges • Clear superiority over the preceding successful airplane, the Ford Trimotor

25. The Fuller Story • The DC-3’s technical innovations were done by others at the same time • Boeing 247 was competitor • DC-1 and DC-2 were not highly successful • Why was the DC-3 the breakthrough? • The hazards of optimality • Boeing carefully optimized their airplane for the airmail-centric routes of the time • Douglas, and his clients, were willing to risk new markets • It takes several tries to get it right • Who made the right, or wrong, choice? Boeing 247 DC-1 DC-2

26. Valuing Agility • If you have no agility you have to aim at the future point that gives you maximum expected utility • Agility is the ability to redirect your acquisition/R&D decisions in the light of new information • What is an appropriate model for valuing agility? • Are models appropriate, if our actual behavior belies what we model? Actual future Future Range An agile system adapts Extrapolated future Today

27. F-16 and F-16XL • The F-16 was architected as the ultimate close-in air-to-air fighter • But…the Air Force uses it almost exclusively for air-ground missions • The F-16XL was built in 1981. It is a much better air-ground aircraft • 40% more range • Double the weapons • Superior low altitude flying • But, the F-16 was not transitioned to the F-16XL, indeed it was never purchased at all. Why? Photograph courtesy of NASA

28. Conclusions • The focus of Systems Architecting is technically solving ill-structured problems • However, beyond the elementary level, it runs directly into strategic considerations • There is, like it or not, synergy and antagonism between technical and strategic choices • Lack of one often leads to lack of the other • Lack of clarity in one typically implies lack of clarity in the other • We’ve looked here at some laboratories for observing this interplay, and considered some of ways in which we might address them