Report of Architecture and Product Working Group. ICM Workshop Washington, DC July 17, 2008. J. D. Baker, BAE Systems A. Winsor Brown, USC-CSSE Karl Brunson, Lockheed Martin Paul Croll, CSC Thomas Knott, OSD Art Pyster, Stevens Paul Russell, Aerospace
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
July 17, 2008
Suggest OSD initiatives and other actions to address those issuesWorking Group Charter
Don Firesmith (from the SEI): The set of all the most important, pervasive, higher-level strategic decisions, inventions, engineering trade-offs, and assumptions (DIETAs), and their associated rationales concerning how the system meets its allocated and derived product and process requirements.Definition of Architecture
The Firesmith definition is the more useful for CP and ICM
As defined by Firesmith, the architecture includes many DIETAs and their rationale, not just the risky ones.
For CP and anchor points in the ICM, we will focus on risky DIETAs; i.e., DIETAs with weak rationale which, if wrong, could have a significant negative impact on program cost, schedule, or performance.
Strong rationale is based on objective evidence. Weak rationale is based on assertion and opinion.Focus
Systems and software engineers establish the most critical requirements/objectives – including those for “ilities”
Systems and software architects develop a system and software architecture that the architects believe will simultaneously support all critical requirements/objectives
Engineers evaluate the architecture for how well it really supports critical requirements/objectives, creating substantiating evidence for the architecture or identifying weaknesses in it
Today, it is common for any of these activities to be shortchanged, especially the third.System Architecting Paradigm
Scenario-based execution of prototypes
Scenario-based execution of simulations
Appeal to historical analogy (we did something similar several times before)
Architecture Quality Cases (analogous to safety cases) with claims, arguments, and evidence
Process execution results, such as test results from early software buildsTypes of Evidence
Action: Develop architectural representation guidance requiring DIETAs to be developed in sufficient detail to support rigorous evaluation. For example, DoDAF architectures typically don’t contain enough information to perform safety case analyses or to understand the security properties of the system.
The “ilities” are often understated in the requirements/objectives, yet are often a key source of problems later in system development. An architectural view for each of the relevant quality characteristics is required.
Action: Develop guidance requiring “ilities” to be sufficiently documented and articulating what sufficient means.
Action: Research how to present sufficient information in the views to support adequate evaluation.CP/ICM Issues and Actions (unordered)
Software IndependenceExamples of Quality Characteristics
Ease of Learning
Action: Develop guidance requiring the rationale for DIETAs to be stated in sufficient detail and articulating what sufficient means.
There is no guidance for what evidence is adequate for any given situation or how that evidence should be presented (analogous to the problem of knowing when you have tested enough). How much prototyping is “enough”? How much evidence is “enough”?
Action: Conduct research on how much prototyping and evidence is enough and then document the research results in guidance.
Action: Engage Chris Powell on his dissertation research based upon his assessment of ACAT 1D program architectures since July 2004.CP/ICM Issues and Actions (unordered)
Action: Consider forming an architecture assessment team (and other types of assessment teams) at the OSD level that would be a resource available to interested programs.
Since competing suppliers will have different architectures, the architectures will have different risk profiles and therefore require different evidence. Who decides what evidence will be provided? The government? The supplier? How will the government fairly evaluate competing prototypes when presented with different types of evidence?
Action: Investigate legal and contractual implications of specific evidence requirements.CP/ICM Issues and Actions (unordered)
Action: Investigate legal and contractual implications of requesting regular submission of evidence and propose ways to enable regular submission of evidence.
Creating evidence is often dependent on exercising scenarios, which are extremely difficult to generate in sufficient number and sufficient diversity to uncover weak DIETAs, especially for SoS.
Action: Research how to generate an adequate and diverse set of scenarios, especially for SoS or investigate alternative approaches to developing scenarios.CP/ICM Issues and Actions (unordered)
Action: Research how to perform impact analysis across elements and how to respond to “breakage” in synchronization across elements.
As development progresses from milestone to milestone, new evidence reconfirming key DIETAs is needed. There is no guidance as to what that evidence should be and how often it should be collected.
Action: Research what evidence is required to reconfirm key DIETAs and then document the approaches in guidance.ICM Issues and Actions (Unordered)
Action: Investigate the cost and feasibility of independent non-advocate reviews vs. the cost of inadequate review by failing to use independent reviewers.ICM Issues and Actions (Unordered)