TPF-C Architecture Trade A route map for the next few years

1. TPF-C Architecture TradeA route map for the next few years Charley Noecker Ball Aerospace & Technologies Corp 28 August 2006

2. Goals of this presentation • Describe a process and documentation practice to organize the decisions we need to make • Telescope size • Starlight Suppression System (SSS) • Wavefront sensing and control approach • Begin the list of specific candidates • Examples of how specific they should be • Begin the list of evaluation criteria • Identify useful metrics TPF-C architecture trade process

3. This week we will • Approve a process (like this one?) • Agree on the list of criteriaAgree on the list of candidates • As complete as possible • Later additions and modifications are expected • Assign action items to begin assessing metrics TPF-C architecture trade process

4. This week we will not • Complete the analysis of metrics or Begin the scoring • Make any actual decision among possible architectures • Take potshots at each other’s concepts • Perform detailed design (except on your own time) • Hoard innovations that could benefit another architecture • “Mix and match” will benefit planet finding TPF-C architecture trade process

5. Trade matrix features Decision: Choose new family car Options Hummer H2 Toyota Prius MUSTS Metric Score Metric Score Metric Fit in 20 ft garage Length y 17 ft y 14.6 ft DISCRIMINATORS Weight Subweight Sunroof 40 10 y 10 y Important, no difference Gas Mileage 50 2.4 10 City mpg 0.6 2 12 10 52 Important big difference Highway mpg 0.4 3 16 10 45 Towing capacity 10 10 6700 lb 3 400 lb? Unimportant big difference Totals: 100 620 930 • Decision statement: clear, concise, complete. • Identifies full scope of the question; get everyone thinking at the same level • Options: Brief identifier for each candidate. Details provided elsewhere • Musts: All of the pass / fail criteria. (Expect all realistic candidates to “pass”.) • Metrics may be shown for support • Discriminators: all of the better / worse criteria • All the ways we can compare the merits of each option TPF-C architecture trade process

6. Trade matrix scoring Decision: Choose new family car Options Hummer H2 Toyota Prius MUSTS Metric Score Metric Score Metric Fit in 20 ft garage Length y 17 ft y 14.6 ft DISCRIMINATORS Weight Subweight Sunroof 40 10 y 10 y Important, no difference Gas Mileage 50 2.4 10 City mpg 0.6 2 12 10 52 Important big difference Highway mpg 0.4 3 16 10 45 Towing capacity 10 10 6700 lb 3 400 lb? Unimportant big difference Totals: 100 620 930 • Metrics • Quantify important characteristics of candidates — things that we “value” • Scores • Subjective (numeric) ratings based on those metrics, range 0-10 • Weights: • Declare how important each discriminator is to us • Subweights • Relative weighting of metrics contributing to single discriminator TPF-C architecture trade process

7. Combining scores Decision: Choose new family car Options Hummer H2 Toyota Prius MUSTS Metric Score Metric Score Metric Fit in 20 ft garage Length y 17 ft y 14.6 ft DISCRIMINATORS Weight Subweight Sunroof 40 10 y 10 y Important, no difference Gas Mileage 50 2.4 10 City mpg 0.6 2 12 10 52 Important big difference Highway mpg 0.4 3 16 10 45 Towing capacity 10 10 6700 lb 3 400 lb? Unimportant big difference Totals: 100 620 930 • Totals show a numeric rollup of all our judgments • This arithmetic is “truthy” • Conveys a false sense of truth or authority • Really it’s only a tool we use by choice • Authority comes from our choices and how we defend them TPF-C architecture trade process

8. Final negotiation • The real meat of the decision is captured in our choices for • Scores — Weights • Algorithms in the spreadsheet • So now we reassess: • Does each discriminator have the right importance in the result? • Could reasonable tweaks in weights and scores change the answer? • Did we leave out something important? • Do we all believe the answer we’re getting? • Adjust scores and weights until we reach a consensus view TPF-C architecture trade process

9. Common scoring practices • Example from a similarly large-scale TPF-I architecture trade • Scoring meeting: 9-10 December 2004 (alpha-lib:Collection-24885) • Linear relationship was used for 55 of 56 discriminators • Choose linear relation between scores and the metric • Define top score to be 10 • Choose lowest score by mean, median, or mode of a vote • Nonlinear relationship chosen once • Curve gives score vs. star counts • Score = 0 for <100 stars • Next 60 stars have a high value • Lower value per star beyond that TPF-C architecture trade process

10. Features / benefits • Acknowledges subjectivity of decision making, but keeps it grounded in analysis • Numbers and arithmetic reflect our judgments, or we change them • Scoring by a group: balance many opinions, differing expertise • Transparently documents the decision • Factors considered – Metrics used • Value judgments – Importance judgments • Robustness of the result • Decision stands on all judgments taken together • Simplifies re-evaluation with new concepts / data TPF-C architecture trade process