Wing Commander Malcolm G. Tutty MEng, CPEng, FIE(Aust), FRAeS

1. An International Code of Best Practice for T&E & Experimentation of Complex, Adaptive Aerospace Mission Capabilities – sure we can do that, it can’t be that hard Wing Commander Malcolm G. Tutty MEng, CPEng, FIE(Aust), FRAeS Director Simulation, Trials & Ranges – Royal Australian Air Force, PhD Candidate & Vice President, Southern Cross Chapter, ITEA Si ego Certiorem Faciam, Mihi Tu Delendus Eris – the Greek Goddess of Chaos! Homer

2. Complex systems Design Mechanism Function Purpose The “glider” pattern Nature By suitably arranging these patterns, one can simulate a Turing Machine. Paul Rendell.http://rendell.server.org.uk/gol/tmdetails.htm

3. Complex systems Emergence:the creation of a new entity, one which has new properties (often a group or a system), through the interaction among multiple autonomous elements. Multiscalarity: everything is both an entity and a group. A level of abstraction has both a specification (requirements) and an implementation. Throwing away the specification once an implementation exists produces a reductionist blind spot. It’s the specification (of the interface) that ensures loose coupling. Interaction—even (or especially) intra-system —occurs through an environment. An environment that provides functionality that facilitates interaction is a platform. Architectures: agents and platforms vs. stovepipes and functional decomposition. Platform governance becomes a fundamental issue. Who owns it, runs it, controls it? Evolutionary processes are unavoidable — leading to unexpected consequences. They are also the source of all creativity. Their essence combines exploration with exploitation of discoveries. Organizations can plan to be innovative. Groups are nature’s way to build systems. We can build powerful groups because we evolved to live in groups and we can learn. How can a group’s wisdom be distilled as action? Bottom-up resource allocation. Nature and markets have self-validating criteria: reproductive success and profits. By looking carefully you can see the world in a grain of sand.

4. Two levels of emergence Colony results Ant behaviors Ant chemistry Each layer is a level of abstraction Notice the similarity to layered communication protocols No individual chemical reaction inside the ants is responsible for making them follow the rules that describe their behavior. That the internal chemical reactions together do is an example of emergence. No individual rule and no individual ant is responsible for the ant colony gathering food. That the ants together bring about that result is a second level of emergence.

5. Organizational/system structure:What’s wrong with this picture? Downward pointing arrows: commands. Upward pointing arrows: results/reports. Can be implemented with point-to-point communication links. No horizontal communication. No dashed lines. (Is that good?) It’s not accurate as a communication or operational structure. It may represent how authority is delegated, and it may represent how responsibility is assigned, but it doesn’t represent how communication occurs or how organizations really work. Functional decomposition

6. A somewhat more realistic picture The focus is on interaction among participants in the organization. Everything is both an entity and a group. David Sloan Wilson, Evolution for Everyone

7. Representation of Operations Campaign “Organisations” of Teams Tactics Theatre Teams of Teams Tactics Mission Team Tactics Engagement Individual Tactics Network Enabled JAIME CODEx

8. ADF capability management The ADF is to be concept lead, capability based Joint Vision 2020, ADDP-D.2 A concept is “a thought, idea or notion, often one derived from a generalised mental operation”

9. Technology/Systems Readiness Levels

10. LUST WOMB FISCAL REALITIES! CLEARANCE ACTIVITY CERTIFICATION ACTIVITY ADOLESCENCE TOMB DUST Complex Capability Management Model- Lust to Dust Methodology CAPABILITY IDENTIFIED ACQUISITION INTRODUCTION INTO ADF SERVICE CAPABILITY MAINTAINED CAPMGR ORD MET? Y / N CONCEPT OPS REQT DEFN 0 VEH & SYST SPO ACQUISITION & SYSTEMS ENG (ASC SIM SURVEY) 2 INTRO INTO SERVICE 7 CONOPS REQT ACHIEVED & MAINTAINED 9 PWD 10 EXPERIMENT’N 1 TTP EXPERIMENTATION 8 DT&E 3 QUALIFICATION 4 AT&E 5 OT&E 6

11. Hierarchy of needs • In order for a design to be successful, it must meet people’s basic needs before it can attempt to satisfy higher level needs Level of need Highest Value - Allows users to create and explore areas beyond the original design Creativity Proficiency Usability Reliability Functionality High Value - Empowers users to do more than they could before Moderate Value - Easy to use, tolerates mistakes Low Value - Operation is consistent and reliable Little Value - Meets design requirements, e.g. play, record

12. Aircraft Mission Systems Compatibility • OPERATIONAL CONCEPT / REQUIREMENT • PHYSICAL FIT • ENVIRONMENTAL & STRUCTURAL • FLUTTER • PERFORMANCE & HANDLING QUALITIES • STORE EMPLOYMENT & JETTISON • FUNCTION / BALLISTICS & OFP VERIFICATION • SAFE ESCAPE & TEMPLATES • SYSTEM SAFETY & TESTING SMART • -> OPERATING CONFIGS / LIMITS & PUBS

13. Handing / Flying Qualities Aerodynamic Shape, Primary Aircraft or Store Structural Characteristics, Aeroelastic or Mass Distribution, C of G Shift of  12.7 mm, Store Weight Change of  5%, Pitch, Roll or Yaw Moments of  10%, Interface Connections, Suspension Lug Location, Environmental Tolerances, EMI/EMC Degradation, HERO Degradation Role Functional Concept or Delivery Modes, Any OFP or SMS change, Fuze, Safe/Arm Any MACS Change, Aircraft Thrust, Store Ballistics and/or Propulsion, Explosive or Fragmentation Performance, New or Amended Nomenclature. Significant Changes

14. MIL-HDBK-1763 • Ground Test Procedures - Appendix A • 100 Fit and Function Tests • 101 Fit Test • 102 Function Test • 110 Static Ejection Test • 120 Aeroelastic Ground Vibration Test (GVT) • 130 Structural Integrity Tests • 131 Aircraft-Store/Suspension Equipment Structural Integrity Ground Test • 132 Carrier Suitability Test • 140 Wind Tunnel Tests • 141 Effects of Aircraft on Captive Stores/Suspension Equipment • 142 Effect of Captive Stores/Suspension Equipment on Aircraft • 143 Aeroelastic Effects Test • 144 Separation Tests • 150 Environmental Tests • 151 Vibration Test • 152 Aeroacoustic Test • 153 Hazards of Electromagnetic Radiation to Ordnance (HERO) Test • 154 Electromagnetic Compatibility / EM Interference (EMC/EMI) Test • 155 Temperature Extremes and Thermal Tests • 160 Gun/Rocket/Missile Firing Tests • 161 Gun Firing Test • 162 Rocket/Missile Firing Test

15. MIL-HDBK-1763 • Flight Test Procedures - Appendix B • 200 Inflight Loads Test • 210 Flutter Test • 220 Environmental Tests • 221 Vibration Test • 222 Aeroacoustic Test • 223 Thermal Test • 224 EMC/EMI Test • 230 Flying Qualities Test • 240 Performance and Drag Tests • 250 Captive Flight Profile (CFP) Tests • 251 Handling Qualities Test • 252 Structural Integrity Test • 253 Endurance Test • 260 Carrier Suitability Test • 270 Employment Tests • 271 Release Test • 272 Launch Test or Weapons Survey and Demonstrations (for the Army) • 273 Gun Firing Test • 274 Dispense Test • 280 Jettison Test • 290 Ballistics Tests • 291 Weapon Freestream Ballistics Test • 292 OFP Ballistics Evaluation Test • 293 Separation Effects Derivation Test • 294 OFP Ballistics Verification Test

16. MIL-HDBK-1763 Tests 253 & 271 DT&E - F-111G with SSB / GBU-39 CFP DT&E - F-111G with PLOCAAS – Active Suppression

17. Test Planning & Reporting Describe (see also MIL-HDBK-1763 & MIL-STD-831): • Items Under Test – expected and actual • Outline Test Objectives (the COIs and CTPs) • Method of Test • Sequence of tests and fall-back plans – test cards set-up for each test come later • Review of Training Currency and Needs for Test Team for MoT • Review of Test Locations – before flights, T/O, transit, tests themselves, termination systems, RTB, landing & recovery, after flight • Review of Instrumentation & Analyses (has it actually been done this way before and using the Team?) • Review of Safety and risks – personnel, test items, RF, facilities • Write down everything objectively in third person as to what needs to be done and what happened • Recommendations & Conclusions must address Test Objectives and all findings – the Test Report needs to provide details so others can move on

18. Acknowledgements Elements of this presentation is based on: • Universal Principles of Design (2003)William Lidwell, Kritina Holden, Jill ButlerISBN1-59253-007-9 • MIL-HDBK-1763 / 244 / AAP 7001.067, Air Armament • MIL-STD-3014 • Related Works: • Power to the Edge (2005) Alberts & Hayes, www.dodccrp.org • Critical Mass (2004) Philip Ball, ISBN 0-09-945786-5 • Why most things fail (2005) Paul Ormerod, ISBN 0-571-22013-4 • The Wisdom of crowds (2004) James Surowiecki, ISBN 0-349-11605-9 • Australian aircraft stores capabilities in a network enabled world (2005) Malcolm Tutty, University of South Australia, January 2005 • Defence Capability Development Manual (2008) http://www.defence.gov.au/capability/

19. UN Survey for Mal • The question: “Please give your honest opinion about the shortage of experimentation in complex systems in the rest of the world?“The survey was a HUGE failure because:1. In Eastern Europe they didn't know what "honest" meant.2. In Western Europe they didn't know what "shortage" meant.3. In Africa they didn't know what “experimentation" meant.4. In China they didn't know what "opinion" meant5. In the Middle East they didn't know what "solution" meant6. In South America they didn't know what "please" meant7. In the USA they didn't know what "the rest of the world" meant8. In Australia they hung up as soon as they heard the Indian accent.