Documents in flux. MNS - Mission need statement Non system specific, a needed capability.

1. EMIS 7307 T&E Part 2 DOD 5000 Changes • Documents in flux. • MNS - Mission need statement • Non system specific, a needed capability. • Being replaced by Initial Capabilities Document (ICD). • ICD identifies the capability gap, describes the evaluation of different approaches and proposes an approach to pursue. - Supports the MS A decision.

2. EMIS 7307 T&E Part 2 DOD 5000 Changes • ORD - Operational requirements document • Minimum acceptable capability of possible system. • ORD is being replaced by Capabilities Development Document (CDD) at MS B and Capabilities Production Document (CPD) for MS C. • CDD- Contains operational performance parameters to design and build an affordable and useful increment includes KPPs (Key performance parameters). • CPD - Contains production attributes of a single increment.

3. EMIS 7307 T&E Part 2 ICD CDD CPD Revised DoD 5000

4. EMIS 7307 T&E Part 2 DoD 5000 with planned upgraded capabilities called increments. A spiral is used when upgrades are expected but are TBD. AoA Analysis of Alternatives DAB Defense Acquisition Board JROC Joint Requirements Oversight Council

5. EMIS 7307 T&E Part 2 A significant number of documents have relevancy to T&E.

6. EMIS 7307 T&E Part 2 Test Documentation Relationships Analogous to A, B and C spec relationships

7. EMIS 7307 T&E Part 2

8. EMIS 7307 T&E Part 2 Test Program Documentation

9. EMIS 7307 T&E Part 2 NATIONAL POLAR-ORBITING OPERATIONAL ENVIRONMENTAL SATELLITE SYSTEM (NPOESS) Test and Evaluation Master Plan (TEMP)

10. EMIS 7307 T&E Part 2 NATIONAL POLAR-ORBITING OPERATIONAL ENVIRONMENTAL SATELLITE SYSTEM (NPOESS) TEMP

11. EMIS 7307 T&E Part 2 NATIONAL POLAR-ORBITING OPERATIONAL ENVIRONMENTAL SATELLITE SYSTEM (NPOESS) TEMP

12. EMIS 7307 T&E Part 2 Test Program Documentation • System Test Plan • Describes ‘big picture’ of all the testing for the whole system, not a management document. • This is the foundational technical test document from which subservient plans are derived. • Includes laboratory, component, subsystem, system, software etc. • Detail Test Plan • In a large system, each of these topics (laboratory, component, subsystem, system, software etc.) is likely to be a volume or a book of its own.

13. EMIS 7307 T&E Part 2

14. EMIS 7307 T&E Part 2 Other Documents • Test Procedures • Implements a specific test with carefully constructed step by step directions. • Completeness is paramount as unplanned steps may invalidate the data being collected. • May be augmented by test cards or scripts. • Formality in execution may require QC and/or customer sign-off. • Dry run - find problems in procedure or UUT and fix. • Document changes during execution.

15. EMIS 7307 T&E Part 2 Other Documents • Test Reports • Quick look - very important for management • Based on observation and limited, if any, data analysis • May affect next day’s testing - consider email? • Final report includes:

16. EMIS 7307 T&E Part 2 Test/Program Interrelationships

17. EMIS 7307 T&E Part 2 More about types of testing. • DT&E again. • Contractor perspective. • Demonstrates design and development and in the end-game, completeness i.e. meets spec. • Reduce risks. • High-risk components. • Intra-subsystems. • Inter-subsystem.

18. EMIS 7307 T&E Part 2 • DT&E government perspective. • Demonstrates spec compliance. • Used for acceptance of the EDM(s). • Estimate the military utility. • Confirms ready for field testing (OT&E).

19. EMIS 7307 T&E Part 2 • DT&E by phase. • Prior to program start. • Determine or at least estimate that the technology is feasible. • Concept and technology development. • Assist in selecting between possible alternative system concepts. • Info for SRR and program start decision.

20. EMIS 7307 T&E Part 2 • DT&E by phase. • During System Development and Demonstration. • Risks can be reduced to acceptable levels. • From here on engineering efforts instead of experimental efforts. • Supports decision to build an EDM. • System I&T is a part of this. • Includes EDM acceptance testing. • Includes the testing of the EDM to provide data for the LRIP decision.

21. EMIS 7307 T&E Part 2 • DT&E by phase. • During production and deployment phase. • Prelude to IOT&E. • Navy calls TECHEVAL. • During full rate production and operations. • To verify corrections and/or improvements. • In terms of calendar time and manhours consumed, DT&E is larger than OT&E and has mostly contractor involvement.

22. EMIS 7307 T&E Part 2 Another view of DT&E

23. EMIS 7307 T&E Part 2 DT&E For the MSs

24. EMIS 7307 T&E Part 2 OT&E is complex in it’s simplicity!

25. EMIS 7307 T&E Part 2 Types of OT&E • Early Operational Assessment (EOA). • Performed for the Program Office and other decision makers as a indicator of how operational things are “shaping up”. • Done by operational testers but may be mostly analytical based on DT&E provided info. • Initial OT&E • Supports full rate production decision. • Called OPEVAL by Navy.

26. EMIS 7307 T&E Part 2 Types of OT&E cont’d. • Follow-on OT&E (FOT&E) • After full rate production. • Conducted during fielding. • Checks on correction of IOT&E deficiencies.

27. EMIS 7307 T&E Part 2 More comparisons

28. EMIS 7307 T&E Part 2 Lessons for the PM that you should know too.