Assessment of Alternate Methodologies for Establishing Equivalent Satisfaction of the Ec Criterion for Launch Licensing

1. Assessment of Alternate Methodologies for Establishing Equivalent Satisfaction of the Ec Criterion for Launch Licensing Terry Hardy AST-300/Systems Engineering and Training Division May 19, 2004

2. Project Description • P. Birkeland and J. Greason have proposed alternate methods to satisfy the Ec criterion of 30x10-6. • One alternate methodology uses derived reliability requirements, reliability allocation strategies, and Failure Modes, Effects and Criticality Analysis (FMECA) to demonstrate an equivalent level of safety to Ec criterion. • AST will examine this alternate methodology and compare it to existing approaches.

3. Plan • AST will conduct the following in-house tasks: • Review documentation by Birkeland/Greason on the proposed process. • Examine the FMECA process, and investigate its advantages and disadvantages • Investigate the use of FMECA within FAA, for aircraft, ELV and RLV. • Compare the proposed FMECA process to AST’s existing process. • Document findings in a white paper • Present findings to COMSTAC RLV Working Group • Milestones include: • Status to RLV Working Group May 2004 • Draft Report July 2004 • Final Report September 2004 • Presentation at COMSTAC in October 2004

4. Expected Casualty Analysis Expected Casualty Analysis: • Is a well-established collective risk measure. • Has been successfully used for decades in the aerospace community for both launch vehicle risk analyses and explosive safety analyses. • Quantifies both probability and severity in assessing risk to public safety. But… • Ec analysis method is complex (inputs include characteristics for explosive and inert debris, weights/sizes, influence of wind and aerodynamic properties on debris, shelter effects on casualty area, etc.)

5. Alternate Approach The alternate approach proposed by P. Birkeland can be summarized as follows: • Derive an allowable probability of catastrophic failure for RLVs based on commercial aircraft failure rate requirements and historical ground casualty rate. • Allocate RLV failure probability to hardware and operator and allocate by phase of flight. • Use an FMECA to demonstrate that the allocated failure rate has been obtained.

6. Initial Findings The proposed approach is appealing because: • Setting reliability goals is consistent with approaches used by NASA, FAA for commercial aircraft. • Setting reliability requirements for passenger-carrying RLVs is consistent with approach taken by FAA for commercial aircraft. • Allocating reliability between hardware and non-hardware systems explicitly recognizes that system safety is more than just hardware failure.

7. Initial Findings The proposed approach is appealing because: • The methodology directly links the system safety process to expected reliability objectives. • FMECA is an excellent tool for methodically identifying safety issues and mitigation measures during design, especially when used in parallel with other hazard analysis/risk assessment tools.

8. Initial Findings AST has some concerns with this approach: • It may be difficult to justify a top-level failure probability or reliability allocations based on a comparison to aircraft. • Consequence of a launch vehicle crash is potentially higher than aircraft due to presence of fuel and oxidizer. • Most RLVs will operate at high velocity with higher kinetic energy and higher risk of aerodynamic breakup. • Aircraft industry is mature, with years of history and performance, in comparison to RLV industry. • The proposed approach does not explicitly incorporate population density or casualty area.

9. Initial Findings AST has concerns with any approach based only on an FMECA: • It is likely that the FMECA will miss some failure modes, especially if software and human interactions are not considered. • FMECA does not consider combinations of failures, and most accidents are the result of a confluence of factors, and often due to a combination of seemingly low-consequence failures. • FMECA does not include a quantitative assessment of severity. • FMECA does not usually account for uncertainties in the input data. • FMECA can provide optimistic system reliability estimates.

10. Further Work • AST will continue its investigation of this proposal, comparing it to the existing Ec methodology and other analytical approaches. • AST will prepare a white paper documenting its findings. • AST will present the full report at the next COMSTAC meeting.