An Overview of Quantitative Risk Assessment Methods Fayssal Safie

1. 1 An Overview of Quantitative Risk Assessment MethodsFayssal Safie/MSFCAugust 1, 2000

2. 2 An Overview of Quantitative Risk Assessment Methods Definitions Qualitative and Quantitative FMEA � FMECA Qualitative and Quantitative Fault Tree Analysis (FTA) Probabilistic Risk Assessment (PRA) Reliability Allocation Reliability Prediction Reliability Demonstration Trend Analysis Probabilistic Structural Analysis Design of Experiments (DOE) Statistical Process Control (SPC) Manufacturing Process Capability

3. 3 Definitions Probability: The chance or the likelihood of occurrence of an event. Risk: The chance of occurrence of an undesired event and the severity of the resulting consequences. Risk Assessment: The process of qualitative risk categorization or quantitative risk estimation. Risk Management: The process of risk identification, risk assessment, risk disposition, and risk tracking and control.

4. 4 Definitions Reliability: The probability that an item will perform its intended function for a specified mission profile. Safety: The freedom of injury, damage, or loss of resources. Hazard: The condition that can result in or contribute to a mishap. Mishap: An unintended event that can cause injuries, damage, or loss of resources.

5. 5

6. 6 Items in a typical FMEA sheet for the Shuttle program: Nomenclature and function Failure mode and cause Failure effect on subsystem Failure effect on element Failure effect on mission/crew and reaction time Failure detection Redundancy screens Correcting action/timeframe/remarks Criticality

7. 7

8. 8

9. 9 Failure Modes, Effects, and Criticality Analysis (FMECA) A FMECA is similar to a FMEA; however, a FMECA provides information to quantify, prioritize and rank failure modes. It is an analysis procedure which identifies all possible failure modes, determines the effect of each failure on the system, and ranks each failure according to a severity classification of failure effect. MIL-STD-1629A, Procedures for Performing a FMECA, discusses the FMECA as a two-step process: Failure Modes and Effects Analysis (FMEA). Criticality Analysis (CA). Criticality analysis can be done quantitatively using failure rates or qualitatively using a Risk Priority rating Number (RPN). CA using failure rates requires extensive amount of information and failure data. A RPN is relatively simple measure which combines relative weights for severity, frequency, and detectability of the failure. It is used for ranking high risk items.

10. 10 Failure Modes, Effects, and Criticality Analysis (FMECA)Example

11. 11

12. 12

13. 13 Qualitative Fault Tree Analysis (FTA) X-34 Hydraulic System Example

14. 14

15. 15 Qualitative Fault Tree Analysis (FTA) Benefits: (cont�d) Can identify impact of operator/personal interaction with a system. Can help identify design, procedural, and external conditions which can cause problems under normal operations. Often identifies common faults or inter-related events which were previously unrecognized as being related. Excellent for ensuring interfaces are analyzed as to their contribution to the top undesired event. Can easily include design flaws, human and procedural errors which are sometimes difficult to quantify (and therefore, often ground-ruled out of quantitative analysis). Qualitative FTA requires cutset analysis to attain full benefits of the analysis. (Cutsets: Any group of non-redundant contributing elements which, if all occur, will cause the top event to occur)

16. 16 Considerations: FTA addresses only one undesirable condition or event at a time. Many FTAs might be needed for a particular system. Both Quantitative and Qualitative FTAs are time/resource intensive. In general, design oriented FTAs require much more time than failure investigation FTAs. Management is mostly acquainted with failure investigations FTAs. Such FTA efforts can give a false sense of how quickly a design FTA can be developed. Qualitative Fault Tree Analysis (FTA)

17. 17 Quantitative Fault Tree Analysis (FTA) Quantitative FTA is used as a Reliability and a Safety tool. It diverges from Qualitative FTA in that failure rates or probabilities are input into the tree and the probability of occurrence is computed for the cutsets and the top undesirable event. Tends to be strictly �hardware failure� oriented as opposed to Qualitative FTA (which includes hardware and other less quantifiable faults). Is excellent in comparing different configurations of a system (even if the failure rate data uncertainty is fairly high). Can be used to calculate the probability of occurrence of different cutsets and the top undesirable event for reliability predictions.

18. 18 System Description: Methane loading system - The methane is stored in a tank in a liquid form and then vaporized and loaded as a gas. This example terminated at valve failure.

19. 19 Quantitative Fault Tree Analysis (FTA)X-33 Methane Ground Storage and Loading Example

20. 20 Quantitative Fault Tree Analysis (FTA) Considerations: The probabilities derived from a Quantitative FTA should be viewed with the uncertainty fully understood. It is often difficult to obtain valid reliability data for experimental / non-production related systems. In such cases: Too few items are available for a proper statistical sample Data from �Like� systems and operating environments must be used Quantitative FTA has little or no place in failure investigations.

21. 21

22. 22 Probabilistic Risk Assessment (PRA) A typical PRA process involves: Identification of end state(s) to be assessed. Identification of Initiating Events (IE) leading to the end states. Development of the Event Sequence Diagrams (ESD) for the initiating event. An ESD shows the sequence of events from IE to end states. Quantification of ESDs (event tree). Aggregation of risk for each system end state. Risk analysis which might include: risk ranking, risk reduction, sensitivity analysis, etc.

23. 23 Probabilistic Risk Assessment (PRA)A PRA Process Example

24. 24 Benefits: Imposes logic structure on risk assessment. Evaluates risk at various system levels including system interactions. Handles multiple failures and common causes. Provides more insight into the various system failure modes and the effects of human/process interaction. Provides a tool to combine both qualitative and quantitative risk analysis. Limitations: Could be very expensive. Could be misapplied and misused due to the incorporation of qualitative data.

25. 25

26. 26

27. 27

28. 28 Reliability Allocation Reliability allocation is the top-down process of subdividing a system reliability requirement into subsystem and component requirements. Reliability allocation is performed in order to translate the system reliability requirement into more manageable, lower level requirements.

29. 29 Reliability AllocationExample

30. 30 Reliability Allocation Benefits: Reliability allocation allows design trade-off studies to be performed in order to achieve the optimum combination of subsystems which meets the system reliability requirement.

31. 31 Reliability Prediction Reliability prediction is the process of quantitatively estimating the reliability of a system. Reliability prediction is performed to the lowest level for which data is available. The sub-level reliabilities are then combined to derive the system level prediction. Reliability prediction during design is used as a benchmark for subsequent reliability assessments. Predictions provide managers and designers a rational basis for design decisions.

32. 32 Reliability Prediction Reliability prediction techniques are dependent on the degree of the design definition and the availability of historical data. Similarity analysis techniques: Reliability of a new design is predicted using reliability of similar parts. Probabilistic design techniques: Reliability is predicted using engineering failure models. Techniques that utilize generic failure rates such as MIL-HDBK 217, Reliability Prediction of Electronic Equipment.

33. 33 Reliability PredictionSimilarity Analysis Example  Fuel Turbo Pump

34. 34

35. 35

36. 36

37. 37

38. 38 Reliability Demonstration Reliability Demonstration is a reliability estimation method that primarily uses test data (objective data) and statistical formulas to calculate demonstrated reliability or to demonstrate numerical reliability goal with some statistical confidence. Models and techniques used in reliability demonstration include Binomial, Exponential, Weibull models. Reliability growth techniques, such as the U.S. Army Material Systems Analysis Activity (AMSAA) and Duane models can also be used to calculate demonstrated reliability. Historically, some military and space programs employed this method to demonstrate reliability goals. For example, a reliability goal of .99 at 95% confidence level is demonstrated by conducting 298 successful tests.

39. 39

40. 40 Reliability Demonstration Benefits: It provides a way to validate numerical reliability requirement. It provides a way to calculate the reliability that has been demonstrated so far by the item under consideration. It eliminates the subjectivity that is usually embedded in other reliability estimation methods. Through rigorous reliability demonstration test program, design weakness and failures can be revealed and corrective actions can be taken to significantly improve reliability. Limitations: It is very expensive and time-consuming to run through a reliability demonstration program. Data quantity sensitive.

41. 41

42. 42

43. 43

44. 44 Probabilistic Structural Analysis It is a tool to probabilistically characterize the design and analyze its reliability using engineering failure models. It is a tool to evaluate the expected reliability of a part given the structural capability and the expected operating environment. It is used when failure data is not available and the design is characterized by complex geometry or is sensitive to loads, material properties, and environments.

45. 45 Probabilistic Structural AnalysisTurbo-Pump Bearing Example

46. 46

47. 47

48. 48

49. 49

50. 50 Probabilistic Structural Analysis Benefits: Used to understand the uncertainty of the design and identify high risk areas. Used to perform sensitivity analysis and trade studies for reliability optimization. Used in identifying areas for further testing.

51. 51 Design of Experiments (DOE) DOE is a systematic and scientific approach which allows design, manufacturing, and test engineers to better understand the variability of a design or a process and how the input variables affect the response. It is used as a tool to optimize product design by identifying the critical design parameters that affect the reliability of the design. It is used as a tool to understand manufacturing variability and to identify the critical process variables that affect the quality and the reliability of the product.

52. 52

53. 53

54. 54

55. 55

56. 56

57. 57

58. 58

59. 59

60. 60 RSRM Production Material acceptance data ensures constituents are in family of previously used components and the statistical trends can identify potential subtle changes in vendor processes. One (of many) nozzle phenolic insulator parameters trended is residual volatiles remaining after phenolic sample is heated. SPC evaluation showed changes in residual volatile levels of silica cloth phenolic. Additional investigation revealed unanticipated change in silica vendor furnace brick (resulting in slightly different oven heat environment during silica processing). Corrective action implemented at vendor prior to continued silica production - subsequent data verifies return of parameters to within statistical expectations.

61. 61

62. 62

63. 63

64. 64

65. 65

66. 66

67. 67

68. 68 QRA is a well-established technology that involves methods and techniques beyond conducting classical PRA studies. QRA is essential to understanding uncertainty and controlling our critical processes. Implementation and use of QRA could be enhanced if QRA is incorporated as part of the system management process QRA methods and techniques are viewed as part of the system engineering effectiveness tools QRA is extremely important for the Space Shuttle Program to understand and control risk. QRA techniques are well-established, however, the application of the techniques on a larger scale will require careful planning, extensive training, and strong commitment by Shuttle Program management to pursue long term plans.