Federal Aviation Administration

1. Federal Aviation Administration Safety Management Systems for Design and Manufacturing Organizations Concepts and Implementation for the Pilot Project Participant Part I - Concepts MSMS Team Member FAA Aircraft Certification Service

2. Agenda for the Kickoff Day 1 Afternoon - Introduce SMS Concepts & the . Pilot Project Day 2 All Day - Commence Preliminary Gap Analysis Day 3 Morning - Conclude Preliminary Gap Analysis Afternoon - Wrap-up and Next Steps -Feedback on the Orientation

3. Briefing Objective To ensure a common understanding of Design & Manufacturing (D&M) SMS concepts, the Pilot Project implementation approach, and expectations

4. Detailed Agenda for this Afternoon • Related SMS Activities • SMS Fundamentals • SMS Pilot Project Implementation • Guidance & Tools • Next Steps and Discussion

5. Major SMS Activities • ICAO Activities and Products • Rulemaking – Part 5 & Part 119 • AFS Activities AVS SMS Initiatives ARC D&M Workgroup SMS for D&M AIR SMS Initiatives Int’l Collaboration Group

6. Aviation Safety (AVS) Activities • Flight Standards pilot project • Involved many air carriers and repair facilities for over three years • Drafted and evolved a variety of SMS policies • Aircraft Certification pilot project • Adopted many of their concepts and products • Goal is to remain collaborative and consistent Interoperability Consistency

7. As Applied to the D&M Community… Many SMS requirements are mature and already being adopted by the aviation industry Four SMS Components are the basis Part 5 requirements are in the process of being codified AIR goal is to remain harmonized We must define SMS policies and procedures to meet unique D&M environment

8. How does SMS affect ODA FAA was requested by the ARC to consider how ODA would work with SMS ODA unit functions and processes are not affected by SMS requirements ODA certification functions follow FAA policy and the ODA manual. Showing and finding of compliance to regulations are unaffected ODA may interact with some company QMS processes i.e. planning conformity inspections based on a mature quality system PC holder may obtain an A/W certificate without further showing … ODA may inspect for conformity with the type design Likewise some SMS processes may be useful

9. Agenda • Related SMS Activities • SMS Fundamentals • SMS Pilot Project Implementation • Guidance & Tools • Next Steps and Discussion

10. Aircraft Safety Safety Management System “A systematic approach to managing safety, including the necessary organizational structures, accountabilities, policies and procedures.” ICAO Doc 9859 “… formal, top-down, organization-wide approach to managing safety risk and assuring the effectiveness of safety risk controls. ” Part 5 NPRM – Definition

11. SMS Purpose and Methods Provides: • A systematic way to identify hazards and control risks • Assurance that risk controls remain effective • A formal means of meeting regulatory requirements (Title 14) • A means for the FAA to evaluate an organization’s safety management capability SMS is intended to be performance-based

12. What it isn’t: What it is: Compliance is integral to safety management A substitute for compliance A means for industry to meet safety responsibilities A substitute for oversight A replacement for system safety SMS completes the systems approach A requirement for a new department A set of decision-making processes for management What SMS is not and what it is…

13. SMS Components Safety Policy and Objectives All management systems must define policies, procedures, and organizational structures to accomplish their goals. Policy establishes the structure of the SMS.

14. Safety Policy • Establishes executive and management commitment and objectives for ensuring safety– what the management wants • Sets up framework of organizational structures, accountabilities, plans, procedures, and controls to meet objectives • Establishes clear standards of acceptable behavior • Documented • Communicated • Regularly reviewed

15. Required Safety Management Personnel • Appoint an Accountable Executive with ultimate accountability for the SMS • Designate a management representative to manage, monitor and maintain the SMS processes • Ensure SMS processes are established and maintained • Facilitate hazard identification and risk analysis • Monitor effectiveness of risk controls • Promote safety • Report to accountable executive

16. Emergency Preparedness • Develop and implement procedures, as necessary, that will be followed in the event of an accident or incident

17. SMS Documentation and Records • Documents safety policies, objectives, processes and procedures • May be a stand-alone manual or integrated into existing documentation systems

18. SMS Components Safety Risk Management A formal system of hazard identification, analysis and risk management is essential in controlling risk to acceptable levels

19. Safety Risk Management Understand the system and environment Identify hazards and their causes Analyze and assess risk Develop risk controls

20. SRM SA System Description System Operation Hazard Identification Data Acquisition New Hazard/ Ineffective Control Risk Analysis Analysis Acceptable Conformity Risk Assessment System Assessment Unacceptable Nonconformity Risk Control Corrective Action Design Performance Safety Risk Management and Safety Assurance SMS provides a systematic way to control risk and to provide assurance that those risk controls are effective

21. System Description Hazard Identification Risk Analysis Risk Assessment Risk Control Describe the System • What? • Analyze systems (organization, process, product) to understand critical factors • For the purpose of identifying hazards • When? • Initial design of systems, organizations or products and its operation and maintenance • Development of design and manufacturing processes and procedures • New or recurring hazards being identified • Planned changes

22. Describe the System (cont’d) Consider your which organizations, processes and products are relevant Hazards should be identified along the way System descriptions do not necessarily have to be overly complex Whatever process is used it should be collaborative Effectively describe your system and tasks to the level necessary to identify hazards

23. System Description Hazard Identification Risk Analysis Risk Assessment Risk Control Hazard Identification A hazard is any existing or potential condition that can lead toinjury, illness, or death to people; damage to, or loss of, a system, equipment, or property; and/or damage to the environment. FAA Order VS 8000.367

24. Focusing the Definition of Hazard Identification D&M Organizations System Description Hazard Identification Risk Analysis Risk Assessment Risk Control • Hazard: a condition that could foreseeably cause or contribute to an aircraft accident. • Accident: an occurrence associated with the operation of an aircraft that takes place between the time any person boards the aircraft with the intention of flight and all such persons have disembarked, and in which any person suffers death or serious injury, or in which the aircraft receives substantial damage (from 49 CFR § 830.2, Definitions).

25. WHAT? WHO? WHEN? WHY? HOW? Traditional approach to preventing accidents • Focus on the direct causes of past accidents • Identify unsafe acts committed by personnel • Attach blame/punish for failures to “perform safely” • Address identified safety concern exclusively Identifies: But does not always disclose:

26. Accident Causation Defenses Workplace Organization People Accident Technology Training Regulations Management decisions and organizational processes Errors and Violations Working Conditions Latent conditions and trajectory

27. “Swiss Cheese” Model of Organizational Failure Design Policies and Procedures RISK CONTROLS Reviews and Quality Auditing Staffing and Resources Inter-Department Communication HAZARD 1 Procedures and practices do not represent how tasks are performed or they are not used or followed HAZARD 2 Design reviews and audits fail to address the ‘real’ issues, only addressing ‘low hanging fruit’ CONSEQUENCE HAZARD 3 Insufficient resources or trained staff available to perform tasks HAZARD 4 Poor visibility and reporting of issues with the design to staff Manifest Safety Issue Safety Assurance uses metrics and audits to monitor the performance of risk controls Risk = Severity x Probability of the Consequence

28. Example of an Organizational Failure Staffing reductions are now evaluated against product impact RISK CONTROLS Design Policy requires above 5 EOs must be incorporated in drawing Drawing checks required before release and audits are performed Elect Install specification changes acceptance standards and mandatory key inspections HAZARD 1 Company management reduces engineers to save $ HAZARD 2 Reduction in engineers results in multiple EOs not incorporated into electrical drawings CONSEQUENCE HAZARD 3 Production & QA personnel can’t determine actual design and proceed building electrical assemblies Nonconforming electrical assemblies cause arcing & sparking in flight HAZARD 4 Aircraft are delivered with nonconforming electrical assemblies Alert Service Bulletins and ADs are issued to correct arcing and sparking caused by nonconforming assemblies

29. System Description Hazard Identification Risk Analysis Risk Assessment Risk Control Risk Analysis • Important to distinguish between: • Hazard – a condition • Consequence – result • Risk – likelihood & severity of the consequence • Analyzing risk involves the consideration of both the likelihood and the severity of any consequences. ICAO Doc. 9859

30. System Description Hazard Identification Risk Analysis Risk Assmt Risk Control SRM From Hazard to Risk Deficient Conditions impacting activities = Variable Performance Hazards Causing… Resulting in… Active Failures Consequences Likelihood Severity Risk Analysis Risk Assessment

31. Risk Analysis • Risk is the composite of the predicted likelihood or probability and the severity of each possible consequence of each identified hazard Hazard Active Failure Consequence x Risk Level Severity Likelihood Adapted from ICAO Doc. 9859

32. Risk Assessment System Description Hazard Identification Risk Analysis Risk Assessment Risk Control Risk assessment determines the level of risk to use in making a bottom line decision. Unacceptable mitigation required Acceptable Examples of Risk Levels A risk matrix is one tool used for risk assessment. There are many other tools available to accomplish the same result.

33. System Description Hazard Identification Risk Analysis Risk Assmt Risk Control Risk Control = Risk Mitigation A major component of any safety system is the defenses (controls) put in place to protect people, property or the environment. These defenses are used to reduce the likelihood or severity of the consequences associated with any given hazard or condition. ICAO Doc. 9859

34. System Description Hazard Identification Risk Analysis Risk Assessment Risk Control Risk Control - General Order of Precedence • Design out the hazard • Physical guards or barriers • Warnings or alert signal • Administrative controls • Procedures • Training

35. SMS Components Safety Assurance Once controls are implemented, the SMS must assure they are continually practiced and continue to be effective in a changing environment.

36. Safety Assurance Functions The organization shall monitor their systems and operations to: • identify new hazards • measure the effectiveness of safety risk controls • ensure compliance with regulatory requirements

37. Quality Assurance Safety Assurance vs • SA focuses on ensuring risk controls meet safety objectives • QA focuses on product conformity and customer satisfaction on a continual basis • Integration of both management systems can be beneficial

38. Why do we need SMS if we already have AS9100? Short answer: they’re different. “Quality” is important, but it does not encompass all of the elements of an SMS and visa versa. We performed a comparison and evaluated the strength of the coverage of each element in the SMS standard to AS9100B Although some QMS and SMS are identical in words differences in the scope may be substantial in one system verses another

39. Comparison between AS9100 and SMS • Excerpt from comparison between SMS and AS9100B Relevant item in AS9100 Strength of coverage (-5 to 5 scale) item in AS9100

40. SMS vs AS 9100 Requirements • In several cases it took two or more AS9100 requirements to equal one SMS requirement

41. Conclusions Safety Policy is reasonably well covered with AS9100B, with changes in semantics SRM and SA are very lacking in AS9100B, particularly in: Risk Management & Controls Employee Reporting System Safety Promotion is somewhat lacking, especially: Data Sharing

42. Data Acquisition • Types of Information Sources • Continuous Monitoring • Audits • Evaluations • Investigations • Employee Reporting Systems • Other • Opportunities for improvement are found in the data / information

43. Data Acquisition - Employee Reporting System Employee safety reporting & feedback system is required Must provide confidentiality Employees must be encouraged to use the system Data may identify emerging hazards Data must be included in analysis

44. Analysis • Analyze data to understand effectiveness of risk controls • Identify potential new hazards that need risk controls • Analyze information gathered about the organization’s risk controls

45. System Assessment • Are safety objectives being met? • Risk controls effective? • Is the organization in compliance with the regulations? • Is the SMS doing what it was designed to do? • Are there new hazards? • Is the system catching new hazards? If new hazards are identified – return to SRM to evaluate the hazard and develop a risk control (if necessary).

46. Preventive/Corrective Actions Examples • Revised policies • Redesign/modification • New procedures • Process changes • Enhanced training • Assignment of responsible persons

47. Management Review Top management will conduct regular reviews of the SMS, including: • The outputs of SRM & SA • Lessons learned • Need for changes

48. Continuous Improvement The organization shall continuously improve the effectiveness of the SMS through: • Safety and Quality Policies • Safety Objectives • Audits & Evaluations • Analysis of Data • Corrective and Preventive Actions • Management Reviews

49. SMS Components Safety Promotion The organization must promote safety as a core value with practices that support a positive safety culture.

50. Safety Promotion: Definition • Safety promotion = a combination of: • Safety Culture • Training • Knowledge Sharing • They result in activities that support the implementation and operation of SMS in an organization