Software Safety Engineering

1. Software Safety Engineering In Structured or Object-Oriented Design Steven F. Mattern Science and Engineering Associates, Inc. (505) 346-9839

2. Todays Topics • Software Safety Analysis • A Historical Perspective • A Personal Perspective • Elements of Sound Safety Engineering • Structured Software Safety • Object Oriented Software Safety • Products Produced (examples)

3. Failures…A Performance Issue Unintended Performance Undesired Performance Failures Unsafe Performance Fault-Induced Performance

4. Undesired Event Analysis • System Safety Engineering • Hazards • Hazard Causes • Hazard Mitigation and Fault Avoidance • Reliability Engineering • Faults and Failure Modes • Fault Pathways to Undesired Events • Fault Detection, Fault Tolerance, and Fault Recovery • Operational Effectiveness • Faults and Failure Conditions • Fault Pathways to Undesired Events • Undesired Event Mitigation or Fail Safe/Fail Operational “CAUSED BY” a Combination of: Hardware Software Human Error Software-Influenced Human Error HMI Data Input

5. THIS WAY Software Failure Modes and/or Causal Factors System-Level Undesired Events and/or Conditions Top-Down Approach BRIDGING Between System-Level Events and the System Hardware, Software, and the Operator/Maintainer... UNDESIRED EVENT ...VIA System Safety Engineering Analysis Techniques Human Error Software Hardware

6. Elements of System Safety • Definition of Safety-Critical Functions • Tailored Safety Requirements & Guidelines • Identification of System/Subsystem Hazards & Failure Modes • Determination of System-Level Effects • Categorization of Hazard Severity & Likelihood • Identification of Hazard Causes (HW/SW/Human Interaction) • Derivation of Functional Hazard Mitigation Requirements • Determination of Safety Requirements Implementation • Determination of Residual Safety Risk • Final Categorization Hazard of Severity & Likelihood

7. System Requirements System Architecture System & Subsystem Hazard Analysis Lessons Learned Design Specifications Similar System Analysis User Inputs Systems Engineering Automated Environment Hazard Causal Factor Analysis Generic Software Safety Requirements Guidelines and Specifications Derived Functional Safety Requirements SYSTEM SAFETY REQUIREMENTS Sources of System Safety Requirements Initial Constraints Functional Requirements

8. To The Depth Required ROOT HAZARD PDR-Level Initial Depth of Analysis Software- Failure Mode A Failure Mode B Failure Mode C Failure Mode D Influenced Human Error H/W S/W HE S/WIHE H/W S/W HE S/WIHE H/W S/W HE S/WIHE H/W S/W HE S/WIHE Causal Factors Causal Factors Causal Factors Causal Factors ~ ~ CDR-Level To the Depth Required to Mitigate Effectively

9. Tailored For Customer, Program, & Environment Methods For Causal Factor Analyses • Safety-Critical Functions Analysis • System States/Modes Analysis • Hazards Analysis (System & Subsystem Level) • Fault Tree Analysis (Usually Limited to System-LevelHazards) • Failure Modes & Effects Criticality Analysis • Hybrid Event Trees or Reliability Block Diagrams • Software Data Flow Analysis • Software Functional Flow Analysis • Interface Analysis (Hardware/Software/Operator/Maintainer)

10. Potential Influence of Software • A REQUIRED FUNCTION DOES NOT OCCUR • Failure of the software to perform a required function; that is, the function is never executed, or no • output is produced. • AN UNDESIRED EVENT OCCURS • The software performs a function not required. (i.e. getting the wrong answer, issuing the wrong • control instruction, or doing the right thing but under inappropriate conditions). • AN INCORRECT SEQUENCE OF REQUIRED EVENTS • The software possesses sequencing problems. For example, failing to ensure that two events • happen at the same time, at different times, or in a particular order. • TIMING FAILURES IN EVENT SEQUENCES • The software exceeds maximum time constraints between events, fails to ensure minimum time • constraints between events or possesses duration failures. • AN INCORRECT RESPONSE TO A SAFETY-CRITICAL EVENT • The software fails to recognize a hazardous condition requiring corrective action, fails to initiate • a fault tolerant response to a recognized safety-critical function, or produces the wrong response • to a hazardous condition or failure mode.

11. Object Oriented Design Hazards (HAZ) Segment Scenario (SSC) Software Req (SWR) Segment Behavior (SBE) CSCI Scenario (SCE) Interface (CIM/CID) Testing (VER_SWR)

12. 3 1 2 7 6 5 8 Safety Analysis of Scenarios Notes: MCS SCE00820 Activate a Scheduled Mission SAS SCE00899 Perform Docked Resupply (Fwd Veh) CUI SCE00??? Begin Resupply Mission 1. Can not find the Crew Notification within CUI to begin resupply mission. SCE00899 sends message…but to where? SAS to CUI interfaces describes Resupply_Authorization_Request, but it does not appear in the CUI SRS 2. This Scenario uses the term “USES”. Is “Uses” the same as “Invokes”? 3. It is unsure if SCE00868 and SCE00871 are processed concurrently or are they processed in series 4. This message is used to request resupply guidances. Resupply guidances are provided by the POC, and include substitution rules, end-state guidance, controlled supply rate, and resupply thresholds. However, there is no development of this message in the TDA SRS…No message received and no message provided to SAS 5. AFT SCE01132 does not receive this message from RPC SCE00498, does not obtain inventory data from any know source, nor does it send the data back to RPC SCE00498 6. Same comment as #5 7. Assume the command to Retrieve Projectile Inventory Data is processed internal to RPC SCE00498 8. ACU SCE02338 Has not yet been developed #5 <Commence _Execution _Order> #1 MCS< Commence_ Execution_Command> #2 Notify Crew to Begin Resupply Mission (CUI) #3 Uses SAS SCE00882 SAS SCE00882 Develop Resupply Op Request (Fwd Veh) VMG SCE02057 Deploy SPH #6 <Dock_Deploy_Notification #1 VMG <Dock_Deploy _Notification> #2 Uses SAS SCE00868 RPC SCE00489 Provide Inventory Data ACU SCE02338 Perform Resupply Fuel Transfer (Sender) SAS SCE00868 Obtain Vehicle Inventory #1 <Inventory_Request> (RPC and ACU) #2 <Ammunition_ Inventory _ Data> (RPC) #1 <Inventory_ Request> #2 <Propellant_Inventory _Request> (AFT) #3 Retrieve the Projectile Inventory Data #4 <Propellant_Inventory_Request> #5 <Ammunition_Inventory_Data AFT SCE01132 Provides Propellant Inventory Data

13. Contextual Applicability to Hazards

14. UNDESIRED EVENT Failure Mode A Failure Mode B Failure Mode C Failure Mode D Hardware Failure Human Error Software Error Timing Error Sequencing Error Algorithm Error Test Requirements Based UponCauses & Mitigation • What Causes, Initiates, or • Influences The Undesired Event ? • What Requirements Have Been • Implemented To Mitigate ? • How Will Testing Be Accomplished • to Prove Implementation of • Mitigation Requirements ? • Testing for Functionality and Loss • of Functionality. • How Will The System Respond to • Failure ? Fault Tolerance/Recovery

15. Testing Scenario Completeness Hazards (SHZ) Hazards (HAZ) CSCI Scenario (SCE) CSCI Scenario (SCE) CSCI Scenario (SCE) CSCI Scenario (SCE) CSCI Scenario (SCE) CSCI Scenario (SCE) Testing (VER_SWR) Testing (VER_SWR) Testing (VER_SWR) Testing (VER_SWR) Detail Completeness Segment Element CSCI Context

16. Evidence of Hazard Control Software Requirements Traceability Matrix System Safety Requirement Test Procedure Requirement Affected CSCI Software Requirement Description Test Results The WCS Shall Monitor the Status of the JWS Missiles That Are Powered up. The WCS Shall Safe and Deselect Any JWS Missile That Fails BIT The JWS Missile Shall Withhold Active RF Emissions Until Terminal Impact Phase 1.2.3.1 1.2.3.3 1.3.4.1 FTH WDC FIC FAIL PASS PASS VU0012 VU10002 VU234003 System Safety Requirements Verification Matrix Test or Analysis Activity System Safety Requirement Verified Date Comments The JWS WCS Shall Include Processing Elements to Verify IPL Results The JWS WCS Shall Include Software Elements to Safe the RM During Unsafe States The JWS Missile Shall Include Software to Prevent Active RF Until Terminal Phase Failed 1ST Attempt See Test Log 1001 None JWS WCS JWS WCS JWS Missile 5/7/02 5/9/02 6/3/02

17. EXAMPLE 1.0 Man Machine Interface (MMI) 1.1PROCESS CANCELLATION 1.1.1 The System shall be designed such that the operator may exit from a potentially unsafe state with a single action 1.1.2 Exiting from an unsafe state or condition shall place the system in a known safe state, report the failure, and display the system status to the operator. 1.2 SAFETY-CRITICAL PROCESS INITIATION 1.2.1 Safety-Critical operator displays, legends, and other interface functions shall be clear concise, and unambiguous. 1.2.2 Safety-Critical displays shall be duplicated, where possible, on separate display devices. 1.2.3 Safety-Critical alerts to the crew shall be readily distinguishable from routine alerts. 1.2.4 Upon detection of an unsafe state, the system shall alert the operator to the anomaly detected, the action taken, and the resulting system configuration and status. 1.3 OPERATOR ENTRY ERRORS 1.3.1 The software shall be capable of detecting improper operator entries, or sequence of entries, and thereby prevent the the execution of safety-critical functions. 1.3.2 The software shall alert the operator to an erroneous entry. 1.3.3 Operator alerts shall indicate the error and corrective action. 1.3.4 After operator corrective action to an erroneous entry, the software shall provide positive confirmation of a valid data entry, and a real-time indication that the system is functioning properly. 1.3.5 Safety-Critical functions which require several seconds or longer to process shall provide a status indicator to the operator during processing. 1.4 POSITIVE FEEDBACK 1.4.1 Software control of safety-critical functions shall have positive feedback indicators to the operator to provide assurance that the system is functioning properly. 1.5 SAFETY-CRITICAL ALERTS 1.5.1 The operator shall not be able to clear a safety-critical alert without taking corrective action. Initial Software SafetyConstraints CRUSADER SPH

18. Example Ramification of Failure System Function Relevant SubSystems Safety Significance Fail to break engagement of an incorrectly designated target; Fire projectile into friendly Forces if incorrectly identified friendly target as hostile Tactical & Technical Software Data Displays & Controls Control (Armament) 1. Engagement Control A. Check-Fire; Manual B. Cease-Fire; Automatic Safety-Critical Function Tactical & Technical Software Data Displays & Controls Training Maintenance Inadvertent weapon firing 2. Mode Control A. Tactical: move, shoot, resupply B.Support: training, test, & maintenance Safety-Critical Function Tactical & Technical Software Data Displays & Controls General Processing Mass Memory Sensor/Actuator Interface Servo Amplifier Fail to define baseline data - Results in weapon firing into friendly forces 3. Initialization, Re-Initialization Safety-Critical Function CRUSADER SPH Tactical & Technical Software Data Display & Controls General Processing Mass Memory Sensor/Actuator Interface Servo Amplifier Fail to define baseline data - Results in weapon firing into friendly forces (dependent on modes and states of the weapon system) 4. Re-Configuration Safety-Critical Function Fail to protect self from friendly forces Sensors & Peripherals 5. Transmit Friend Signal To Friendly Forces Safety-Critical Function Fire projectile into friendly forces Tactical & Technical Software 6. Target Integrity Safety-Critical Function Improper actions leading to designation of incorrect target. Fail to accomplish offensive or defensive operation. Fail to protect self or friendly forces Tactical & Technical Software Data Displays & Controls 7. Receive, Verify, & Process Battlefield Information Safety-Critical Function 4. Determine Safety Significance Safety-Critical Function Analysis 1. Define System Function 2. Determine Ramifications of Function Failure 3. Determine Subsystems Impacted

19. 4. Identify Essential Functions of the System State Safe States & Modes Analysis Example 1. Define System States and Substates ESSENTIAL FUNCTIONS PROHIBITED FUNCTIONS PRIMARY CAPABILITIES SYSTEM STATE System Operation of any kind Weapon Control System Test, Training, and Tactical Operation Tactical Links Installed Arm/Enable Switch On Tactical Links Installed Arm/Enable Switch On Training, Maintenance, and Test Modes Weapon System Physical Security Primary Power On Break-Engage Active Primary Power On Training Software Load Primary Power On Sim Software Loaded Fire Command Cleared Primary Power On 1. Unpowered State 2. Maintenance State 3. Test State 4. Training State 5. Tactical State Weapon System Is Shut Down and Powered off Perform Intrusive Fault Protection and Isolation. Perform Servicing, Repair, And LRU Replacement Actions, Etc Test WCS software and hardware end-to-end using synthetic training targets and Operational Software Perform WCS Operator Training using Simulated Environments Using Training Software. No Live-fire, Simulated Target Weapon System Active for Live-fire 2. Define Primary Capabilities of the System State TARGET ACQUISITION SYSTEM 3. Identify Prohibited Functions of the System State

20. LTE Controller Unit LCR LEE LRE LBE LSP Event Queue Processing Unit LCR Bearing/Jam Line Event Queue Processing LCR Event Queue Processing Unit LCR Sector Processing Unit LCR LGD LTP LME Modification Event Processing LCR Processing Unit LCR Get Data Unit LCR A 40 LNE LNW LRW LSD Track Not Eligible Unit LCR #1 Solution Processing Unit LCR #2 Solution Processing Unit LCR Save Data Unit LCR C 40 B 40 Functional Analysis of SoftwareArchitecture Example Analysis • Graphical Representation • for Each Hazard or Failure • Mode • Links Specific Software • Modules to Undesired • Events AN/SWY-3 TAS • Safety Implication/Effects • Communicated to the • Design/Domain Expert

21. Definition of Safety-Critical Functions • Tailored Safety Requirements & Guidelines • Identification of System/Subsystem Hazards & Failure Modes • Determination of System-Level Effects • Categorization of Hazard Severity & Likelihood • Identification of Hazard Causes (HW/SW/Human Interaction) • Derivation of Functional Hazard Mitigation Requirements • Determination of Safety Requirements Implementation • Determination of Residual Safety Risk • Final Categorization Hazard of Severity & Likelihood Do SIL’s Make Sense? Elements of System Safety

22. The Role of SIL’s • Forces the Functional Linking of Software Architecture to Undesired Events or Hazards of the System • Forces “Cause Analysis” to be Accomplished to Prove Software Influences or Causes the Hazard to Initiate • Forces the Software Development Team to Interact with the Safety Team • Forces a Defined Protocol of Software Development Activity in the Design, Code, Test, V&V, and Configuration Management for each of the SIL Categories

23. Issues To Consider • Solving Problems = High Visibility Preventing Problems = Low Visibility • Size and Complexity of the Software • Software Development Life Cycle, Tools/Techniques • Structured Design • Object Oriented Design • Interfaces (Hardware, Software, Human) • Functionality of Interface • Ramification of Loss of Interface • Role of Systems Engineering • Requirements versus Recommendations/Considerations • Budgets for “Specialty Engineering”