Mike D. Kinney, CPF, CSP National Security Technologies, LLC ISM Best Practices Workshop

1. Monitoring the Long-Term Effectiveness of Integrated Safety Management System (ISMS) Implementation Through Use of a Performance Dash Board Process Mike D. Kinney, CPF, CSP National Security Technologies, LLC ISM Best Practices Workshop September 12, 2006

2. Getting Started • Good news! • Opportunity to win free “stuff ” [OK, OK, beads] • After receiving initial approval of their ISMS programs, some contractors have lacked an effective method to monitor long-term effectiveness of their ISMS program • DEAR Clause 48 CFR 970.5223-1, “Integration of Environment, Safety and Health into Work Planning and Execution”, requires the overall integrity of the contractor ISMS program to be maintained

3. Background • DOE has issued guidance to support annual contractor reviews • DOE G 450.4-1B, “Integrated Safety Management System Guide for use with Safety Management System Policies (DOE P 450.4, DOE P 450.5, and DOE 450.6); The Functions, Responsibilities and Authorities Manual; and the Department of Energy Acquisition Regulation” • Contains Continuing Core Expectations (CCEs) to guide ISMS long-term effectiveness reviews

4. Background • The Nevada Site Office (NSO) issued a local Directive addressing long-term ISMS maintenance • NV O 450.4, “Safety Management System Maintenance” • In January of 2006, The Defense Nuclear Facilities Safety Board (DNFSB) has issued a technical report that examines current implementation of ISMS at DP sites • DNFSB/TECH-36, “Integrated Safety Management: The Foundation For a Successful Safety Culture”

5. Challenges • In some instances, DOE/NNSA contractors do not have a consistent method to effectively review long-term implementation of their ISMS program • Some contractors attempt to rely on traditional inspections or assessments in lieu of a more programmatic approach • Results don’t always accurately reflect actual level of implementation

6. Challenges (continued) • Comparison of performance with previous evaluations (e.g., trending) can also prove difficult • This series of collective challenges do not support being able to demonstrate to DOE/NNSA/DNFSB that ISMS is being effectively maintained • Lack of a clear method to monitor long-term implementation of ISMS can also lead to reduced buy-in at the task level as well as lack of commitment from senior management

7. Method • To assist with meeting Headquarters and Local Site Office commitments regarding ISMS long-term maintenance, NSO established an Integrated Safety Management Council (ISMC): • Includes Contractors, User Organizations, and Federal staff • Co-chaired by NSO and User Organization representatives • Monthly meetings are utilized to address new initiatives or programmatic challenges • Coordinates performance of annual ISMS reviews

8. Method • To assist with the ISMS annual review, the NSO ISMC established a performance dash board process: • Utilizes Continuing Core Expectations (CCE) contained in DOE G 450.4-1B • Performance dash board also contains individual evaluation criteria for each CCE • Completed performance dash board identifies current implementation status for each participating organization

9. DOE Continuing Core Expectations (CCEs)

10. ISMS/CCE Performance Dash Board CCE -3: Work activities reflect effective implementation of the functions of ISMS. Contractor A Contractor B Contractor C DOE Work is defined, hazards are identified, and actions to prevent or eliminate the hazards are taken. Controls are developed and implemented. Work is properly authorized. Work is accomplished within controls. Appropriate worker involvement is a priority.

11. Method • NSO ISMC Performance Dash Board: • Contains color gradients, passed on the following rankings: • Blue: Significantly exceeds expectations • Green: Meets expectations • Yellow: Expectations not fully implemented • Red: Limited implementation of expectations, management attention required

12. CCE -3: Work activities reflect effective implementation of the functions of ISMS. Contractor A Contractor B Contractor C DOE Work is defined, hazards are identified, and actions to prevent or eliminate the hazards are taken. Controls are developed and implemented. Work is properly authorized. Work is accomplished within controls. Appropriate worker involvement is a priority. ISMS/CCE Performance Dash Board

13. ISMS/CCE Performance Dash Board Analysis • Contractors A & C may chose to examine other contractors processes for authorization of work and worker involvement • Contractor B may be able to assist other contractors regarding methods to enhance identification and control of hazards

14. Method • In addition to identifying implementation for CCE criteria, implementation to the CCE title level is also identified • To assist with tracking trending and support evaluation results, performance is compared with pervious years • Five year base line • Performance “arrows” utilized to identity annual performance for given color gradient/CCE title • Annual results, including potential site wide challenges, are identified in collaborative manner with ISMC members

15. FY02 FY03 FY04 FY05 FY06 Y, ◄► Y, ▼ Y, ▲ System Performance Indicator Key: Improving ▲; No Change ◄►; Declining ▼ Discussion: Processes for identification of hazards and controls (including selection of standards) have begun to improve. ISMS/CCE Performance Dash Board Analysis

16. ISMS/CCE Performance Dash Board Analysis • Implementation of the CCE title level also supports identification of programmatic challenges • Conveys management level information in a timely manner while readily focusing attention of potential challenges

17. Conclusions • Use of a DOE ISMS CCE Performance Dash Board has proven to provide the following benefits: • Enhanced visibility of ISMS implementing processes • Open communication of potential challenges as well as successes • Ability to track and trend performance • Enhanced participation throughout Contractor, User Organizations, and Federal personnel

18. Conclusions(continued) • Other benefits from application of this process include: • Continued support from ISMC participating organizations • Value added method to address contractual requirements • Ongoing support from management • Perhaps most importantly, use of a DOE ISMS CCE Performance Dash Board is ensuring that the importance of ISMS implementation remains on the forefront versus being viewed as just another regulatory requirement