DOE

DOE PowerPoint PPT Presentation


  • 80 Views
  • Uploaded on
  • Presentation posted in: General

Effectiveness Review. Effectiveness Review (ER) is most frequently cited weakness of feedback and improvement cycle due to lack of followup and recurrence of identified findingsColumbia Shuttle and Davis-Besse reactor corrosion issues are 2 of most well-known of identified and reported problems lac

Download Presentation

DOE

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


1. ISM Best Practices Workshop September 13, 2006 John L. McCabe, P.E., CSP Office of Corporate Safety Analysis DOE/NNSA Corrective Action Effectiveness Review Good Morning. My name is Larry McCabe. I work with Patty Bubar in the EH Office of Corporate Performance Assessment. It is my pleasure this morning to present a summary of corrective action effectiveness reviews in relation to the EH Corrective Action Management Program, or CAMP. As you know, corrective action effectiveness review is a generic term, and there are many types of such reviews within DOE. However, I will confine my remarks to effectiveness reviews in relation to CAMP. Good Morning. My name is Larry McCabe. I work with Patty Bubar in the EH Office of Corporate Performance Assessment. It is my pleasure this morning to present a summary of corrective action effectiveness reviews in relation to the EH Corrective Action Management Program, or CAMP. As you know, corrective action effectiveness review is a generic term, and there are many types of such reviews within DOE. However, I will confine my remarks to effectiveness reviews in relation to CAMP.

2. Effectiveness Review Effectiveness Review (ER) is most frequently cited weakness of feedback and improvement cycle due to lack of followup and recurrence of identified findings Columbia Shuttle and Davis-Besse reactor corrosion issues are 2 of most well-known of identified and reported problems lacking ER followup Effectiveness review is a part of the feedback and improvement cycle that traditionally was given little attention. In the past, conventional thinking was to quickly develop CA in response to a finding, implement the CA ASAP, and then close the finding. However, many folks began to realize that this may not be a smart approach because without effectiveness reviews, the finding is more likely to recur in future safety and health assessments. Two well-known recent events in which the absence of effectiveness reviews may have played a role were the Columbia Shuttle explosion and the Davis-Besse reactor head corrosion event. Programmatic weaknesses in both systems were identified and reported before the event occurred, but for various reasons, corrective action was not effective in preventing the event.Effectiveness review is a part of the feedback and improvement cycle that traditionally was given little attention. In the past, conventional thinking was to quickly develop CA in response to a finding, implement the CA ASAP, and then close the finding. However, many folks began to realize that this may not be a smart approach because without effectiveness reviews, the finding is more likely to recur in future safety and health assessments. Two well-known recent events in which the absence of effectiveness reviews may have played a role were the Columbia Shuttle explosion and the Davis-Besse reactor head corrosion event. Programmatic weaknesses in both systems were identified and reported before the event occurred, but for various reasons, corrective action was not effective in preventing the event.

3. Discussion Effectiveness Review process is 4th step of feedback and improvement core safety function, discussed as part of Corrective Action Management Program (CAMP) Feedback and Improvement Cycle This is the feedback/improvement cycle and is taken directly from our CA Program Guide (DOE G 414.1-5) that was issued in March 2006. Effectiveness review is illustrated in red type in the upper left quadrant as the 4th step in this cycle, starting with identification of the finding during the assessment… This is the feedback/improvement cycle and is taken directly from our CA Program Guide (DOE G 414.1-5) that was issued in March 2006. Effectiveness review is illustrated in red type in the upper left quadrant as the 4th step in this cycle, starting with identification of the finding during the assessment…

4. CAMP Background Systematic process to effectively resolve: SP-40 ES&H and Emergency Management findings Type A Accident Investigation Judgments of Need Office of Aviation Management findings Findings directed by the Secretary or Deputy Secretary CAMP initiated in response to DNFSB Recommendation 98-1 CAMP responsibilities and requirements in DOE O 414.1C Resources: CA Program Guide (DOE G414.1-5); website: www.eh.doe.gov/camp Now, let me step back and give a little background on the CAMP process itself. The CAMP program is a systematic process…(the 3rd bullet, Aviation Mgmt findings were added to the Order in 2005 and encompass findings for all 8 DOE aviation organizations). CAMP initiated on Mar 10 1999 by DOE implementation plan…Board 98.1 “Resolution of Safety Issues Identified by DOE Internal Oversight”…Board said “we want results” we want findings identified during these assessment reviews fixed. Board focused on ER’s as the “missing link”. Sites were developing/implementing CA, but the findings would recur…there was no ER. Bad situation because the sites in good conscience believed the finding was fixed…people were lulled into a false sense of security. Therefore, CAMP roles/responsibilities were put in Attachment 4 of the QA Order. For folks interested in Effectiveness Reviews, I recommend visiting our CAMP website as well as looking over the CA Program Guide and Attachment 4 in the QA Order.Now, let me step back and give a little background on the CAMP process itself. The CAMP program is a systematic process…(the 3rd bullet, Aviation Mgmt findings were added to the Order in 2005 and encompass findings for all 8 DOE aviation organizations). CAMP initiated on Mar 10 1999 by DOE implementation plan…Board 98.1 “Resolution of Safety Issues Identified by DOE Internal Oversight”…Board said “we want results” we want findings identified during these assessment reviews fixed. Board focused on ER’s as the “missing link”. Sites were developing/implementing CA, but the findings would recur…there was no ER. Bad situation because the sites in good conscience believed the finding was fixed…people were lulled into a false sense of security. Therefore, CAMP roles/responsibilities were put in Attachment 4 of the QA Order. For folks interested in Effectiveness Reviews, I recommend visiting our CAMP website as well as looking over the CA Program Guide and Attachment 4 in the QA Order.

5. Effectiveness Review Objectives verify successful finding closure. determine if CA resolved issue and prevented recurrence if CA was not effective, why not prepare revised/additional CA to improve effectiveness & resolve finding collect followup data for subsequent analysis and/or lessons learned Moving on, let’s talk about some of the major goals of a successful CA program…I’ve listed 5 ER objectives that are important…the first is to verify that the finding was successfully closed. The second…the third…the fourth…and finally, we should collect…Moving on, let’s talk about some of the major goals of a successful CA program…I’ve listed 5 ER objectives that are important…the first is to verify that the finding was successfully closed. The second…the third…the fourth…and finally, we should collect…

6. Examples of Effectiveness Reviews Documentation Review Performance Indicators Walkaround/Facility Tours Performance Test/Field Ex Interviews Now let’s look at some recommended examples of how to perform ER’s…of course, please understand that ER approval comes from the field element manager (usually the site mgr) so there is no one size fits all approach…these 5 are simply the most common methods. First is a…, second is a look at the PI’s (site database or DOE database like CAIRS/ORPS)…third is simply walking around your work area to which the finding applies…fourth is a performance test/field ex (shave 2 minutes from Emergency Response times via additional training)…and finally talk to the workers about the CA…did it work?Now let’s look at some recommended examples of how to perform ER’s…of course, please understand that ER approval comes from the field element manager (usually the site mgr) so there is no one size fits all approach…these 5 are simply the most common methods. First is a…, second is a look at the PI’s (site database or DOE database like CAIRS/ORPS)…third is simply walking around your work area to which the finding applies…fourth is a performance test/field ex (shave 2 minutes from Emergency Response times via additional training)…and finally talk to the workers about the CA…did it work?

7. Possible Causes for Ineffective CA Causal factors incorrectly identified. Causal factors correctly identified, but CA inappropriate CA not fully implemented as stated in the CAP CA implementation not timely CA creating new/different problems Org/personnel reject ownership Of course, there are times when CA just doesn’t solve the problem. Let’s look at some general causes for CA not properly working…first, were the causal factors…, second, were…, third was the CA fully implemented…, fourth was the CA timely…fifth did the CA create a new or different problem…and finally, is there a cloud over who has responsibility for fixing the finding, such as multiple players in one facility or work area. This is especially prevalent in construction areas where you got many subcontractors running around.Of course, there are times when CA just doesn’t solve the problem. Let’s look at some general causes for CA not properly working…first, were the causal factors…, second, were…, third was the CA fully implemented…, fourth was the CA timely…fifth did the CA create a new or different problem…and finally, is there a cloud over who has responsibility for fixing the finding, such as multiple players in one facility or work area. This is especially prevalent in construction areas where you got many subcontractors running around.

8. Effectiveness Review Ratings Effective: CA closed and findings are resolved Partially Effective: CA closed and findings partially resolved w/additional or revised CA needed Ineffective: CA did not resolve finding and/or will not prevent recurrence of finding Now, let’s look at the ER ratings following implementation of our CA…did it work…was the CA successful in resolving the finding with a strong likelihood that it will not recur, or were the findings only partially resolved once the CA were performed, with additional or revised CA’s needed to fix the problem. And finally, was the CA not effective at resolving the finding and is there a strong likelihood that the finding will recur? If so, let’ look at the next slide.Now, let’s look at the ER ratings following implementation of our CA…did it work…was the CA successful in resolving the finding with a strong likelihood that it will not recur, or were the findings only partially resolved once the CA were performed, with additional or revised CA’s needed to fix the problem. And finally, was the CA not effective at resolving the finding and is there a strong likelihood that the finding will recur? If so, let’ look at the next slide.

9. Fixing Unresolved Findings Re-analyze causal factors Re-examine CA and determine why finding was unresolved Develop/implement revised or additional CA Conduct followup effectiveness review This is the real challenge…we have developed and implemented CA but it has not worked…now what? This is what the feedback and improvement cycle is all about…first we re-analyze…second we re-look at the CA and determined why it was not effective, third we come up with revised or new CA elements and implement them, finally we re-evaluate the CA…was it effective in fixing the finding?This is the real challenge…we have developed and implemented CA but it has not worked…now what? This is what the feedback and improvement cycle is all about…first we re-analyze…second we re-look at the CA and determined why it was not effective, third we come up with revised or new CA elements and implement them, finally we re-evaluate the CA…was it effective in fixing the finding?

10. CAMP Effectiveness Review Requirements Field Element Manager (FEM) reviews CA effectiveness for each finding FEM decides how, when, and by whom review conducted Report approved by FEM 6 months after CAP completion Report describes review activities, results, conclusions, rating, and recommendations for each finding Approved report recommendations implemented and followed up Now, let’s take a quick look at the CAMP ER requirements contained in the QA Order…first the field element manager (usually the site manager) reviews the CA effectiveness for each finding…second, the FEM decides…, however, keep in mind that the FEM has 6 months after completion of the CAP to approve the ER. Generally, the ER report approved by the FEM will contain …Finally, the FEM assures that the approved ER report recommendations are implemented and followed-up.Now, let’s take a quick look at the CAMP ER requirements contained in the QA Order…first the field element manager (usually the site manager) reviews the CA effectiveness for each finding…second, the FEM decides…, however, keep in mind that the FEM has 6 months after completion of the CAP to approve the ER. Generally, the ER report approved by the FEM will contain …Finally, the FEM assures that the approved ER report recommendations are implemented and followed-up.

11. CAMP Effectiveness Review Results 10 completed since old QA Order (O 414.1-B) revised in APR 2004 7 open (3 late and 4 on schedule) Managers are complying and experiencing results Feedback from field has helped improve ER process This slide is a snapshot of where we are with ER results, as of August 22, 2006. So far, the CAMP’s Corrective Action Tracking System database has a total of 10 completed ER’s since the old “Bravo” version of the QA order was revised to the “Charlie” version in APR 2004. CAP’s closed out before that date were simply “grandfathered”. Right now there are 7 open ER’s with 3 being late and 4 being on schedule. Also, I would like to note that the quality of the ER’s that we have seen are very well done. Most importantly, field managers have stated that performing ER’s are worthwhile and results justify the time and effort in working through the ER process. Also, feedback from the field was very helpful in getting our CA Program Guide (DOE G 414.1-5) finalized and issued in March 2006.This slide is a snapshot of where we are with ER results, as of August 22, 2006. So far, the CAMP’s Corrective Action Tracking System database has a total of 10 completed ER’s since the old “Bravo” version of the QA order was revised to the “Charlie” version in APR 2004. CAP’s closed out before that date were simply “grandfathered”. Right now there are 7 open ER’s with 3 being late and 4 being on schedule. Also, I would like to note that the quality of the ER’s that we have seen are very well done. Most importantly, field managers have stated that performing ER’s are worthwhile and results justify the time and effort in working through the ER process. Also, feedback from the field was very helpful in getting our CA Program Guide (DOE G 414.1-5) finalized and issued in March 2006.

12. Conclusions Purpose and objectives of Effectiveness Reviews (ER) are being better understood Effective CA improves safety, optimizes operational performance, and reduces costs Must maintain management involvement and ER emphasis Finally, I would like to leave you all with the following thoughts…first, field management has grown more aware of the benefits of a good ER process. They want the CA to be effective in fixing the findings. Hey, when the findings are fixed, safety is improved, operational performance is optimized, and costs are reduced. And also, the likelihood of lost worktime and missed milestone schedules due to accidents is reduced and people can safely go home each day to their families. And that’s what it’s all about. Thank you for your attention and I will be glad to take any questions.Finally, I would like to leave you all with the following thoughts…first, field management has grown more aware of the benefits of a good ER process. They want the CA to be effective in fixing the findings. Hey, when the findings are fixed, safety is improved, operational performance is optimized, and costs are reduced. And also, the likelihood of lost worktime and missed milestone schedules due to accidents is reduced and people can safely go home each day to their families. And that’s what it’s all about. Thank you for your attention and I will be glad to take any questions.

  • Login