Compliance Performance Measure Proposals and Preliminary Trends

1. Compliance Performance Measure Proposals and Preliminary Trends Greg Pierce – Chair, Performance Measures Task Force Compliance and Certification Committee Meeting March 10-11, 2010

2. Topics • PMTF Tasks and Approach • Performance Measure Proposals • Preliminary Trends • Outreach Effort • Next Steps

3. PMTF Tasks • Establish an open, consistent and repeatable metric development process to identify and evaluate metrics • Develop compliance improvement measures at the highest levels to provide insight in terms of a set of compliance principles • Coordinate and work with Regional Entities, RMWG, OC, PC and CIPC to ensure agreement • Recommend how to benefit from the compliance improvement measures

4. Approach • Adopt RMWG’s open metric development process • Review data source on compliance and other key input factors • Propose a set of performance measures at high levels • Gather data and validate trends • Solicit feedback and other ideas from stakeholders • Prepare a draft report starting April 2010

5. Proposed Performance Measures • Compliance Culture • Timeliness of CVI/CIQ • Average Duration of Mitigation Plan Completion • Violation Risk Index • Average Duration of Audit Report Completion w/ No Violations • Average Duration of Audit Report Completion w/ Potential Violations • Availability/timeliness of 'lessons-learned’ guidance from audits, operating experience, violations and disturbances • Feedback to standards development • The detailed individual metric definitions and specifications are available at http://www.nerc.com/filez/pmtf.html.

6. Compliance Culture Measure • Metric – Compare Self-Disclosed Violations to the Total Number of Discovered Violations • Self-Disclosed = Self-report + Self-certification • Indicates the level of trust and cooperation within the “NERC Enterprise” as well as the culture of compliance of the stakeholder community

7. Trend By Discovery Method

8. Compliance Culture Trend

9. Timeliness of CVI/CIQ • Average Duration – for CVI/CIQ with Open status • Total number of CVIs/CIQs for Open and Closed status • Number of lessons learned and trends issued to industry

10. CVI/CIQ Timeliness Measure

11. Triggers for NERC/Regional CVIs/CIQs

12. Average Duration of Mitigation Plan Completion • Duration – from Date plan received to Mitigation close date • Sum of all durations divided by total number of violations discovered in a particular quarter • Remove zero duration days

13. Average Number of Days for Completing Mitigation Plans

14. Violation Risk Index (VRI) • Violation Risk Index is defined as a risk factor and severity level weighted average based on unmitigated violations • Assess potential consequences of a particular unmitigated violation. • Unmitigated violations of higher risk factor requirements have more weighting value than of lower risk factors. • Determine the change in reliability levels due to Standard requirement violations. • Weighting factors and an example are included in Appendix A.

15. NERC Unmitigated Violations(6/18/07 to 12/31/2009)

16. Violation Risk Trend

17. Average Duration of Audit Report Completion w/ No Violations • Measure audit process efficiency and ability to promptly advise industry of audit results • These reports do not have potential violations • Starting point: audit ending point • Ending point: report finalized with entity and received by NERC

18. Average Duration of Audit Report Completion w/ No Violations

19. Average Duration of Audit Report Completion w/ Potential Violations • Measure enforcement process efficiency and ability to promptly advise industry of audit results • These reports have potential violations • Starting point: audit ending point • Ending point: report finalized with entity and received by NERC • Data is not available until second quarter of 2010

20. Timeliness of Lessons-Learned • Establish a common and central platform to share lessons-learned • Timeliness of actionable lesson-learned guidance from operating experience, violations and disturbance events • Lessons-Learned guidance • Recommendations from event analysis • Compliance analysis • Advisories

21. Feedback to Standards • Timeliness of compliance results/feedback to standards • # of SARs (e.g., refine a particular requirement) • # of suggestion and comment submittals The Standards input form is available at http://www.nerc.com/files/Standards_Input_Form_Final_2008June30.doc • Direct input to “results-based” standards initiative • Focus on risk-based standards, using first assessment triage sheet

22. Outreach Effort • Chair met with and presented to the RMWG on February 18, 2010 • Chair presented to the NERC/Regional Entity Compliance Group at its conference call on February 26, 2010 • Chair presented to the OC/PC/CIPC at their joint WebEx conference call on March 9, 2010

23. Next Steps • Continue reaching out to seek guidance • Members will meet with OC and PC on March 16-17, 2010 • Recommend the proposals in June, desiring CCC approval in September 2010 • Comments and suggestions are due April 2, 2010 • Send to PMTF Chair, Greg Pierce at GPIERC2@entergy.com and NERC at jessica.bian@nerc.net

24. Question & Answer

25. Appendix A • Violation Weighting Factor Table Note: 1. The weighting values are derived by applying similar ratios developed in the Base Penalty Amount Table described in section 4 of the ERO Sanction Guidelines, Appendix 4B to the NERC Rules of Procedure. 2. Reference materials are available in a NERC white paper “Toward Ensuring Reliability: Reliability Performance Metrics”. It can be viewed at: http://www.nerc.com/docs/pc/rmwg/Reliability_Metrics_white_paper.pdf.

26. VRI Example