August 14 2003 blackout
Download
1 / 64

August 14, 2003 Blackout - PowerPoint PPT Presentation


  • 225 Views
  • Uploaded on

August 14, 2003 Blackout. Summary Based on Interim Report of the United States – Canada Power Outage Task Force November 19, 2003. U.S.-Canada Interim Report. Released November 19, 2003 Result of an exhaustive bi-national investigation

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' August 14, 2003 Blackout' - makana


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
August 14 2003 blackout

August 14, 2003 Blackout

Summary Based on Interim Report of the

United States – Canada Power Outage Task Force

November 19, 2003


U s canada interim report
U.S.-Canada Interim Report

  • Released November 19, 2003

  • Result of an exhaustive bi-national investigation

    • Working groups on electric system, nuclear plant performance and security

    • Hundreds of professionals on investigation teams performed extensive analysis

  • Interim report produced by the teams and accepted by the bi-national Task Force


Overview
Overview

  • Overview of power system and reliability

  • Pre-outage conditions on August 14

  • Trigger events and start of cascade

  • Wide area cascade

  • Root causes

  • Next steps



Reliability overview
Reliability Overview

  • Balance generation and demand

  • Balance reactive power supply and demand

  • Monitor flows and observe thermal limits

  • Observe power and voltage stability limits

  • Operate for unplanned contingencies

  • Plan, design and maintain a reliable system

  • Prepare for emergencies

Reliably operate the system you have!






Nerc immediate response to blackout
NERC Immediate Response to Blackout

  • First hours

    • Worked closely with NERC Reliability Coordinators

    • Identified what had tripped and extent of outage

    • Assessed restoration efforts

    • Maintained open line with DOE/FERC

    • Communicated with DHS, White House, and NRC

  • First days

    • Assigned project manager

    • Established Steering Group with industry executive experts

    • Began organizing investigation teams

    • 90+ volunteers + entire NERC staff


Investigation organization overview
Investigation Organization Overview

Steering Group

U.S – Canada

Task Force

Investigation Team Lead

Root Cause Analysis

Cooper Systems

Investigation Process Review

Vegetation/ROW Management

Project Planning and Support

Transmission System Performance, Protection, Control

Maintenance & Damage

NERC & Regional Standards/Procedures & Compliance

Sequence of Events

MAAC/ECAR/NPCC

Coordinating Group

MAAC

Operations - Tools, SCADA/EMS Communications Op Planning

Generator Performance, Protection, Controls

Maintenance & Damage

Restoration

ECAR

Frequency/ACE

Data Requests and Management

NPCC

System Modeling and Simulation Analysis

System Planning, Design, & Studies

MEN Study Group


Data gathering and analysis
Data Gathering and Analysis

  • Three fact-finding meetings

    • August 22

    • September 8-9

    • October 1-3

  • Onsite interviews and inspections

  • Secure database of outage information

  • Extensive corroboration of data to determine facts

  • Analysis by teams of technical experts


Root cause analysis
Root Cause Analysis

  • Logical structure for investigating complex problems

  • Identifies changes, conditions, actions, or inactions at each causal step

  • Starts with final event and drills back through each branch of causal tree.

  • Asks “why?” at each step.

  • Accurate, reliable, defensible understanding of the root causes.

Successfully used to investigate root causes of PJM voltage stability condition

in July 1999 and established history in nuclear and defense industries.


Root cause analysis phases

16:15

16:06

15:05

Root Cause Analysis Phases

BLACKOUT

Sammis – Star

Star – South Canton

Hanna – Juniper

Chamberlin - Harding

Initial Focus

Pre-Existing Conditions

E.g. voltages, wide- area transfers,

line and generator outages, etc.


August 14 conditions prior to blackout
August 14 Conditions Prior to Blackout

  • Planned outages

    • Cook 2, Davis Besse nuclear plants

    • East Lake 4, and Monroe 1

  • Transfers high to northeast U.S. + Ontario

    • Not unusually so and not above transfer limits

  • Critical voltage day

    • Voltages within limits

    • Operators taking action to boost voltages

  • Frequency

    • Typical for a summer day

  • System was within limits prior to 15:05, on both actual and contingency basis






Blackout was not caused by
Blackout was NOT Caused by 8/13/2003

  • Heavy wide-area transfers

  • Low voltages, voltage collapse

  • Lack of IPP voltage/reactive support

  • Frequency anomalies

  • Cinergy outages starting at 12:08

  • East Lake 5 trip at 13:31

    • Contributing factor to later events, but not by itself causal to the blackout

  • DPL Stuart-Atlanta trip at 14:02

    • Contributing factor to loss of MISO real-time monitoring, but not electrically significant


Outage sequence of events transmission map key
Outage Sequence of Events 8/13/2003Transmission Map Key


East lake 5 trip 1 31 34 pm

ONTARIO 8/13/2003

ONTARIO

East Lake 5 Trip: 1:31:34 PM

2

1




Miso state estimator and reliability analysis
MISO State Estimator and Reliability Analysis 8/13/2003

  • MISO state estimator and contingency analysis ineffective from 12:37 to 16:04

    • State estimator not solving due to missing information on lines out in Cinergy then DPL

    • Human error in not resetting SE automatic trigger

  • Using Flowgate Monitoring tool to monitor conditions on previously identified critical flowgates


Firstenergy computer failures
FirstEnergy Computer Failures 8/13/2003

  • 14:14 Alarm logger fails and operators are not aware

    • No further alarms to FE operators

  • 14:20 Several remote consoles fail

  • 14:41 EMS server hosting alarm processor and other functions fails to backup

  • 14:54 Backup server fails

    • EMS continues to function but with very degraded performance (59 second refresh)

    • FE system data passed normally to others: MISO and AEP

    • AGC function degraded and strip charts flat-lined

  • 15:08 IT warm reboot of EMS appears to work but alarm process not tested and still in failed condition

  • No contingency analysis of events during the day including loss of East Lake 5 and subsequent line trips


Phone calls to firstenergy
Phone Calls to FirstEnergy 8/13/2003

  • FE received calls from MISO, AEP, and PJM indicating problems on the FE system but did not recognize evolving emergency

    • 14:32 AEP calls regarding trip and reclose of Star-S. Canton

    • 15:19 AEP calls again confirming Star-S. Canton trip and reclose

    • 15:35 Calls received about “spikes” seen on system

    • 15:36 MISO calls FE regarding contingency overload on Star-Juniper for loss of Hanna-Juniper

    • 15:45 FE tree trimming crew calls in regarding Hanna-Juniper flashover to a tree

    • PJM called MISO at 15:48 and FE at 15:56 regarding overloads on FE system



Chamberlin harding indication of ground fault due to tree contact as measured by dfr at juniper
Chamberlin-Harding Indication of Ground Fault Due to Tree Contact as Measured by DFR at Juniper


Hanna-Juniper Contact as Measured by DFR at Juniper

(3:32:03)

(3:05:41)


Hanna juniper confirmed as tree contact at less than emergency ratings of line
Hanna Juniper Confirmed as Tree Contact at Contact as Measured by DFR at JuniperLess than Emergency Ratings of Line


Effects of ambient conditions on ratings
Effects of Ambient Conditions on Ratings Contact as Measured by DFR at Juniper


(3:05:41) Contact as Measured by DFR at Juniper

(3:32:03)

Star- S. Canton (3:41:35)


Situation after initial trips 3 05 41 3 41 35

ONTARIO Contact as Measured by DFR at Juniper

Situation after Initial Trips 3:05:41 – 3:41:35


Canton Central – Tidd Contact as Measured by DFR at Juniper

(3:45:41)


138 kv lines overload and cascade near akron
138 kV Lines Overload and Cascade Near Akron Contact as Measured by DFR at Juniper


138 kV Cascade Contributes Further Contact as Measured by DFR at Juniperto Overload of Sammis-Star

15:51:41 EDT

15:05:41 EDT

16:05:55 EDT

15:32:03 EDT

15:41:35 EDT


Sammis-Star Contact as Measured by DFR at Juniper

(4:05:57.5)


Sammis star zone 3 relay operates on steady state overload
Sammis-Star Zone 3 Relay Operates Contact as Measured by DFR at Juniperon Steady State Overload


Actual loading on critical lines
Actual Loading on Critical Lines Contact as Measured by DFR at Juniper


Actual voltages leading to sammis star
Actual Voltages Leading to Sammis-Star Contact as Measured by DFR at Juniper


Major path to cleveland blocked after loss of sammis star 4 05 57 5 pm

Remaining Contact as Measured by DFR at Juniper

Paths

Major Path to Cleveland Blocked after Loss of Sammis-Star 4:05:57.5 PM


345 kv lines trip across ohio to west

ONTARIO Contact as Measured by DFR at Juniper

345 kV Lines Trip Across Ohio to West


Generation trips 4 09 08 4 10 27 pm

ONTARIO Contact as Measured by DFR at Juniper

Generation Trips 4:09:08 – 4:10:27 PM




Power transfers shift at 4 10 38 6 pm
Power Transfers Shift at 4:10:38.6 PM after 4:10:37.5 – 4:10:38.6 PM


Eastern eastern michigan detroit unstable voltage and frequency collapse and pole slipping
Eastern Eastern Michigan (Detroit) Unstable after 4:10:37.5 – 4:10:38.6 PM Voltage and Frequency Collapse and Pole Slipping

Ontario – Michigan Interface Flow and Voltages Beginning 16:10:38


Generator trips to 16 10 38
Generator Trips to 16:10:38 after 4:10:37.5 – 4:10:38.6 PM


Generator trips next 7 seconds
Generator Trips – Next 7 Seconds after 4:10:37.5 – 4:10:38.6 PM


Overloads on pjm ny ties 4 10 39 pm
Overloads on PJM – NY Ties 4:10:39 PM after 4:10:37.5 – 4:10:38.6 PM


Pjm ny separating 4 10 44 pm
PJM – NY Separating 4:10:44 PM after 4:10:37.5 – 4:10:38.6 PM


Cleveland toledo island 4 10 39 4 10 46 pm cleveland blacks out
Cleveland – Toledo Island 4:10:39 - 4:10:46 PM after 4:10:37.5 – 4:10:38.6 PM Cleveland Blacks Out


Northeast completes separation from eastern interconnection 4 10 43 4 10 45 pm
Northeast Completes Separation from Eastern Interconnection 4:10:43 – 4:10:45 PM

North of Lake

Superior


Conditions at niagara indicate progressively worsening stability conditions with prior events
Conditions at Niagara Indicate Progressively Worsening Stability Conditions with Prior Events


Island breaks up 4 10 46 4 13 pm
Island Breaks Up: 4:10:46 – 4:13 PM Stability Conditions with Prior Events


Frequency in ontario and new york during breakup niagara generation stays with western ny
Frequency in Ontario and New York during Breakup Stability Conditions with Prior EventsNiagara Generation Stays with Western NY


Generator trips after 16 10 44
Generator Trips – After 16:10:44 Stability Conditions with Prior Events


End of the cascade

Some Local Load Interrupted Stability Conditions with Prior Events

Areas Affected by the Blackout

Service maintained

in some area

End of the Cascade


Blackout root cause group 1 fe situational awareness
Blackout Root Cause Group 1 Stability Conditions with Prior EventsFE Situational Awareness

  • FE did not ensure a reliable system after contingencies occurred because it did not have an effective contingency analysis capability

  • FE did not have effective procedures to ensure operators were aware of the status of critical monitoring tools

  • FE did not have effective procedures to test monitoring tools after repairs

  • FE did not have additional high level monitoring tools after alarm system failed


Blackout out root cause group 2 vegetation management
Blackout Out Root Cause Group 2 Stability Conditions with Prior EventsVegetation Management

  • FE did not adequately manage tree growth in its transmission rights of way


Blackout cause group 3 reliability coordinator diagnostics
Blackout Cause Group 3 Stability Conditions with Prior EventsReliability Coordinator Diagnostics

  • MISO’s state estimator failed due to a data error.

  • MISO’s flowgate monitoring tool didn’t have real-time line information to detect growing overloads

  • MISO operators couldn’t easily link breaker status to line status to understand changing conditions.

  • PJM and MISO ineffective procedures and wide grid visibility to coordinate problems affecting their common boundaries


Near-Term Industry Actions Stability Conditions with Prior Events

Responses from Control Areas and Reliability Coordinators Due December 15

  • Voltage support/reactive supply

  • Reliability communications

  • Computer failure response & notifications

  • Emergency action plans and capabilities

  • Operator training for emergencies

  • Vegetation management


Next steps
Next Steps Stability Conditions with Prior Events

  • U.S./Canada Power Outage TF hearings

    • Public hearings to allow comment on report and input on recommendations

      • December 4

      • December 5

      • December 8 – Toronto

    • Industry technical conference

      • December 10 – Philadelphia

  • NERC next steps

    • NERC executive committees December 11

    • NERC committees meet January 13-14

    • Continue investigation

      • Near term analysis and recommendations in support of U.S. Canada Task Force

      • Long term analysis and recommendations for NERC


ad