Policies and Protocols for Interoperability, Cyber-Security, and Resilience

1. Policies and Protocols for Interoperability, Cyber-Security, and Resilience GridSchool 2010 March 8-12, 2010  Richmond, Virginia Institute of Public Utilities Argonne National Laboratory Dr. John C. Hoag School of Information and Telecommunication Systems Consortium for Energy, Economics and the Environment Ohio University hoagj@ohio.edu  877-660-4624 Do not cite or distribute without permission MICHIGAN STATE UNIVERSITY

2. Themes of Today’s Talk • General design elements like protocols, methodologies, security mechanisms, and control systems • Standards – organizations, relevant documents, status • Research and practice “gaps”

3. Outline of first part, systems design • Interoperability and protocols • Security • Methodology • Conferral of degrees

4. Interoperability • We start here because of statute. Cyber security (CS) is more important – now and in the future. • Hardware interconnects; software interoperates. • End-to-End Principle: systems must work together regardless of • What connects them • Their underlying “platforms” • Without interoperability, we simply bridge systems. Security infers that we compartmentalize subsystems anyway.

5. Interoperation, demonstrating end-to-end nature

6. Interoperation: concealing non-dependent features

7. Principles for Interoperation • Protocols exist for every software layer. Their scope includes • Syntax, structure of message • Semantics, meaning of elements in context • Per layer, there is a PDU, Protocol Data Unit • Fixed or variable length • Numeric address scheme, global or local • Payload • Optional internal error detection/correction scheme • Protocols can be de facto or de jure; open or closed/proprietary • “Show Me” rule: Internet-related protocols must work before consideration as a standard

8. Interoperability Design Factoids • TCP/IP and OSI are common layered protocol architectures • Principles • Communications substrate • Stable and reusable • Embedded in network devices (router, switch)embedded • End-to-End “Transport” services • Part of operating system, reside in computers (hosts) • Know context, are necessary for reliable messaging • Applications • Dumb Networks prevail: intelligence resides in endpoints/hosts

9. Security, Prelude • What is necessary and sufficient? • Firewalls are necessary but not sufficient • Cryptography is necessary but not sufficient • Security Policies (rules) are necessary but not sufficient • Concept of “perimeter” is not valid due to mobility • Are security standards (or certifications) necessary or sufficient?

10. Security, snapshot today • Largest threats • DDoS: distributed denial of service attacks; e.g., botnets • Exfiltration of data • No platform is invulnerable; some have higher value as targets (HVT) • Many platform services needlessly run “privileged” as superuser, thus targets of exploits. Virus authors know this! • Solution: turn off unnecessary services • Solution: run as little as possible as superuser • Solution: compartmentalize access to all resources • Solution: self-check integrity for unauthorized code and configmods

11. Security Tools and Techniques • Network security has two verbs: permit and deny (traffic) • Based on source or destination address, protocol, etc. • Tools and techniques (very important) • Stateless (context-free) and stateful deductive filters • Intrusion detection and prevention inductive alerters/filters • Distributed detection (probes) and directed response • These expire after use: • Preferred crypto system, PKI, based on public+private key pairing • Strong, situational, multi-phase credential authentication • by user AND device AND session

12. Recognized Systems Development Methodologies • Cf. “Lifecycle” • “Waterfall” [Each phase has work products subject to review] • Analysis, requirements, and specification • Architecture (feasible technology choices) • Detailed Design • Implementation, Test, Acceptance; Operation and Maintenance • Cyclic • Emphasis on prototype • Add features incrementally • Underlying goal is risk avoidance • Note: Neither type begins with standards development!

13. Outline of second part, Standards • Standards activities • Relevant Standards • Comments

14. Standards Activities, Public • NIST, EISA (2007) • “…responsibility to coordinate development of a framework … for interoperability of SG devices” • Cyber Security Coordination Task Group (now CSWG) • Work product: NISTIR 7168, second draft February 2010 • NERC: Bulk Power, 8 CIP Standards • DHS • CSWG: Control Systems WG, for SCADA • NPPD: National Protection Programs Directorate • CS&C: Cybersecurity and Communications • NCSD: National Cyber Security Division: CERT alerts; STORM • IP: NIPP, 18 sectors • NSA: crypto; NOC-of-last-resort; contributed SELinux • USDoE National Labs • LBNL: OpenADR

15. Standards Activities, Non-Public • Hierarchy • ISO • IEC.ch • TC57 “Power systems mgmt and associated information exchange” • ANSI • IEEE • Committee 802, LAN networking including wireless • Committee P1901, PLC, ISP-neutral • Committee P2030, SG interoperability • EPRI UCA, Intelligrid • DNP.org: DNP3 protocol • Google.org PowerMeter • Manufacturer Alliances: HomePlug, ZigBee, Wi-Fi

16. DNP • Evolution of MODICON, Ladder Logic, Programmable Load Controller • Effective for automation, closed loop control (PID) • Scope: substations, relays, etc., pre-”IED” • “outstation” PC is gateway from PLC nodes to network, SCADA • Upstream connections via dialup and LAN • Application-layer protocol of coded messages • Device addresses custom to utility • Parameter selection and values unique to device instance • Polled or event-driven traffic • Comment: can be kept confidential with integrity, subject to DoS • Comment: essentially implementation-specific, non-standard

17. IEC 62351, Security Standards for TC57 Protocols • Rule: Not-enough expected processor power means the standard will embrace weak authentication and weak encryption. • Part 3: TCP/IP traffic • End-to-End with authentication, well-encrypted TLS (SSL) • Part 4: ICCP messaging • Both secure (TLS) and non-secure nodes supported • Part 5: for IED/RTU class equipment: substations, relays, etc. • DNP3 TCP/IP LAN version, TLS but without authentication • Symmetric/shared keying • Single User per node (no concept of software multiplexer) • Serial version, not enough CPU for encryption; weak authentication • Part 6: for Peer-to-Peer comms including Web services (SOA) • App: relay control < 4ms, thus no delay allowed for encryption • Part 7: SNMP wannabe, lacked MIBs as of 2007

18. Minimum protocol approach mandated

19. SCADA Case Study, Thailand, 2008 • WiwatAmornimit, BSEE, Metro Electricity Authority, Bangkok • SCADA/DMS with some RTUs • Protocols • To RTUs/IEDs in substations for relays, meters • IEC 60870-5 via LAN and serial comms • DNP3, security governed by IEC 62351-5 • To Control Centers (backup and one neighbor) • Company-owned fiber running SONET; also TETRA radio • IEC 60870-6 / TASE / ICCP (SSL-ish) over TCP/IP

20. SCADA, Requirements and Expectations • Horowitz, et al., IEEE PES Magazine, March/April 2010 • Future T&D systems to mitigate outage exposure based on better (but sparse) VERY timely information, new logic, and advanced relays • Title: State Estimator, now one per control area • Incorporate new technologies: • Phase Monitoring Units (PMUs) • Synchronized to 1 microsecond, sending timestamped info • Should locate on 1/3 of system buses • Automatic calibration by PMUs to compensate for known errors • “Incomplete Observability” approach produces optimal model results • Comment: Not clear where to locate new “zone boundaries”

21. NIST 7628 SG CS Strategy and Requirements • SGIP-CSWG (Interoperability Panel), 369 Members • Second draft, February 2010, 305 pages • Public Comments, 238pp, reduced herein to 10 slides • Individuals submitted comments on behalf of: public interest groups (esp. regarding privacy), industry and trade associations, utilities (esp. telcos), DHS, and Microsoft, among others. • Comments included: recitals, rants, requests, specific responses. Comments ranged from merely editorial to overstatement. • No responses beyond North America • NIST prepared and published responses inline • Important concerns were escalated

22. NIST 7628 Comments (1): EEI • The Draft NISTIR should be revised to make clear that it does not create Smart Grid Cyber Security “requirements,” but rather is a strategy document intended to facilitate the development of such requirements through the SGIP/SGIPGB processes. • Since the Use Cases in Appendix A are not comprehensive, this appendix should be revised to make clear that the Use Case are not mandatory. Furthermore, the Use Cases in Appendix A of the Draft NISTIR should be revised to eliminate statements that imply broad and general requirements. The following are some non-exhaustive examples of such statements that should be either eliminated or qualified.

23. More (2): EEI, referring to the Open SG on AMI • “The most secure option would allow for one way communications from the NAN to the HAN and not allow data to flow from the HAN to the NAN.” • The requirements identified in the OpenHAN SRS establish the need for two way communications between the NAN and HAN to meet the industry’s long term functional goals. The addition of two way communications between the NAN and the HAN introduces additional risk for unauthorized access to the AMI system • For these reasons, compartmentalization of the AMI system and boundary protection should be employed

24. More (3): NERC …NERC notes that there is no such thing as full cybersecurity. That is, there is no “cyber security model” that, once achieved, will ensure full protection of the Smart Grid. There currently is no cyber security model that will accomplish complete security of the Smart Grid. Therefore, it is important that NIST understand that there are different levels of maturity involving the Smart Grid, and integration of new parts and pieces into the Smart Grid could present cyber risks because there is no industry-accepted cyber security strategy.

25. More (4): Verizon • The Smart Grid communications between the AMI meter and the utility company should not include communications from the home area network (HAN). The HAN’s untrusted environment introduces additional vulnerabilities into the communications infrastructure, and communications from HAN to the utility company should not be permitted to be sent through the meter. • The AMI meter will likely have limited processing capability and not be able to provide the robust routing, filtering, and security mechanisms that are necessary to protect the communications channels and content.

26. More (5): Verizon • …all AMI devices have a unique key to identify each device. Including the same key on all devices from a single manufacturer (and separately authenticating the messages) is insufficient to protect against spoofing. Single-key schemes have been demonstrated repeatedly to be weak against cryptographic attacks. • DNS services should not be used within the AMI system. Although they are easy to use and implement, DNS services have a history of being highly susceptible to compromise. • The AMI system must also be protected against unauthorized changes to the software in the device. In other words, it must employ some type of integrity protection or tamperproof mechanism, and be able to fail to a known state should the firmware be altered via unauthorized access.

27. More (6): DHS NPPD NCSD CSSP • None of the communication protocols currently used (primarily Distributed Network Protocol (DNP3) and sometimes International Electrotechnical Commission (IEC) 61850) are typically implemented with security measures, although IEC 62351 (which are the security standards for these protocols) is now available but implementation adoption and feasibility is not yet clear. • Many devices have no notion of a user or a role making security management a challenge. • Many of the SCADA Masters may have no way to add security without complete replacement • The information exchange requirements between the DMS and the AMI head-end, except for outage information, are not known.

28. More (7): IEEE • This requires a definition of what constitutes a security event in a power system. • IEC62351-8 recommends time limited credentials which would be one solution to the issue of revocation without a centralized security manager. • "Many of the SCADA Masters may have no way to add security without complete replacement". This is dangerously general and probably not true.. • Sections 3.5-3.11 ignore remote access connections for device maintenance and configuration. This is a grave oversight.

29. More (8): AT&T • RATHER THAN ALL INDIVIDUAL COMPONENTS PROVIDING DEFENSE AGAINST DENIAL OF SERVICE, THE AMI ARCHITECTURE SHOULD BUILD IN CAPABILITIES THAT DEFEND AGAINST INDIVIDUAL METERS BEING COMPROMISED AND, SHOULD THAT OCCUR, THAT STEPS CAN BE TAKEN TO ISOLATE THE BREACH • RE: AMI: • THE TERM “OPERATIONAL SYSTEM BOUNDARY” AND THE GENERAL TERM “BOUNDARY(IES)” ARE NOT DEFINED. • THE PROTECTION SHOULD BE APPLIED TO THE NETWORK AND THE APPLICATION LAYER. • THE PATH IS ONLY ONE PART OF THE ENTIRE CONNECTION AND THE PATH ITSELF MAY BE VIRTUAL OR PHYSICAL.

30. More (9): ATT • NIST SHOULD TAKE STEPS TO CERTIFY ALGORITHMS FOR SMALLER INTELLIGENT ELECTRONIC DEVICES (“IEDS”) WITH PROCESSORS THAT HAVE LIMITED COMPUTING POWER AND ARE NOT CAPABLE OF SUPPORTING ADVANCED ENCRYPTION STANDARD (“AES”). • AUTHENTICATION AND INTEGRITY OF AMI DEVICE-TO-DEVICE COMMUNICATION SHOULD BE PERFORMED AT THE APPLICATION LAYER.

31. More (10): Google.org PowerMeter • Comments to California PUC • “… We understand why there is a desire to delay HAN enabled devices until the standard settles a bit more to help avoid stranded assets - both for consumers as well as utilities. However, consumers should not wait an indeterminate amount of time before they start seeing the benefits of their investment in the smart grid. • The standards process for home area networking has made significant progress since the Commission approved the AMI rollouts. At least one nationally recognized standard for HANs has already been released (Zigbee 1.0) and a second version (Zigbee 2.0) with improved networking and security protocols should be available next year.

32. More (11) USDOE INL • White paper, 2009 • AMI Security, as it currently stands, is insufficient to protect the national power grid from attack by malicious and knowledgeable groups (Carpenter; Goodspeed), 2009. • Attackers can extract data from the memory of these devices including keys used for network authentication and how the device memory can be modified to insert malicious software. Once the device is connected, it can be used to attack other parts of the SG by communicating through the network. Attacks that originate with an AMI wireless network device can lead to direct control systems compromise.

33. More (11) Microsoft • Previous attempts to bolt security on late in the lifecycle (after development and deployment) have not worked in the past in other sectors and will not work in the Smart Grid. • As technology is designed and developed for the Smart Grid is it important that vendors and integrators learn from the body of knowledge available such as secure software engineering practices including developer training, organizational processes, and tools.

34. More (12) USDOE INL / Idaho • There must be a coordinated and ongoing effort to secure the SG that includes the full development lifecycle. The development lifecycle includes requirements, design, implementation, verification, validation, procurement, installation, operations, and maintenance. A failure in any phase of the lifecycle leads to defects, which leads to vulnerabilities that can be exploited by a skilled attacker.

35. Third part, research and practice gaps • G1: Ad hoc methodology • G2: Current efforts not capturing SCADA requirements • G3: Current efforts not capturing security requirements • G4: Current product offerings • G5: Public policy, w.r.t. risk and consequential damages

36. Black Swans (exist) • Black Swans and the Impact of the Highly Improbable, N.N. Taleb • Math Induction – it worked yesterday and today, it will tomorrow • No, this demonstrates over-confidence in models • Software reliability is like mechanical reliability • No, software flaws are latent and inert, waiting for activation • Rare events are Gaussian, distributed along a bell curve • No, they follow a Power-Law, “heavy-tailed” distribution • Silent Evidence • Not the known knowns, or the unknown knowns, or even the known unknowns …. but the unknown unknowns! • NIST’s critics may have registered a false negative

37. Layers of Importance and Order in SG Design

38. Where designers should focus • System State • Both distributed and hierarchical • Define modes of operation; parameters; state transitions • Address zones and locality, cf. micro and macro • THEN can develop information architecture, transaction protocols • End-to-end, authenticated transactions • Connection-oriented • Device, User, and session duration! • AMI/Retail mode • Separated, “air-gapped” from SCADA • Security operations

39. Closed-loop and Open-loop Perspectives on SG

40. Grand Challenges • At what hierarchic levels is system state to be maintained? • What are the variables of system state? • Can we extend the SG to the home without a set-top box (STB)? • Should regulators be concerned with… • Yet another network overbuild? • Retail SG (HAN) “essential” and prudent? • Obsolescence of devices? • Moral hazard? • Consequential damages?

41. Special credit to Robert E. Burns, JD • Additional resources will be made available on this site: • http://www.its.ohiou.edu/grid • Contact Dr. John C. Hoag, hoagj@ohiou.edu for more information