Systems of Systems: Cybersecurity Vulnerabilities and Opportunities

1. Systems of Systems: Cybersecurity Vulnerabilities and Opportunities Donald Wunsch, ACIL Director Ann Miller, TSL Director Applied Mathematics for Deregulated Electric Power Systems: Optimization, Control, and Computational Intelligence Crystal City, November 2003 Applied Computational Intelligence Lab & Trustworthy Systems Lab University of Missouri - Rolla

2. Funding NSF Sandia Boeing MK Finley Professorship Cindy Tang Professorship Senior Personnel Ganesh Kumar Venayagamoorthy Ron Harley Daryl Beetner Danil Prokhorov Raonak Uz-Zaman Frank Harary Personnel Narayan Vishwanathan Amit Agarwahl Sam Mulder Wenxin Liu Nian Zhang Alexander Novokhodko Xindi Cai Rohit Dua Hu Xiao Rui Xu Brian Blaha Paul Pigg Arvind Rapka Nath Qiang Yao Kevin Bollum Anjaya Shrestra Karthik Balasubramanian Pinar Demircan Daniel Treat Ian Downard Eyad Salah Tagiedin Ganesh Sridharan Jason White Krishnaprasad Balasubramanian Dayle Majors Nartaj Lakshminarasimhan Siddarth Panchal Robert Wayne Denier Tongquan Wei Jimish Doshi Ravikiran Sharda Acknowledgements

3. “system of systems” Grown/evolved by adding components not initially designed to be part of the system Interdependencies not easily identified Potential for cascading failures Potential for hidden robustness Systems of Systems: Interdependencies

4. Issues in Systems ofSystems • Trustworthiness • Testing • Market Demands • Complexity • Safety • Life-Cycle Model • Integration

5. 100 Moore’s Law 10 Recommended disk space, MB Math package 1 Math package 2 Math package 3 1 1984 86 88 90 92 94 96 1998 Complexity: Software Size Growth Source: IEEE Spectrum, January 1998

6. Complexity: Software Size Growth

7. Complexity: Interdependencies A graph representing almost 6 million lines of computer code. The graph contains approximately 33 thousand nodes and 34 thousand relations. Source: NATO Report on Visualization, 1999.

8. Failure Rates – System Calls Memory management File directory Access I/O Primitives Process Primitives Process environment Memory management File directory Access I/O Primitives Process Primitives Process environment Memory management File directory Access I/O Primitives Process Primitives Process environment Memory management File directory Access I/O Primitives Process Primitives Process environment LINUX NT Win-2000 Win-CE 25 5 20 15 30 0 10 (Source: Carnegie Mellon, CS Dept.) Normalised Failure Rate, %

9. Cascading failures Opportunities for errors Control, Communication, IT Pres. Commission on Critical Infrastructure Protection Particularly EMS & SCADA Voltage Collapse Effects of Complexity and Growth

10. Even brief – expensive Circuit fab: 20 min = ($30 M) Recent large disruption caused deaths Backups no guarantee Well-known in software safety circles Therac 25 classic example High-Consequence

11. At 0903 CST on 18 December 1997, at the Olathe (Kansas City) Air Route Traffic Control Center, a technician routed power through half of the redundant uninterruptible power system, preparatory to performing the annual preventive maintenance on the other half. Apparently the wrong board was pulled. Complexity: Ripple Effect Example

12. Results: Power only out for 4 minutes Radar and communications working within 17 minutes However, at least 300 planes were in the Olathe-controlled airspace; domino effect: hundreds of flights canceled, diverted, or delayed with problems well into the evening. Complexity: Ripple Effect Example

13. Not only did the Air Route Traffic Control Center have redundant systems, there were also standby generators and emergency batteries. Yet, that December morning, these back-up systems were bypassed. Why? Back-up Systems Are Not a Guarantee

14. The back-up systems were bypassed because the system was in a maintenance state. This particular combination of inputs was not anticipated to occur when the system was in maintenance mode. Complex Interactions: States and Inputs

15. Dramatic growth in number of knowledgeable experts Potential to insert incorrect data or Denial of Service attacks High leverage / low risk Tempting Target

16. Neural Net Intrusion Detection ADP Robust Controls Combinatorial Optimization for reconfigurability Computational Intelligence Tools Can Help

17. RBFNs can be used for misuse and anomaly detection using sequences of system calls Data are obtained from 1998 DARPA Intrusion Detection Evaluation program Also collaboration with Sandia Red Team Intrusion Detection with Neural Nets

18. RBFNN Generalization on unknown test data

19. PNN + ADABOOST

20. Multi-Machine Power System Control

21. Dw Dw 5 1 2 4 Governor Governor 1 900 Km 2 G1 G2 S S Turbine Turbine Exciter Exciter P 900 Km ref1 900 Km V P e1 ref2 V V e2 t1 AVR V t2 V AVR ref2 3 V ref1 G3 Multi-Machine Power System with Conventional Controllers

22. P P Governor ref2 5 ref1 4 Governor 1 900 Km 2 G1 G2 S S Turbine Turbine Exciter D P Exciter 1 900 Km 900 Km D P V 2 f1 V f2 D w V D w V V 2 t2 S 1 t1 ref1 V S ref2 3 V D TDL TDL TDL TDL ref1 D V ref2 Neurocontoller Neurocontoller G3 Multi-Machine Power System with DHP Neurocontrollers

23. Yref PLANT Y(t) TDL ACTION Neural Network A(t)  (t+1) MODEL Neural Network CRITIC Neural Network TDL TDL TDL  + +  +  - - - EC2(t) MODEL Neural Network CRITIC Neural Network TDL TDL DHP Critic Network Adaptation

24. 1.08 1.07 1.06 1.05 1.04 Terminal voltage in pu 1.03 1.02 1.01 AVR 1 DHP 0.99 0.98 1 2 3 4 5 6 7 8 Time in seconds Terminal Voltage of Generator G2 for a 5% Step Change in its Desired Terminal Voltage & Operating Point Changed

25. -3 x 10 1.5 1 0.5 0 Speed deviation of G1 in pu -0.5 Conventional -1 Neurocontroller -1.5 0 1 2 3 4 5 6 7 8 Time in seconds Speed Deviation of Generator G2 - Operating Point Changed

26. Great benchmark NP – complete Maps to other NP – complete problems Public databases Big need – get learning capability of NN without brittleness of other techniques. Traveling Salesman Problem

27. Previous contributions -- disappointing

28. Divide problem into clusters using ART in O(n) Use Lin-Kernighan algorithm for global tour Use Lin-Kernighan algorithm for local tours Merge local tours in O(n) time Global operations limited to O(n) time Clustered Traveling Salesman

29. Algorithm Overview cluster LK O(k2.2) cluster LK O(k2.2) ART O(n lg n) cluster LK O(k2.2) Merge Clusters O(n) Read problem from file O(n) Result

30. Implemented in C++ thread-safe code Uses Windows threads for parallelism Operating System-specific code isolated to one file Should be easy to port to other parallel systems Implementation

31. #cities Tour Length 1P Time 2P Time Vig factor % off Speedup 1000 2.58E+07 0.422 0.281 0.7 10.40% 1.50 2000 3.61E+07 1.031 0.672 0.7 10.64% 1.53 8000 7.14E+07 8.328 4.281 0.72 10.97% 1.95 10000 7.97E+07 11.359 7.297 0.75 10.57% 1.56 20000 1.12E+08 24.641 14.406 0.8 10.53% 1.71 250000 4.00E+08 315.078 209.687 0.92 11.64% 1.50 1000000 7.94E+08 1468.165 986.48 0.97 11.03% 1.49 10000000 2.52E+09 10528.7 0.98 1.27% CONCORDE 1000 2.34E+07 1.670 2000 3.26E+07 3.500 8000 6.43E+07 26.570 10000 7.20E+07 37.620 20000 1.01E+08 84.830 250000 3.58E+08 1379.540 1000000 7.15E+08 9013.53 10000000 2.495E+09 43630.7

32. 1k

33. 4k

34. 8k

35. 10k

36. 20k

37. 50k

38. 85k

39. 150k

40. 250k

41. 1 M

42. Continued Scaling Results Parallelizability Memory Management Even better news…

43. Clear Need for more advanced architectures Especially to Learn from Experience Cellular Structures necessary Same with SRNs Therefore, combine them and ACDs BUT – To Move Beyond

44. Obviously achieve dynamic behaviors Possible similarity to adaptive systems but with fixed parameters Simultaneous recurrent nets particularly challenging, esp. architectures Recurrent Nets

45. Generalized Maze Problem • Graph Theoretic Representation • SRN Necessary (Werbos & Pang, ’96 & ’98) • Cellular structure – scaling • Closed form now • Convergence time now • Importance of design principles

46. Design from output backward Require for the output node: x16 = (x2 / x1)[min{x6, x5, x4, x3} + 1]. This is a known SRN!

47. Cellular SRN Structure Complete Output J = (x2/x1) * sum = x16(a,b) S +1 / * Product Nodes Current Node inputs Feedback inputs Neighbor node inputs (Occurs at each node (a,b) in maze.)

48. Analyze worst-case convergence WCT = N2 - 2N + N - 3 = N2 - N - 3. Note that this is convergence in J steps. Also true for N x N maze by simple induction proof.

49. Power networks inherit the full range of “systems of systems” issues. These are amenable to computational intelligence solutions: Detection Robust Control Reconfigurability Combinatorial Optimization Conclusions