Evaluating the Security Threat of Instruction Corruptions in Firewalls

1. Evaluating the Security Threat of Instruction Corruptions in Firewalls Shuo Chen, Jun Xu, Ravishankar K. Iyer, Keith Whisnant Center of Reliable and High Performance Computing Coordinated Science Laboratory University of Illinois at Urbana-Champaign June 24, 2002

2. Objectives • Can transient errors cause security vulnerabilities in firewall software? • Combine fault injection measurement with processor architecture details to develop a SAN model depicting the reliability, performance, and security of the firewall. • Use the SAN model and publicly available security data to assess the relative significance of error-caused security violations.

3. Definitions of Terms • Error-caused security vulnerability occurs when an error results in putting the software in a state where any packet can enter the system unchecked. • Window of vulnerability is the time period during which such a vulnerability persists • Security violation occurs when a number of malicious packets sufficient to launch an actual attack enter the system during a window of vulnerability

4. Errors, Vulnerabilities and Security Violations Window of permanent security vulnerability Window of temporary security vulnerability t2 t3 t5 t6 t7 t8 t1 t4 Time Fault crashes the system Fault crashes the system Fault is not manifested Detected by intrusion detection systems, or system crash by new faults or latent faults Temporary SV Permanent SV Erroneous instruction is evicted from cache Security vulnerability window Error System reboot Malicious packets

5. Fault Injection Experiment Firewall Address Pool Rule: Reject packet from attacker machine. 1 Driver-based Linux Kernel Fault Injector 2 3 Firewall Code Attacker Machine 5 4 Log Firewall machine

6. Outcomes of Fault Injection Experiments • Four categories of outcomes • Not Activated or Not manifested: 78% • CRASH + HANG: 20% • Temporary security vulnerability: disappears when the erroneous location is overwritten, cached out or the system is re-booted. 2% • Permanent security vulnerability: corrupts the semantic or structural integrity of the permanent data structures. Removing the errors does not eliminate the permanent security vulnerability. 0.05% • Fault injection results used as parameters in the SAN model.

7. Error Sub-model Input Gates task switch Workload Sub-model Overview of the SAN Model SAN Model: quantifies the relationship between processor architecture, workload, and error’s characteristics Error sub - model not manifested T_SV error processor flush all error occurrence execution core crash/hang places firewall error reboot execution cache cache fetch cache replacement P_SV maintenance reboot rp _out non - firewall CPU working workload execution firewall enable non - firewall Workload sub-model workload enable job dispatch job packet packet processing non - firewall non - firewall workload workload processing idle idle time

8. 0.78 0.02 0.20 0.0005 Error Sub-Model NA+NM Temp. Security Vulnerability processor execution core error occurrence rate Crash+Hang firewall ex error cache Perm. Security Vulnerability cache fetch cache replacement non-firewall workload ex • Calculate the probability that a token arrives into Temporary Security Vulnerability or Permanent Security Vulnerability places • Calculate the number of packets getting through the firewall in a single vulnerability window

9. Workload Sub-Model job dispatch job packet packet processing non-firewall workload non-firewall workload processing idle idle time

10. Rates of Security Vulnerabilities Average 14.9/year Average 0.37/year Rate of Temporary Security Vulnerability (TSV) with 0.1 Error/Day for 20 Firewall Machines Rate of Permanent Security Vulnerability (PSV) with 0.1 Error/Day for 20 Firewall Machines

11. Size of Vulnerability Windows • Vulnerability window size links security vulnerabilities and security violations • In order to calculate the rates of security violations, we need the distribution of the size of the security vulnerability window Assume 30% packets are malicious

12. Distribution of Number of Packets in a Vulnerability Window Probability of Security Violation, given a security vulnerability P(security violation | security vulnerability)=0.197 Probability Distribution: Processor Utilization by firewall = 50% non-firewall workload=10% malicious packet rate=30%

13. Frequency of Security Violations Rate of Error-Caused Security Violations Rate of Kernel-Related Software Security Bugs

14. Conclusions • There exist error-caused security vulnerabilities in firewall software. • Transient errors can cause permanent security vulnerability. • Errors propagate to permanent data structures. • There is a non-negligible probability that error-caused security vulnerabilities become security violations.

15. Major References D. Stott. Automated Fault-Injection-Based Dependability Analysis of Distributed Computer Systems. Ph.D. Dissertation, UIUC, 2001. A. Ghosh et al. “An Automated Approach for Identifying Potential Vulnerabilities in Software”. IEEE Symp. on Security and Privacy, May 1998. J. Xu, S. Chen, Z. Kalbarczyk, R. Iyer. “An Experimental Study of Security Vulnerabilities Caused by Errors”. IEEE DSN’01. July 2001. http://www.securityfocus.com. 12/30/2001