CSC 382: Computer Security

1. CSC 382: Computer Security Network Monitoring CSC 382: Computer Security

2. Topics • Principles • Models of Intrusion Detection • False Positives • Architecture of an IDS • IDS Deployment • Active Response (IPS) • Host-based IDS and IPS CSC 382: Computer Security

3. Principles of Intrusion Detection Characteristics of systems not under attack • User, process actions conform to statistically predictable pattern. • User, process actions do not include sequences of actions that subvert the security policy. • Process actions correspond to a set of specifications describing what the processes are allowed to do. Systems under attack do not meet at least one. CSC 382: Computer Security

4. Example Goal: insert a back door into a system • Intruder will modify system configuration file or program. • Requires privilege; attacker enters system as an unprivileged user and must acquire privilege. • Nonprivileged user may not normally acquire privilege (violates #1). • Attacker may break in using sequence of commands that violate security policy (violates #2). • Attacker may cause program to act in ways that violate program’s specification (violates #3). CSC 382: Computer Security

5. Goals of IDS • Detect wide variety of intrusions • Previously known and unknown attacks. • Need to adapt to new attacks or changes in behavior. • Detect intrusions in timely fashion • May need to be be real-time, especially when system responds to intrusion. • Problem: analyzing commands may impact response time of system. • May suffice to report intrusion occurred a few minutes or hours ago. CSC 382: Computer Security

6. Goals of IDS • Present analysis in easy-to-understand format. • Ideally a binary indicator. • Usually more complex, allowing analyst to examine suspected attack. • User interface critical, especially when monitoring many systems . • Be accurate • Minimize false positives, false negatives. • Minimize time spent verifying attacks, looking for them. CSC 382: Computer Security

7. Deep Packet Inspection • IDS requires, some firewalls do too. • DPI = Analysis of Application Layer data • Protocol Standard Compliance • Is port 53 traffic DNS or a covert shell session? • Is port 80 traffic HTTP or tunneled IM or P2P? • Protocol Anomaly Detection • Traffic is valid HTTP. • But suspicious URL contains directory traversal. CSC 382: Computer Security

8. Models of Intrusion Detection • Anomaly detection • What is usual, is known. • What is unusual, is bad. • Misuse detection • What is bad is known. • Look for what is bad, hope it doesn’t change. CSC 382: Computer Security

9. Anomaly Detection Analyzes a set of characteristics of system, and compares their values with expected values; report when computed statistics do not match expected statistics. • Threshold metrics • Statistical moments • Markov model CSC 382: Computer Security

10. Threshold Metrics • Counts number of events that occur • Between m and n events (inclusive) expected • If number falls outside this range, anomalous. • Example • Windows: lock user out after k failed sequential login attempts. Range is (0, k–1). • k or more failed logins deemed anomalous • Threshold depends on typing skill. CSC 382: Computer Security

11. Sequences of System Calls • Define normal behavior in terms of sequences of system calls. • Example normal trace: open read write open write close • Doesn’t normally run other programs. • Attack trace: open read write open exec write close CSC 382: Computer Security

12. Finding Features • Which features best show anomalies? • CPU use may not, but I/O use may. • Use training data • Anomalous data marked. • Feature selection program picks features, clusters that best reflects anomalous data. • Use compiler techniques to build program model. CSC 382: Computer Security

13. Misuse Detection • Determines whether a sequence of instructions being executed is known to violate the site security policy. • Descriptions of known or potential exploits grouped into rule sets. • IDS matches data against rule sets; on match, potential attack found. • Cannot detect new attacks: • No rules to cover them. CSC 382: Computer Security

14. Example: snort Network Intrusion Detection System • Sniffs packets off wire. • Checks packets for matches against rule sets. • Logs detected signs of misuse. • Alerts adminstrator when misuse detected. CSC 382: Computer Security

15. Snort Rules • Rule Header • Action: pass, log, alert • Network Protocol • Source Address (Host or Network) + Port • Destination Address (Host or Network) + Port • Rule Body • Content: packet ASCII or binary content • TCP/IP flags and options to match • Message to log, indicating nature of misuse detected CSC 382: Computer Security

16. Snort Rule Example Example: rule for ssh shell code exploit alert tcp $EXTERNAL_NET any -> $HOME_NET 22 (msg:"EXPLOIT ssh CRC32 overflow NOOP"; flow:to_server,established; content:"|90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90|"; reference:bugtraq,2347; reference:cve,CVE-2001-0144; classtype:shellcode-detect; sid:1326; rev:3;) CSC 382: Computer Security

17. Comparison and Contrast • Misuse detection: if all policy rules known, easy to construct rulesets to detect violations. • Usual case is that much of policy is unspecified, so rulesets describe attacks, and are not complete. • Anomaly detection: detects unusual events, but these are not necessarily security problems. CSC 382: Computer Security

18. False Positives • A new test for a disease that is 95% accurate • Assume 1 in 1000 people have disease. • Should everyone get the test? • Sample size: 1000 • Expect 0.95 + (999 * 0.05) positives • Ergo, 50 people will be told they have disease • If you test positive, only 2% chance you have it. CSC 382: Computer Security

19. IDS Architecture An IDS is essentially a sophisticated audit system • Agentgathers data for analysis. • Director analyzes data obtained from the agents according to its internal rules. • Notifier acts on director results. • May simply notify security officer. • May reconfigure agents, director to alter collection, analysis methods. • May activate response mechanism. CSC 382: Computer Security

20. Agents Obtain information and sends to director. Preprocessing • Simplifying and reformatting of data. Push vs Pull • Agents may push data to Director, or • Director may pull data from Agents. CSC 382: Computer Security

21. Host-Based Agents • Obtain information from logs • May use many logs as sources. • May be security-related or not. • May use virtual logs if agent is part of the kernel. • Agent generates its information • Analyzes state of system. • Treats results of analysis as log data. CSC 382: Computer Security

22. Network-Based Agents • Sniff traffic from network. • Use hubs, SPAN ports, or taps to see traffic. • Need agents on all switches to see entire network. • Agent needs same view of traffic as destination • TTL tricks, fragmentation may obscure this. • End-to-end encryption defeats content monitoring • Not traffic analysis, though. CSC 382: Computer Security

23. Aggregation of Information Agents produce information at multiple layers of abstraction. • Application-monitoring agents provide one view of an event. • System-monitoring agents provide a different view of an event. • Network-monitoring agents provide yet another view (involving many packets) of an event. CSC 382: Computer Security

24. Director • Reduces information from agents • Eliminates unnecessary, redundent records. • Analyzes information to detect attacks • Analysis engine can use any of the modelling techniques. • Usually run on separate system • Does not impact performance of monitored systems. • Rules, profiles not available to ordinary users. CSC 382: Computer Security

25. Example • Jane logs in to perform system maintenance during the day. • She logs in at night to write reports. • One night she begins recompiling the kernel. • Agent #1 reports logins and logouts. • Agent #2 reports commands executed. • Neither agent spots discrepancy. • Director correlates log, spots it at once. CSC 382: Computer Security

26. Adaptive Directors • Modify profiles, rulesets to adapt their analysis to changes in system • Usually use machine learning or planning to determine how to do this. • Example: use neural nets to analyze logs • Network adapted to users’ behavior over time. • Used learning techniques to improve classification of events as anomalous. • Reduced number of false alarms. CSC 382: Computer Security

27. Notifier • Accepts information from director • Takes appropriate action • Notify system security officer • Respond to attack • Often GUIs • Use visualization to convey information. CSC 382: Computer Security

28. Example Architecture: snort CSC 382: Computer Security

29. IDS Deployment IDS deployment should reflect your threat model. Major classes of attackers: • External attackers intruding from Internet. • Internal attackers intruding from your LANs. Where should you place IDS systems? • Perimeter (outside firewall) • DMZ • Intranet • Wireless CSC 382: Computer Security

30. IDS Deployment CSC 382: Computer Security

31. Sguil NSM Console CSC 382: Computer Security

32. Intrusion Prevention Systems • What else can you do with IDS alerts? • Identify attack before it completes. • Prevent it from completing. • How to prevent attacks? • Directly: IPS drops attack packets. • Indirectly: IPS modifies firewall rules. • Is IPS a good idea? • How do you deal with false positives? CSC 382: Computer Security

33. IPS Deployment Types Inline Intranet IPS Non-Inline IPS Intranet CSC 382: Computer Security

34. Active Responses by Network Layer • Data Link: Shut down a switch port. Only useful for local intrusions. Rate limit switch ports. • Network: Block a particular IP address. • Inline: can perform blocking itself. • Non-inline: send request to firewall. • Transport: Send TCP RST or ICMP messages to sender and target to tear down TCP sessions. • Application: Inline IPS can modify application data to be harmless: /bin -> /ben CSC 382: Computer Security

35. Host IDS and IPS • Anti-virus and anti-spyware • AVG anti-virus, SpyBot S&D • Log monitors • swatch, logwatch • Integrity checkers • tripwire, osiris, samhain • Application shims • mod_security CSC 382: Computer Security

36. Key Points • Intrusion detection is a form of auditing. • Models of IDS: • Anomaly detection: unexpected events. • Misuse detection: violations of policy. • The problem of false positives. • IDS Architecture: • Agents. • Director. • Notifiers. • Active Response: Intrusion Prevention Systems CSC 382: Computer Security

37. References • Richard Bejtlich, The Tao of Network Security Monitoring, Addison-Wesley, 2004. • Matt Bishop, Computer Security: Art and Science, Addison-Wesley, 2003. • Brian Caswell, et. al., Snort 2.0 Intrusion Detection, Snygress, 2003. • William Cheswick, Steven Bellovin, and Avriel Rubin, Firewalls and Internet Security, 2nd edition, 2003. • The Honeynet Project, Know Your Enemy, 2nd edition, Addison-Wesley, 2004. • Richard A. Kemmerer and Giovanni Vigna, “Intrusion Detection: A Brief History and Overview,” IEEE Security & Privacy, v1 n1, Apr 2002, pp 27-30. • Steven Northcutt and Julie Novak, Network Intrusion Detection, 3rd edition, New Riders, 2002. • Michael Rash et. al., Intrusion Prevention and Active Response, Syngress, 2005. • Rafiq Rehman, Intrusion Detection Systems with Snort: Advanced IDS Techniques Using Snort, Apache, MySQL, PHP, and ACID, Prentice Hall, 2003. • Ed Skoudis, Counter Hack Reloaded 2/e, Prentice Hall, 2006. • Ed Skoudis and Lenny Zeltser, Malware: Fighting Malicious Code, Prentice Hall, 2003. CSC 382: Computer Security