15-744: Computer Networking

1. 15-744: Computer Networking L-24 Network Intrusion Detection Systems (NIDS)

2. NIDS • Background • Bro: A NIDS • NIDS Traffic Normalization • Honeycomb: NIDS signature generation

3. Recall: Network “101” vs. Reality Traditional view: “Dumb” network Reality: Lots of in-network processing Appliances or Middleboxes:IDS, Firewall, Proxies, Load balancers….

4. Recall: Middleboxes Galore! Data from a large enterprise Survey across 57 network operators APLOMB (SIGCOMM’13)

5. Firewalls • Network-level firewalls: • Limit access to the network • Installed at perimeter of the network • Allows traffic specified in the policy • Drops everything else Internal Network Firewall Internet

6. Typical Firewall Configuration Internet • Internal hosts can access DMZ and Internet • External hosts can access DMZ only, not Intranet • DMZ hosts can access Internet only • Advantages? • If a service gets compromised in DMZ it cannot affect internal hosts DMZ X X Intranet

7. Intrusion Detection Systems (IDS) • Firewalls allow traffic only to legitimate hosts and services • Traffic to the legitimate hosts/services may contain attacks • Solution? • Intrusion Detection Systems • Monitor data and behavior • Report when identify attacks

8. Internet Internet Firewall vs. NIDS • Firewall • Active filtering • Fail-close • Network IDS • Passive monitoring • Fail-open • Advantages and disadvantages? FW NIDS

9. Types of IDS Signature-based Anomaly-based Host-based Network-based

10. Signature-based IDS • Characteristics: • Uses known pattern matching to signify attack • Can identify intrusions from packet header/data • May use Boolean operators in rule set • ‘this_string’ • ‘this_variable’ AND ‘that_number’ • ‘this_string’ AND ‘that_variable’ NOT ‘that_tcp_flag’

11. Signature-based IDS • Advantages • Widely available • Easy to implement • Easy to update • Disadvantages • Cannot detect attacks for which it has no signature • Must be updated for each new attack and attack variant • Lag time from new exploit to update can be dangerous • ‘New’ attack variant can be created by changing a single string • May be resource intensive

12. Anomaly-based IDS • Characteristics • Uses statistical models or a machine learning engine to characterize normal usage behaviors • Recognizes departures from normal as potential intrusions

13. Anomaly-based IDS • Advantages • Can detect attempts to exploit new and unforeseen vulnerabilities • Can recognize unusual traffic based on a number of characteristics: • Payload • Source address • Time • Can recognize authorized usage that falls outside the normal pattern • Disadvantages • Generally slower, more resource intensive compared to signature-based IDS • Greater complexity, difficult to configure • Higher percentages of false alerts • Link between abnormal and intrusive may be weak

14. Network-based IDS • Network monitor • Passively captures traffic and inspects it • Can also function in a client-server model • Sensors are located on multiple machines across the network • All sensors feed data to console • Console machine handles logging and alerting

15. Network-based IDS • Advantages • Positioned properly, can test effectiveness of firewalls, router access lists, etc. • Can monitor multiple machines from one physical and logical location • Console can generate an alert if a monitored machine/network has ceased to send information • Disadvantages: • Since it is capturing all network packets, can produce large log/alert files • Can be difficult to cull through vast amount of information • Console machine generally must be quite powerful, similar to a workgroup server • If console machine goes down then multiple machines may be left unmonitored • Communication from sensors to console may increase overall network traffic levels

16. NIDS • Background • Bro: A NIDS • NIDS Traffic Normalization • Honeycomb: NIDS signature generation

17. Bro: Detecting intruders in real-time • Bro is a standalone NIDS developed by Vern Paxson • Designed to keep LBL an open environment (to resist the need to install a firewall) • Goals • High-speed monitoring, no packet drops • Real-time notification • Separate mechanism from policy • Extensibility • Simple to use, guard against mistakes • Tolerate attacks on NIDS • More powerful than Snort, but less popular. Why? • Misuse detection (signature-based) or anomaly detection (specification-based or statistical-based)? Paxon, 1998

18. Policy script Alerts/notifications Policy Script Interpreter Event control Event stream Event Engine Filtered packet stream tcpdump filters libpcap Packet stream Network Bro system architecture Complexity of operations increases Volume of data decreases

19. libpcap Layer • Only passes relevant packets to Event Engine • Uses BSD Packet Filter (BPF) to efficiently filter packets • Filter rules • tcp port finger or tcp port ftp or tcp port telnet or port 111 or tcp[13] & 7 != 0

20. Reminder: TCP header format

21. Event engine • State for each connection, based on <SrcIP, SrcPort, DstIP, DstPort> • If state not present, allocate fresh state • TCP processing • Update state based on SYN/FIN/RST flags • Process acknowledgment • SYN generates a timer event, if nothing happens after 5 min, generate connection_attempt event • UDP processing • Initial packets generate udp_request and udp_reply events

22. Policy script interpreter • Clear separation of event generation and response to achieve clear separation between mechanism and policy • Advantage: extensibility (adding a new protocol analyzer and new event handler usually separate from other components) • Event are stored in a FIFO queue and processed sequentially • Policy script interpreter executes event handlers

23. Bro policy scripts • Goal: A clear and error-free language • Written in a specialized language: • Network types (IP addresses, connections, protocol, etc.) • Typed constants, variables • Network operators (comparison, ranges, etc.) • Control statements (IF/THEN, etc.) • Regular expressions • It can • Generate alerts • Call exterior programs

24. Bro policy scripts: variables and operators if ([H, S] in allowed_services) … it’s okay …

25. Offline Analysis • Checkpointing in order to • Reclaim memory of dormant connections • Offline-analysis • Logs maintained for a long time for • Forensics on past break-ins • Complex analysis that would be too expensive to be real-time

26. Attacks on Bro • Overload attack • Send packets that match filters • Send packet streams that generate events • Try to generate events that lead to recording to disk • Defense strategy 1: Assume policy script is secret. • Good assumption? • Defense strategy 2: Lower the load (e.g., stop capturing HTTP traffic) • Effective if attacker does not know how Bro lowers load

27. Attacks on Bro monitor • Crash attack • Find packet sequence that crashes monitor • Exhaust memory, disk resources • Defense strategy 1: Careful code analysis • Defense strategy 2: OS-level watchdog timer with subsequent packet capturing (sacrificing real-time detection)

28. Attacks on Bro monitor • Subterfuge attack • Mislead Bro: find traffic patterns that Bro and the end system interpret differently • Example: Carefully setting TTL field • Defense strategy: Traffic normalization

29. NIDS • Background • Bro: A NIDS • NIDS Traffic Normalization • Honeycomb: NIDS signature generation

30. NIDS: Evasion andNormalization • Problems • NIDS only has partial knowledge of what traffic the host sees (e.g., TTL expires, MTU) • Ambiguities in TCP/IP (e.g., Overlapping IP & TCP fragments) • Different OS implement standard differently • Approach: traffic normalization Handely et al., 2001

31. NIDS sees: A T T I A C K Internet NIDS Host End-host sees: Attacker’s data stream A T T I A C K A T T A C K Small TTL attack same TCP seq #, “I” has short TTL

32. NIDS sees: A T T A I C K Internet NIDS Host End-host sees: Attacker’s data stream A T T A I C K A T T A I C K same TCP seq # or same IP frag offset Fragmentation overlap attack

33. Internet Approach: traffic normalizer • Introduce “bump in the wire”: traffic normalizer to evade protocol ambiguities NIDS Normalizer Host

34. Alternative approaches • Host-based IDS • Loses the advantages of monitoring the entire site cheaply • Major deployment and management efforts • Detailed Intranet map • Required knowledge of every OS and app • Bifurcating analysis • If the NIDS does not know which of the two interpretations the end system may apply to an input packet, split the analysis context • State explosion? • Aside: New opportunities given SDN/NFV?

35. NIDS • Background • Bro: A NIDS • NIDS Traffic Normalization • Honeycomb: NIDS signature generation

36. Honeycomb: Motivation • NIDS work based on signatures • How to generate signatures to begin with? • Common practice is manual and expertise-based • Can we do better? Kreibich and Corwcroft, 2004

37. Honeycomb: Background • Good NIDS signatures should be • Narrow enough; otherwise, high false positives • Flexible enough; otherwise, high false negatives • Honeypots: decoy computer resources to detect or counteract computer resources • Examples: dummy database items, dummy web servers, … • Key Idea behind Honeycomb: • Traffic sent to a honeypot should be malicious • We can extract its pattern and use it as a NIDS signature

38. Honeycomb: Architecture • Good NIDS signatures:

39. Summary • Network intrusion detection systems is a complement to a firewall • Goal: Finding malicious traffic both low false positives and false negatives • Decoupling policy from analysis is important • Protocols are ambiguous • Unclear how end-hosts implement the ambiguous parts • Can be used to evade NIDS • Possible solution: Protocol normalization (a.k.a. protocol scrubbing) • Opportunities for automatic signature generation