The Internet Motion Sensor: A Distributed Blackhole Monitoring System

1. The Internet Motion Sensor:A Distributed Blackhole Monitoring System Michael Bailey*, Evan Cooke*, Farnam Jahanian*†, Jose Nazario†, David Watson* Presenter: Anup Goyal

2. What is a Blackhole?

3. Telescope/ Blackhole/ DarkNet/ Sink • Monitoring of unused Address space is very useful • CAIDA-Network Telescope • Internet Motion Sensor – blackhole • Team Cymru – DarkNets • IUCC/IDC Internet Telescope • Isink • Investigating DDOS • Tracking Worms • Characterizing emerging Internet Thread

4. Motivation Passive Monitoring has been around for a while, why build another one? • Need broader coverage • Need to be able to differentiate threats Our Solutions: • Distributed Monitoring Infrastructure Increas coverage of threat activity • Lightweight Active Responder Capture payloads to differentiate threats • Payload signature and caching Improve performance and scalability

5. Why do you need to be distributed?

6. Local Preference across sensors

7. Distributed Monitoring Infrastructure

8. Differentiate Threats?

9. Why not Build Service Modules?

11. Lightweight Active Responder • Goal: obtain enough fidelity to differentiate threats with the minimum resource cost • TCP needs to establish a connection before data is sent • Use Lightweight SYN-ACK active Responder

12. Blaster Worm – Live Host

13. Blaster Worm – Passive Monitor

14. Blaster Worm - IMS

15. Active Responder Limitations • Application-level responder may be required to differentiate threats (e.g. Agobot) • Threats like Agobot can be differentiated using scanning behaviour • Sensors can be fingerprinted and avoided • IMS focused on globally scoped threats (threat model does not include targated manual attacks) • Many sensors of different sizes in many networks near live hosts makes avoidance very hard • Rotate active responders within address block

16. Payload Signature and Caching

17. IMS Deployment • Initial /8 deployment in 2001. Currently 60 address blocks at 18 networks on 3 continent • Tier 1 ISPs, Large Enterprise, Broadband, Academic, National ISPs, Regional ISPs

18. Deployment Statistics • 17,094,016 IPs monitored • 27 /8 block with an IMS sensor • 1.25% of routed IPv4 space • 21% of all routable /8 blocks have at least one sensor IMS Has provided insight into: • Internet Worms • Reconnaissance/Scanning • DDos

19. Internet Worms

20. Scanning

21. SCO DDoS

22. Summary & Outreach Summary: • IMS provides a lightweight but effective platform for tracking, and characterizing Internet threats • Successful deployment on 60 address blocks at 18 organization Outreach: • IMS currently used daily by operator community to investigate new threats • We provide reports and forensics to NSP-SEC • We are now focusing academic networks • Data traces will be available through DHS PREDICT project

23. Scalability, Fidelity, and Containment in thePotemkin Virtual Honeyfarm Michael Vrable, Justin Ma, Jay Chen, David Moore, Erik Vandekieft, Alex C. Snoeren, Geoffrey M. Voelker,Stefan Savage Presenter: Anup Goyal

24. Background • Large-scale host exploitation a serious problem • Worms, viruses, bots, spyware… • Supports an emerging economic criminal enterprise • SPAM, DDoS, phishing, piracy, ID theft… • Quality and sophistication of malware increasing rapidly

25. Motivation • Intelligence about new threats is critical for defenders • Principal tool is the network honeypot • Monitored system deployed for the purpose of being attacked • Honeyfarm: Collection of honeypots • Provide early warning, accurate inference of global activity,cover wide range of software • Design issues • Scalability: How many honeypots can be deployed • Fidelity: How accurately systems are emulated • Containment: How well innocent third parties are protected • Challenge: tension between scalability and fidelity

26. Honeyfarm Scalability/Fidelity Tradeoff

27. Approach • Dedicated honeypot systems are overkill • Can provide the illusion of dedicated systems via aggressive resource multiplexing at network and host levels

28. Network-level Multiplexing • Most addresses don’t receive traffic most of the time • Apply late binding of IP addresses to honeypots • Most traffic that is received causes no interesting effects - Allocate honeypots only long enough to identify interesting behavior • Recycle honeypots as soon as possible • How many honeypots are required? - For a given request rate, depends upon recycling rate

29. Effectiveness of Network-Level Multiplexing

30. Host-Level Multiplexing • CPU utilization in each honeypot quite low (milliseconds to process traffic) • Use VMM to multiplex honeypots on a single physical machine • Few memory pages actually modified when handling network data • Share unmodified pages among honeypots within a machine • How many virtual machines can we support? - Limited by unique memory required per VM

31. Effectiveness of Host-Level Multiplexing Further 2-3 order of magnitude improvement

32. The Potemkin Honeyfarm Architecture • Two components: • Gateway • VMM • Basic operation: • Packet received by gateway • Dispatched to honeyfarm server • VM instantiated • Adopts IP address

33. Potemkin VMM Requirements • VMs created on demand • VM creation must be fast enough to maintain illusion • Many VMs created - Must be resource-efficient

34. Potemkin VMM Overview • Modified version of Xen 3.0 (pre-release) • Flash cloning • Fork copies from a reference honeypot VM • Reduces VM creation time—no need to boot • Applications all ready to run • Delta virtualization • Copy-on-write sharing (between VMs) • Reduces per-VM state—only stores unique data • Further reduces VM creation time

35. Flash Cloning Performance • Time required to clone a 128 MB honeypot: Control tools overhead 124 ms Low-level clone 11 ms Device setup 149 ms Other management overhead 79 ms Networking setup & overhead 158 ms Total 521 ms • 0.5 s already imperceptible to external observers unless looking for delay, but we can do even better

36. Delta Virtualization Performance • Deployed using 128 MB Linux honeypots • Using servers with 2 GB RAM, have memory available to support ~ 1000 VMs per physical host • Currently tested with 100 VMs per host • Hits artificial resource limit in Xen, but this can be fixed

37. Containment Policies • Must also care about traffic going out • We deliberately run unpatched, insecure software in honeypots • Containment: Should not permit attacks on third parties • As with scalability, there is a tension between containment and fidelity • Various containment policies we support: • Allow no traffic out • Allow traffic over established connections • Allow traffic back to original host

38. Containment Implementation in Gateway • Containment policies implemented in network gateway • Tracks mappings between IP addresses, honeypots, • connections • Modular implementation in Click • Gateway adds insignificant overhead ( 1 ms)

39. Traffic Reflection Example gateway policy: Redirect traffic back to Honeyfarm • Packets sent out to third parties. . . • . . .may be redirected back into honeyfarm Reuses honeypot creation functionality

40. Challenges • Honeypot detection • If malware detects it is in a honeypot, may act differently • How easy it is to detect virtualization? • VMware detection code used in the wild • Open arms race between honeypot detection and camouflage • Resource exhaustion • Under high load, difficult to maintain accurate illusion • Large-scale outbreak • Honeypot denial-of-service • Challenge is intelligently shedding load

41. Summary • Can achieve both high fidelity and scalability • Sufficient to provide the illusion of scale • Potemkin prototype: 65k addresses -> 10 physical hosts • Largest high-fidelity honeypot that we are aware of • Provides important tool for study of and defenses against malware