1 / 74

Real-time Traffic monitoring and containment

Real-time Traffic monitoring and containment. A. L. Narasimha Reddy Dept. of Electrical Engineering Texas A & M University reddy@ee.tamu.edu http://ee.tamu.edu/~reddy/. Outline. Motivation DOS attacks Partial state routers DDOS attacks, worms Aggregate Packet header data as signals

Download Presentation

Real-time Traffic monitoring and containment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Real-time Traffic monitoring and containment A. L. Narasimha Reddy Dept. of Electrical Engineering Texas A & M University reddy@ee.tamu.edu http://ee.tamu.edu/~reddy/

  2. Outline • Motivation • DOS attacks • Partial state routers • DDOS attacks, worms • Aggregate Packet header data as signals • Signal/image based anomaly/attack detectors Texas A & M University

  3. Real-time traffic monitoring • Attacks motivate us to monitor network traffic • Potential anomaly/attack detectors • Potentially contain/throttle them as they happen • Line speeds are increasing • Need simple, effective mechanisms • Attacks constantly changing • CodeRed yesterday, MyDoom today, what next Texas A & M University

  4. Motivation • Most current monitoring/policing tools are tailored to known attacks • Look for packets with port number 1434 (CodeRed) • Contain Kaaza traffic to 20% of the link • Become ineffective when traffic patterns or attacks change • New threats are constantly emerging Texas A & M University

  5. Motivation • Can we design generic (and generalized) mechanisms for attack detection and containment? • Can we make them simple enough to implement them at line speeds? Texas A & M University

  6. Introduction • Why look for Kaaza packets • They consume resources • Consumes resources more than we want • Not much different from DOS flood • Consumes resources to stage attacks • Why not monitor resource usage? • Do not want to rely on attack specific info Texas A & M University

  7. Attacks • DOS attacks • Few sources = resource hogs • DDOS attacks, worms • Many sources • Individual flows look normal • Look at the aggregate picture Texas A & M University

  8. DOS attacks & Network Flows • Too many flows to monitor each flow • Maintain a fixed amount of state/memory • State not enough to monitor all flows (Partial state) • Manage the state to monitor high-bandwidth flows • How? • Sample packets • High-BW flows more likely to be selected • Use a cache and employ LRU type policy • Traffic driven • Cache retains frequently arriving flows Texas A & M University

  9. Partial State Approach • Similar to how caches are employed in computer memory systems • Exploit locality • Employ an engineering solution in an architecture-transparent fashion Texas A & M University

  10. Identifying resource hogs • Lots of web flows • Tend to corrupt the cache quickly • Apply probabilistic admission into cache • Flow has to arrive often to be included in cache • Most web flows not admitted • Works well in identifying high-BW flows • Can apply resource management techniques to contain cached/identified flows Texas A & M University

  11. LRU with probabilistic admission • Employ a modified LRU • On a miss, flow admitted with probability p • When p is small, keeps smaller flows out • High-BW flows more likely admitted • Allows high-BW flows to be retained in cache • Nonresponsive flows more likely to stay in cache Texas A & M University

  12. Traffic Driven State Management • Monitor top 100 flows at any time • Don’t know the identity of these flows • Don’t know how much BW these may consume Texas A & M University

  13. Policy Driven State Management • An ISP could decide to monitor flows above 1Mbps • Will need state >= link capacity/1 Mbps • Could monitor flows consuming more than 1% of link capacity • For security reasons • At most 100 flows with 1% BW consumption Texas A & M University

  14. Partial State –Trace-driven evaluation Texas A & M University

  15. Partial State –Trace-driven Evaluation Texas A & M University

  16. UDP Cache Occupancy Texas A & M University

  17. TCP Cache Occupancy Texas A & M University

  18. Resource Management Texas A & M University

  19. Preferential Dropping 1 drop prob maxp minth maxth Queue length drop prob for high bandwidth flows drop prob for other flows Texas A & M University

  20. Multiple possibilities • SACRED: Monitor flows above certain rate (policy driven), differential RED, (iwqos99) • LRU-RED: Traffic driven state management, differential RED (Globecom01) • Approximately fair BW distribution • LRU-FQ: Traffic driven state management, fair queuing (ICC 04) • Contain DOS attacks • Provide shorter delays for short-term flows Texas A & M University

  21. LRU-FQ Resource Management Texas A & M University

  22. LRU-FQ flow chart – enqueue event Does Cache Have space? Is Flow in Cache? No No Admit flow with Probability ‘p’ Packet Arrival Yes Yes Is Flow Admitted? Record flow details Initialize ‘count’ to 0 Yes Increment ‘count’ Move flow to top of cache No Is ‘count’ >= ‘threshold’ No Yes Enqueue in Normal Queue Enqueue in Partial state Queue Texas A & M University

  23. Linux IP Packet Forwarding Local packet Deliver to upper layers UPPER LAYERS Route to destination Update Packet Error checking Verify Destination IP LAYER Packet Enqueued Scheduler invokes Bottom half Design space Scheduler runs Device driver LINK LAYER Request Scheduler To invoke bottom half Device Prepares packet Packet Departure Packet Arrival Check & Store Packet Enqueue pkt Texas A & M University

  24. Linux Kernel traffic control • Filters are used to distinguish between different classes of flows. • Each class of flows can be further categorized into sub-classes using filters. • Queuing disciplines control how the packets are enqueued and dequeued Texas A & M University

  25. LRU-FQ Implementation • LRU component of the scheme is implemented as a filter. • All parameters: threshold, probability and cache size are passed as parameters to the filter • Fair Queuing employed as a queuing discipline. • Scheduling based on queue’s weight. • Start-time Fair Queuing Texas A & M University

  26. LRU-FQ - Results

  27. Experimental Setup Texas A & M University

  28. Long-Term flow differentiation Normal TCP fraction = 0.07 Probability = 1/25 Cache size= 11 threshold= 125 Texas A & M University

  29. Long-term flow differentiation Probability = 1/25 Cache size= 11 threshold= 125 Texas A & M University

  30. Protecting Web Mice Texas A & M University

  31. Long Term TCP Flows 20 LongTerm UDP Flows 2 – 4 Web Clients 20 Probability 1/50 Threshold 125 LRU Cache Size 11 LRU : Normal Queue 1:1 Protecting Web mice Experimental Setup Texas A & M University

  32. UDP Flows UDP Flows UDP Tput UDP Tput # Web Requests # Web Requests TCP Tput TCP Tput TCP Fraction TCP Fraction 2 2 45.73 89.45 1313 13915 44.92 5.88 0.062 0.49 3 3 45.73 89.80 13828 1284 5.55 44.83 0.058 0.49 4 4 46.24 89.13 927 13632 6.21 44.51 0.49 0.065 Protecting Web Mice Bandwidth Results Normal Router LRU-FQ Router Texas A & M University

  33. Protecting Web Mice Timing Results Normal Router LRU-FQ Router Texas A & M University

  34. Summary of LRU-FQ • Provides a good control of DOS attacks with limited number of flows • Provides better delays for short-term flows • Automatically identifies resource hogs • Partial state packet handling cost -not an issue at 100Mbps. Texas A & M University

  35. Applications of Partial State • More intelligent control of network traffic • Accounting and measurement of high bandwidth flows • Denial of Service (DOS) attack prevention • Tracing of high bandwidth flows • QOS routing Texas A & M University

  36. Aggregated packet analysis Texas A & M University

  37. Approach Anomaly Detection (Thresholding) Signal Generation & Data Filtering (Address correlation) Statistical or Signal Analysis (Wavelets or DCT) Detection Signal Network Traffic Texas A & M University

  38. Signal Generation • Traffic volume (bytes or packets) • Analyzed before • May not be a great signal when links are always congested (typical campus access links) • Lot more information in packet headers • Source address • Destination address • Protocol number • Port numbers Texas A & M University

  39. Signal Generation • Per packet cost is important driver • Update a counter for each packet header field • Too much memory to put in SRAM • Break the field into multiple 8-bit fields • 32-bit address into four 8-bit fields • 1024 locations instead of 2^32 locations • In general, 256* (k/8) instead of 2^k • k/8 counter updates instead of 1 Texas A & M University

  40. Signal Generation • What kind of signals can we generate with addresses, port numbers and protocol numbers? Texas A & M University

  41. Addresses are correlated • Most of us have habits • Access same web sites • Large web sites get significant part of traffic • Google.com, hp.com, yahoo.com • Large downloads correlate over time • ftp, video • On an aggregate, addresses are correlated Texas A & M University

  42. Address Correlation –attacks? • Address correlation changes when traffic patterns change abruptly • Denial of service attacks • Flash crowds • Worms • Results in differences in correlation • High --single attack victim • Low – lots of addresses --worm Texas A & M University

  43. Address correlation signals • Address correlation: • Simplified Address correlation: Texas A & M University

  44. Address Correlation Signals Texas A & M University

  45. Address Correlation Signals Texas A & M University

  46. Signal Analysis • Capture information over a sampling period • Of the order of a few seconds to minutes • Analyze each sample to detect anomalies • Compare with historical norms • Post-mortem/Real-time analysis • May use different amounts of data & analysis • Detailed information of past few samples • Less detailed information of older samples Texas A & M University

  47. Signal Analysis • Address correlation as a time series signal • Employ known techniques to analyze time series signals • Wavelets –one powerful technique • Allows analysis in both time and frequency domain • Per-sample analysis has more flexibility • Not in forwarding path Texas A & M University

  48. Does this work? Texas A & M University

  49. Analysis of address signal Texas A & M University

  50. Image based analysis • Treat the traffic data as images • Apply image processing based analysis • Treat each sample as a frame in a video • Video compression techniques lead to data reduction • Scene change analysis leads to anomaly detection • Motion prediction leads to attack prediction Texas A & M University

More Related