Real time traffic monitoring and containment
Download
1 / 74

Real-time Traffic monitoring and containment - PowerPoint PPT Presentation


  • 136 Views
  • Uploaded on

Real-time Traffic monitoring and containment. A. L. Narasimha Reddy Dept. of Electrical Engineering Texas A & M University [email protected] http://ee.tamu.edu/~reddy/. Outline. Motivation DOS attacks Partial state routers DDOS attacks, worms Aggregate Packet header data as signals

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Real-time Traffic monitoring and containment ' - gisela-fields


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Real time traffic monitoring and containment

Real-time Traffic monitoring and containment

A. L. Narasimha Reddy

Dept. of Electrical Engineering

Texas A & M University

[email protected]

http://ee.tamu.edu/~reddy/


Outline
Outline

  • Motivation

  • DOS attacks

    • Partial state routers

  • DDOS attacks, worms

    • Aggregate Packet header data as signals

    • Signal/image based anomaly/attack detectors

Texas A & M University


Real time traffic monitoring
Real-time traffic monitoring

  • Attacks motivate us to monitor network traffic

    • Potential anomaly/attack detectors

    • Potentially contain/throttle them as they happen

  • Line speeds are increasing

    • Need simple, effective mechanisms

  • Attacks constantly changing

    • CodeRed yesterday, MyDoom today, what next

Texas A & M University


Motivation
Motivation

  • Most current monitoring/policing tools are tailored to known attacks

    • Look for packets with port number 1434 (CodeRed)

    • Contain Kaaza traffic to 20% of the link

  • Become ineffective when traffic patterns or attacks change

    • New threats are constantly emerging

Texas A & M University


Motivation1
Motivation

  • Can we design generic (and generalized) mechanisms for attack detection and containment?

  • Can we make them simple enough to implement them at line speeds?

Texas A & M University


Introduction
Introduction

  • Why look for Kaaza packets

    • They consume resources

    • Consumes resources more than we want

  • Not much different from DOS flood

    • Consumes resources to stage attacks

  • Why not monitor resource usage?

    • Do not want to rely on attack specific info

Texas A & M University


Attacks
Attacks

  • DOS attacks

    • Few sources = resource hogs

  • DDOS attacks, worms

    • Many sources

    • Individual flows look normal

    • Look at the aggregate picture

Texas A & M University


Dos attacks network flows
DOS attacks & Network Flows

  • Too many flows to monitor each flow

  • Maintain a fixed amount of state/memory

    • State not enough to monitor all flows (Partial state)

    • Manage the state to monitor high-bandwidth flows

    • How?

  • Sample packets

    • High-BW flows more likely to be selected

  • Use a cache and employ LRU type policy

    • Traffic driven

    • Cache retains frequently arriving flows

Texas A & M University


Partial state approach
Partial State Approach

  • Similar to how caches are employed in computer memory systems

    • Exploit locality

  • Employ an engineering solution in an architecture-transparent fashion

Texas A & M University


Identifying resource hogs
Identifying resource hogs

  • Lots of web flows

    • Tend to corrupt the cache quickly

  • Apply probabilistic admission into cache

    • Flow has to arrive often to be included in cache

    • Most web flows not admitted

  • Works well in identifying high-BW flows

  • Can apply resource management techniques to contain cached/identified flows

Texas A & M University


Lru with probabilistic admission
LRU with probabilistic admission

  • Employ a modified LRU

  • On a miss, flow admitted with probability p

    • When p is small, keeps smaller flows out

    • High-BW flows more likely admitted

    • Allows high-BW flows to be retained in cache

  • Nonresponsive flows more likely to stay in cache

Texas A & M University


Traffic driven state management
Traffic Driven State Management

  • Monitor top 100 flows at any time

    • Don’t know the identity of these flows

    • Don’t know how much BW these may consume

Texas A & M University


Policy driven state management
Policy Driven State Management

  • An ISP could decide to monitor flows above 1Mbps

    • Will need state >= link capacity/1 Mbps

  • Could monitor flows consuming more than 1% of link capacity

    • For security reasons

    • At most 100 flows with 1% BW consumption

Texas A & M University




Udp cache occupancy
UDP Cache Occupancy

Texas A & M University


Tcp cache occupancy
TCP Cache Occupancy

Texas A & M University


Resource management
Resource Management

Texas A & M University


Preferential dropping
Preferential Dropping

1

drop prob

maxp

minth

maxth

Queue length

drop prob for high bandwidth flows

drop prob for other flows

Texas A & M University


Multiple possibilities
Multiple possibilities

  • SACRED: Monitor flows above certain rate (policy driven), differential RED, (iwqos99)

  • LRU-RED: Traffic driven state management, differential RED (Globecom01)

    • Approximately fair BW distribution

  • LRU-FQ: Traffic driven state management, fair queuing (ICC 04)

    • Contain DOS attacks

    • Provide shorter delays for short-term flows

Texas A & M University


Lru fq resource management
LRU-FQ Resource Management

Texas A & M University


Lru fq flow chart enqueue event
LRU-FQ flow chart – enqueue event

Does

Cache Have

space?

Is Flow in

Cache?

No

No

Admit flow with

Probability ‘p’

Packet

Arrival

Yes

Yes

Is Flow

Admitted?

Record flow details

Initialize ‘count’ to 0

Yes

Increment ‘count’

Move flow to top of cache

No

Is

‘count’ >= ‘threshold’

No

Yes

Enqueue in Normal

Queue

Enqueue in Partial state

Queue

Texas A & M University


Linux ip packet forwarding
Linux IP Packet Forwarding

Local packet

Deliver to upper layers

UPPER LAYERS

Route to destination

Update Packet

Error checking

Verify

Destination

IP LAYER

Packet Enqueued

Scheduler invokes

Bottom half

Design space

Scheduler runs

Device driver

LINK LAYER

Request

Scheduler

To invoke

bottom half

Device

Prepares

packet

Packet Departure

Packet Arrival

Check & Store

Packet

Enqueue pkt

Texas A & M University


Linux kernel traffic control
Linux Kernel traffic control

  • Filters are used to distinguish between different classes of flows.

  • Each class of flows can be further categorized into sub-classes using filters.

  • Queuing disciplines control how the packets are enqueued and dequeued

Texas A & M University


Lru fq implementation
LRU-FQ Implementation

  • LRU component of the scheme is implemented as a filter.

    • All parameters: threshold, probability and cache size are passed as parameters to the filter

  • Fair Queuing employed as a queuing discipline.

    • Scheduling based on queue’s weight.

    • Start-time Fair Queuing

Texas A & M University



Experimental setup
Experimental Setup

Texas A & M University


Long term flow differentiation
Long-Term flow differentiation

Normal TCP fraction = 0.07

Probability = 1/25 Cache size= 11 threshold= 125

Texas A & M University


Long term flow differentiation1
Long-term flow differentiation

Probability = 1/25 Cache size= 11 threshold= 125

Texas A & M University


Protecting web mice
Protecting Web Mice

Texas A & M University


Protecting web mice1

Long Term TCP Flows

20

LongTerm UDP Flows

2 – 4

Web Clients

20

Probability

1/50

Threshold

125

LRU Cache Size

11

LRU : Normal Queue

1:1

Protecting Web mice

Experimental Setup

Texas A & M University


Protecting web mice2

UDP Flows

UDP Flows

UDP Tput

UDP Tput

# Web Requests

# Web Requests

TCP Tput

TCP Tput

TCP Fraction

TCP Fraction

2

2

45.73

89.45

1313

13915

44.92

5.88

0.062

0.49

3

3

45.73

89.80

13828

1284

5.55

44.83

0.058

0.49

4

4

46.24

89.13

927

13632

6.21

44.51

0.49

0.065

Protecting Web Mice

Bandwidth Results

Normal Router

LRU-FQ Router

Texas A & M University


Protecting web mice3
Protecting Web Mice

Timing Results

Normal Router

LRU-FQ Router

Texas A & M University


Summary of lru fq
Summary of LRU-FQ

  • Provides a good control of DOS attacks with limited number of flows

  • Provides better delays for short-term flows

  • Automatically identifies resource hogs

  • Partial state packet handling cost -not an issue at 100Mbps.

Texas A & M University


Applications of partial state
Applications of Partial State

  • More intelligent control of network traffic

  • Accounting and measurement of high bandwidth flows

  • Denial of Service (DOS) attack prevention

  • Tracing of high bandwidth flows

  • QOS routing

Texas A & M University


Aggregated packet analysis
Aggregated packet analysis

Texas A & M University


Approach
Approach

Anomaly Detection

(Thresholding)

Signal Generation

& Data Filtering

(Address correlation)

Statistical or Signal Analysis

(Wavelets or DCT)

Detection Signal

Network Traffic

Texas A & M University


Signal generation
Signal Generation

  • Traffic volume (bytes or packets)

    • Analyzed before

    • May not be a great signal when links are always congested (typical campus access links)

  • Lot more information in packet headers

    • Source address

    • Destination address

    • Protocol number

    • Port numbers

Texas A & M University


Signal generation1
Signal Generation

  • Per packet cost is important driver

  • Update a counter for each packet header field

    • Too much memory to put in SRAM

  • Break the field into multiple 8-bit fields

    • 32-bit address into four 8-bit fields

    • 1024 locations instead of 2^32 locations

    • In general, 256* (k/8) instead of 2^k

    • k/8 counter updates instead of 1

Texas A & M University


Signal generation2
Signal Generation

  • What kind of signals can we generate with addresses, port numbers and protocol numbers?

Texas A & M University


Addresses are correlated
Addresses are correlated

  • Most of us have habits

    • Access same web sites

  • Large web sites get significant part of traffic

    • Google.com, hp.com, yahoo.com

  • Large downloads correlate over time

    • ftp, video

  • On an aggregate, addresses are correlated

Texas A & M University


Address correlation attacks
Address Correlation –attacks?

  • Address correlation changes when traffic patterns change abruptly

    • Denial of service attacks

    • Flash crowds

    • Worms

  • Results in differences in correlation

    • High --single attack victim

    • Low – lots of addresses --worm

Texas A & M University


Address correlation signals
Address correlation signals

  • Address correlation:

  • Simplified Address correlation:

Texas A & M University


Address correlation signals1
Address Correlation Signals

Texas A & M University


Address correlation signals2
Address Correlation Signals

Texas A & M University


Signal analysis
Signal Analysis

  • Capture information over a sampling period

    • Of the order of a few seconds to minutes

  • Analyze each sample to detect anomalies

    • Compare with historical norms

  • Post-mortem/Real-time analysis

    • May use different amounts of data & analysis

      • Detailed information of past few samples

      • Less detailed information of older samples

Texas A & M University


Signal analysis1
Signal Analysis

  • Address correlation as a time series signal

  • Employ known techniques to analyze time series signals

  • Wavelets –one powerful technique

    • Allows analysis in both time and frequency domain

  • Per-sample analysis has more flexibility

    • Not in forwarding path

Texas A & M University


Does this work
Does this work?

Texas A & M University


Analysis of address signal
Analysis of address signal

Texas A & M University


Image based analysis
Image based analysis

  • Treat the traffic data as images

  • Apply image processing based analysis

  • Treat each sample as a frame in a video

    • Video compression techniques lead to data reduction

    • Scene change analysis leads to anomaly detection

    • Motion prediction leads to attack prediction

Texas A & M University


Signal generation3
Signal Generation

Texas A & M University


Two dimensional images
Two dimensional images

  • Horizontal/vertical lines indicate anomalies

    • Infected machine contacting multiple destinations (worm propagation)

    • Multiple source machines targeting a destination (DDOS)

Texas A & M University


Dct analysis of addresses
DCT analysis of addresses

Texas A & M University


Semi random attacks
Semi-random attacks

Texas A & M University


Random attacks
Random attacks

Texas A & M University


Better than volume analysis
Better than volume analysis

Texas A & M University


Motion prediction
Motion prediction

Texas A & M University


Advantages
Advantages

  • Not looking for specific known attacks

  • Generic mechanism

  • Works in real-time

    • Latencies of a few samples

    • Simple enough to be implemented inline

Texas A & M University


Prototypes
Prototypes

  • Linux-PC boxes

  • On Intel Network processors

    • Can push to Gbps packet forwarding rates

    • Forwarding throughput not impacted

    • Sampling rates of a few ms possible

Texas A & M University


Conclusion
Conclusion

  • Real-time resource accounting is feasible

  • Real-time traffic monitoring is feasible

    • Simple enough to be implemented inline

  • Can rely on many tools from signal/image processing area

    • More robust offline analysis possible

    • Concise for logging and playback

Texas A & M University


Thank you for more information http ee tamu edu reddy reddy@ee tamu edu
Thank you !!For more information,http://ee.tamu.edu/[email protected]

Texas A & M University


Other work
Other work

  • Enhancements to TCP

    • TCP-DCR for wireless losses, packet reordering

    • Layered TCP for high-speed(Gbps) links

  • Alternate routing for improving service availability during link transients

    • Continues routing packets until routing tables are recomputed

    • Important for VOIP applications

Texas A & M University


Tcp enhancements
TCP Enhancements

  • TCP-DCR:

    • Modifies TCP’s congestion response to tolerate non-congestion events (channel errors, packet reordering)

  • LTCP (Layered TCP):

    • Improves TCP’s performance in high-speed networks

Texas A & M University


Tcp dcr channel errors
TCP-DCR –channel errors

Texas A & M University


Tcp dcr packet reordering
TCP-DCR –packet reordering

Texas A & M University


LTCP

Texas A & M University


Thank you for more information http ee tamu edu reddy reddy@ee tamu edu1
Thank you !!For more information,http://ee.tamu.edu/[email protected]

Texas A & M University


Lru red results
LRU-RED Results

Texas A & M University


Rtt bias tcp flows
RTT Bias -TCP flows

Texas A & M University


Impact of cache size
Impact of Cache size

  • Effect of varying cache size

    • to study impact of cache size on performance of the scheme

    • probability= 1/55, threshold = 125

    • number of TCP flows=20

    • equal weights for both queues.

Texas A & M University


Results cache size
Results – Cache size

Texas A & M University


Normal workloads
Normal Workloads

  • Performance under normal workloads

    • working of scheme when non-responsive loads are absent or use their fair share of bandwidth

    • cache size = 9, threshold =125

    • probability = 1/55

Texas A & M University


Results normal workload
Results – Normal workload

Texas A & M University


Normal mixed workload
Normal Mixed workload

Texas A & M University


ad