Real time traffic monitoring and containment
This presentation is the property of its rightful owner.
Sponsored Links
1 / 74

Real-time Traffic monitoring and containment PowerPoint PPT Presentation


  • 68 Views
  • Uploaded on
  • Presentation posted in: General

Real-time Traffic monitoring and containment. A. L. Narasimha Reddy Dept. of Electrical Engineering Texas A & M University [email protected] http://ee.tamu.edu/~reddy/. Outline. Motivation DOS attacks Partial state routers DDOS attacks, worms Aggregate Packet header data as signals

Download Presentation

Real-time Traffic monitoring and containment

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Real time traffic monitoring and containment

Real-time Traffic monitoring and containment

A. L. Narasimha Reddy

Dept. of Electrical Engineering

Texas A & M University

[email protected]

http://ee.tamu.edu/~reddy/


Outline

Outline

  • Motivation

  • DOS attacks

    • Partial state routers

  • DDOS attacks, worms

    • Aggregate Packet header data as signals

    • Signal/image based anomaly/attack detectors

Texas A & M University


Real time traffic monitoring

Real-time traffic monitoring

  • Attacks motivate us to monitor network traffic

    • Potential anomaly/attack detectors

    • Potentially contain/throttle them as they happen

  • Line speeds are increasing

    • Need simple, effective mechanisms

  • Attacks constantly changing

    • CodeRed yesterday, MyDoom today, what next

Texas A & M University


Motivation

Motivation

  • Most current monitoring/policing tools are tailored to known attacks

    • Look for packets with port number 1434 (CodeRed)

    • Contain Kaaza traffic to 20% of the link

  • Become ineffective when traffic patterns or attacks change

    • New threats are constantly emerging

Texas A & M University


Motivation1

Motivation

  • Can we design generic (and generalized) mechanisms for attack detection and containment?

  • Can we make them simple enough to implement them at line speeds?

Texas A & M University


Introduction

Introduction

  • Why look for Kaaza packets

    • They consume resources

    • Consumes resources more than we want

  • Not much different from DOS flood

    • Consumes resources to stage attacks

  • Why not monitor resource usage?

    • Do not want to rely on attack specific info

Texas A & M University


Attacks

Attacks

  • DOS attacks

    • Few sources = resource hogs

  • DDOS attacks, worms

    • Many sources

    • Individual flows look normal

    • Look at the aggregate picture

Texas A & M University


Dos attacks network flows

DOS attacks & Network Flows

  • Too many flows to monitor each flow

  • Maintain a fixed amount of state/memory

    • State not enough to monitor all flows (Partial state)

    • Manage the state to monitor high-bandwidth flows

    • How?

  • Sample packets

    • High-BW flows more likely to be selected

  • Use a cache and employ LRU type policy

    • Traffic driven

    • Cache retains frequently arriving flows

Texas A & M University


Partial state approach

Partial State Approach

  • Similar to how caches are employed in computer memory systems

    • Exploit locality

  • Employ an engineering solution in an architecture-transparent fashion

Texas A & M University


Identifying resource hogs

Identifying resource hogs

  • Lots of web flows

    • Tend to corrupt the cache quickly

  • Apply probabilistic admission into cache

    • Flow has to arrive often to be included in cache

    • Most web flows not admitted

  • Works well in identifying high-BW flows

  • Can apply resource management techniques to contain cached/identified flows

Texas A & M University


Lru with probabilistic admission

LRU with probabilistic admission

  • Employ a modified LRU

  • On a miss, flow admitted with probability p

    • When p is small, keeps smaller flows out

    • High-BW flows more likely admitted

    • Allows high-BW flows to be retained in cache

  • Nonresponsive flows more likely to stay in cache

Texas A & M University


Traffic driven state management

Traffic Driven State Management

  • Monitor top 100 flows at any time

    • Don’t know the identity of these flows

    • Don’t know how much BW these may consume

Texas A & M University


Policy driven state management

Policy Driven State Management

  • An ISP could decide to monitor flows above 1Mbps

    • Will need state >= link capacity/1 Mbps

  • Could monitor flows consuming more than 1% of link capacity

    • For security reasons

    • At most 100 flows with 1% BW consumption

Texas A & M University


Partial state trace driven evaluation

Partial State –Trace-driven evaluation

Texas A & M University


Partial state trace driven evaluation1

Partial State –Trace-driven Evaluation

Texas A & M University


Udp cache occupancy

UDP Cache Occupancy

Texas A & M University


Tcp cache occupancy

TCP Cache Occupancy

Texas A & M University


Resource management

Resource Management

Texas A & M University


Preferential dropping

Preferential Dropping

1

drop prob

maxp

minth

maxth

Queue length

drop prob for high bandwidth flows

drop prob for other flows

Texas A & M University


Multiple possibilities

Multiple possibilities

  • SACRED: Monitor flows above certain rate (policy driven), differential RED, (iwqos99)

  • LRU-RED: Traffic driven state management, differential RED (Globecom01)

    • Approximately fair BW distribution

  • LRU-FQ: Traffic driven state management, fair queuing (ICC 04)

    • Contain DOS attacks

    • Provide shorter delays for short-term flows

Texas A & M University


Lru fq resource management

LRU-FQ Resource Management

Texas A & M University


Lru fq flow chart enqueue event

LRU-FQ flow chart – enqueue event

Does

Cache Have

space?

Is Flow in

Cache?

No

No

Admit flow with

Probability ‘p’

Packet

Arrival

Yes

Yes

Is Flow

Admitted?

Record flow details

Initialize ‘count’ to 0

Yes

Increment ‘count’

Move flow to top of cache

No

Is

‘count’ >= ‘threshold’

No

Yes

Enqueue in Normal

Queue

Enqueue in Partial state

Queue

Texas A & M University


Linux ip packet forwarding

Linux IP Packet Forwarding

Local packet

Deliver to upper layers

UPPER LAYERS

Route to destination

Update Packet

Error checking

Verify

Destination

IP LAYER

Packet Enqueued

Scheduler invokes

Bottom half

Design space

Scheduler runs

Device driver

LINK LAYER

Request

Scheduler

To invoke

bottom half

Device

Prepares

packet

Packet Departure

Packet Arrival

Check & Store

Packet

Enqueue pkt

Texas A & M University


Linux kernel traffic control

Linux Kernel traffic control

  • Filters are used to distinguish between different classes of flows.

  • Each class of flows can be further categorized into sub-classes using filters.

  • Queuing disciplines control how the packets are enqueued and dequeued

Texas A & M University


Lru fq implementation

LRU-FQ Implementation

  • LRU component of the scheme is implemented as a filter.

    • All parameters: threshold, probability and cache size are passed as parameters to the filter

  • Fair Queuing employed as a queuing discipline.

    • Scheduling based on queue’s weight.

    • Start-time Fair Queuing

Texas A & M University


Lru fq results

LRU-FQ - Results


Experimental setup

Experimental Setup

Texas A & M University


Long term flow differentiation

Long-Term flow differentiation

Normal TCP fraction = 0.07

Probability = 1/25Cache size= 11 threshold= 125

Texas A & M University


Long term flow differentiation1

Long-term flow differentiation

Probability = 1/25Cache size= 11 threshold= 125

Texas A & M University


Protecting web mice

Protecting Web Mice

Texas A & M University


Protecting web mice1

Long Term TCP Flows

20

LongTerm UDP Flows

2 – 4

Web Clients

20

Probability

1/50

Threshold

125

LRU Cache Size

11

LRU : Normal Queue

1:1

Protecting Web mice

Experimental Setup

Texas A & M University


Protecting web mice2

UDP Flows

UDP Flows

UDP Tput

UDP Tput

# Web Requests

# Web Requests

TCP Tput

TCP Tput

TCP Fraction

TCP Fraction

2

2

45.73

89.45

1313

13915

44.92

5.88

0.062

0.49

3

3

45.73

89.80

13828

1284

5.55

44.83

0.058

0.49

4

4

46.24

89.13

927

13632

6.21

44.51

0.49

0.065

Protecting Web Mice

Bandwidth Results

Normal Router

LRU-FQ Router

Texas A & M University


Protecting web mice3

Protecting Web Mice

Timing Results

Normal Router

LRU-FQ Router

Texas A & M University


Summary of lru fq

Summary of LRU-FQ

  • Provides a good control of DOS attacks with limited number of flows

  • Provides better delays for short-term flows

  • Automatically identifies resource hogs

  • Partial state packet handling cost -not an issue at 100Mbps.

Texas A & M University


Applications of partial state

Applications of Partial State

  • More intelligent control of network traffic

  • Accounting and measurement of high bandwidth flows

  • Denial of Service (DOS) attack prevention

  • Tracing of high bandwidth flows

  • QOS routing

Texas A & M University


Aggregated packet analysis

Aggregated packet analysis

Texas A & M University


Approach

Approach

Anomaly Detection

(Thresholding)

Signal Generation

& Data Filtering

(Address correlation)

Statistical or Signal Analysis

(Wavelets or DCT)

Detection Signal

Network Traffic

Texas A & M University


Signal generation

Signal Generation

  • Traffic volume (bytes or packets)

    • Analyzed before

    • May not be a great signal when links are always congested (typical campus access links)

  • Lot more information in packet headers

    • Source address

    • Destination address

    • Protocol number

    • Port numbers

Texas A & M University


Signal generation1

Signal Generation

  • Per packet cost is important driver

  • Update a counter for each packet header field

    • Too much memory to put in SRAM

  • Break the field into multiple 8-bit fields

    • 32-bit address into four 8-bit fields

    • 1024 locations instead of 2^32 locations

    • In general, 256* (k/8) instead of 2^k

    • k/8 counter updates instead of 1

Texas A & M University


Signal generation2

Signal Generation

  • What kind of signals can we generate with addresses, port numbers and protocol numbers?

Texas A & M University


Addresses are correlated

Addresses are correlated

  • Most of us have habits

    • Access same web sites

  • Large web sites get significant part of traffic

    • Google.com, hp.com, yahoo.com

  • Large downloads correlate over time

    • ftp, video

  • On an aggregate, addresses are correlated

Texas A & M University


Address correlation attacks

Address Correlation –attacks?

  • Address correlation changes when traffic patterns change abruptly

    • Denial of service attacks

    • Flash crowds

    • Worms

  • Results in differences in correlation

    • High --single attack victim

    • Low – lots of addresses --worm

Texas A & M University


Address correlation signals

Address correlation signals

  • Address correlation:

  • Simplified Address correlation:

Texas A & M University


Address correlation signals1

Address Correlation Signals

Texas A & M University


Address correlation signals2

Address Correlation Signals

Texas A & M University


Signal analysis

Signal Analysis

  • Capture information over a sampling period

    • Of the order of a few seconds to minutes

  • Analyze each sample to detect anomalies

    • Compare with historical norms

  • Post-mortem/Real-time analysis

    • May use different amounts of data & analysis

      • Detailed information of past few samples

      • Less detailed information of older samples

Texas A & M University


Signal analysis1

Signal Analysis

  • Address correlation as a time series signal

  • Employ known techniques to analyze time series signals

  • Wavelets –one powerful technique

    • Allows analysis in both time and frequency domain

  • Per-sample analysis has more flexibility

    • Not in forwarding path

Texas A & M University


Does this work

Does this work?

Texas A & M University


Analysis of address signal

Analysis of address signal

Texas A & M University


Image based analysis

Image based analysis

  • Treat the traffic data as images

  • Apply image processing based analysis

  • Treat each sample as a frame in a video

    • Video compression techniques lead to data reduction

    • Scene change analysis leads to anomaly detection

    • Motion prediction leads to attack prediction

Texas A & M University


Signal generation3

Signal Generation

Texas A & M University


Two dimensional images

Two dimensional images

  • Horizontal/vertical lines indicate anomalies

    • Infected machine contacting multiple destinations (worm propagation)

    • Multiple source machines targeting a destination (DDOS)

Texas A & M University


Dct analysis of addresses

DCT analysis of addresses

Texas A & M University


Semi random attacks

Semi-random attacks

Texas A & M University


Random attacks

Random attacks

Texas A & M University


Better than volume analysis

Better than volume analysis

Texas A & M University


Motion prediction

Motion prediction

Texas A & M University


Advantages

Advantages

  • Not looking for specific known attacks

  • Generic mechanism

  • Works in real-time

    • Latencies of a few samples

    • Simple enough to be implemented inline

Texas A & M University


Prototypes

Prototypes

  • Linux-PC boxes

  • On Intel Network processors

    • Can push to Gbps packet forwarding rates

    • Forwarding throughput not impacted

    • Sampling rates of a few ms possible

Texas A & M University


Conclusion

Conclusion

  • Real-time resource accounting is feasible

  • Real-time traffic monitoring is feasible

    • Simple enough to be implemented inline

  • Can rely on many tools from signal/image processing area

    • More robust offline analysis possible

    • Concise for logging and playback

Texas A & M University


Thank you for more information http ee tamu edu reddy reddy@ee tamu edu

Thank you !!For more information,http:[email protected]

Texas A & M University


Other work

Other work

  • Enhancements to TCP

    • TCP-DCR for wireless losses, packet reordering

    • Layered TCP for high-speed(Gbps) links

  • Alternate routing for improving service availability during link transients

    • Continues routing packets until routing tables are recomputed

    • Important for VOIP applications

Texas A & M University


Tcp enhancements

TCP Enhancements

  • TCP-DCR:

    • Modifies TCP’s congestion response to tolerate non-congestion events (channel errors, packet reordering)

  • LTCP (Layered TCP):

    • Improves TCP’s performance in high-speed networks

Texas A & M University


Tcp dcr channel errors

TCP-DCR –channel errors

Texas A & M University


Tcp dcr packet reordering

TCP-DCR –packet reordering

Texas A & M University


Real time traffic monitoring and containment

LTCP

Texas A & M University


Thank you for more information http ee tamu edu reddy reddy@ee tamu edu1

Thank you !!For more information,http:[email protected]

Texas A & M University


Lru red results

LRU-RED Results

Texas A & M University


Rtt bias tcp flows

RTT Bias -TCP flows

Texas A & M University


Impact of cache size

Impact of Cache size

  • Effect of varying cache size

    • to study impact of cache size on performance of the scheme

    • probability= 1/55, threshold = 125

    • number of TCP flows=20

    • equal weights for both queues.

Texas A & M University


Results cache size

Results – Cache size

Texas A & M University


Normal workloads

Normal Workloads

  • Performance under normal workloads

    • working of scheme when non-responsive loads are absent or use their fair share of bandwidth

    • cache size = 9, threshold =125

    • probability = 1/55

Texas A & M University


Results normal workload

Results – Normal workload

Texas A & M University


Normal mixed workload

Normal Mixed workload

Texas A & M University


  • Login