200 likes | 301 Views
Learn about Coordinated Sampling for effective network management, traffic engineering, and anomaly detection. Discover algorithms, analysis techniques, and optimization strategies for maximizing flow coverage and respecting resource constraints. Explore the challenges and solutions for implementing cSamp without OD-pair identification.
E N D
Coordinated Sampling sans Origin-Destination Identifiers: Algorithms and Analysis Vyas Sekar, Anupam Gupta, Michael K. Reiter, HuiZhang Carnegie Mellon University Univ. of North Carolina Chapel-Hill
Flow Monitoring is critical for effective Network Management Traffic Engineering Accounting Need high-fidelity measurements Worm Detection Network Forensics High flow coverage Provide network-wide goals Analyze new user apps ……. Respect resource constraints Botnet analysis Anomaly Detection
How do we meet the requirements? High flow coverage Flow Sampling Provide network-wide goals cSamp [NSDI’08] Network-Wide Coordination & Optimization Respect resource constraints
Network-wide coordination Sampling Manifest [1,5] [3,7] [7,9] [1,3] [1,2] [5,8] Assign non-overlapping ranges per OD-pair or path All routers configured with same hash function/key
Generating Sampling Manifests Objective: Max iεODPairsCoverageiTrafficiSubject to achieving maximum Mini εODPairs{ Coveragei} Inputs Linear Program OD-pair info Traffic, Path(routers) Output Sampling manifests Network-wide Optimization (@ NOC) {<OD-Pair,Hash-range>} per router Router constraints e.g., SRAM for flow records
cSamp algorithm on each router Sampling Manifest Flow memory OD Range [5,10] 2 1 [1,4] Red vs. Green? 2 1. Get OD-Pair from packet 2. Compute hash (flow = packet 5-tuple) 3. Look up hash-range for OD-pair from sampling manifest 4.Log if hash falls in range for this OD-pair
1. Get OD-Pair from packet Why is this challenging? OD-pair identification might be ambiguous Multi-exit peers (and prefix aggregation) (Even with MPLS) How does cSamp overcome this? Ingresses compute and add this to packet headers Need to modify packet headers/add shim header Extra computation on ingresses May require overhauling routing infrastructure
Can we realize the benefits of cSamp without OD-pair identification? Use local information to make sampling decisions “Stitch” coverage across routers on a path
Outline • Background and Motivation • Problem Formulation • Algorithms and Heuristics • Evaluation
What local info can I get from packet and routing table? SamplingSpec Granularity at which sampling decisions are made R R1 R2 R3 How much to sample for this SamplingSpec? {Previous Hop, My Id, NextHop} SamplingAtom Discrete hash-ranges, select some to log
“Stitching” together coverage R1 R6 union = R3 R4 R5 R2 R7 union =
Problem Formulation SamplingAtom Coverage for path Pi Load on router Rj SamplingSpec Maximize: Total flow coverage: iTiCi Minimum fractional coverage: mini {Ci} Subject To:j, LoadjLj
Outline • Background and Motivation • Problem Formulation • Algorithms and Heuristics • Evaluation
NP-hard! Maximize: Total flow coverage: i TiCi Min. frac coverage: mini {Ci } Subject To: j, Loadj Lj Min: Hard to approximate! Total flow coverage: Submodular maximization with partition-knapsack Efficient greedy algorithm is near-optimal Min. fractional flow coverage: Need “resource augmentation”Intelligent resource augmentation Incrementally add OD-pair identifiers
Leveraging submodularity for ftot A function F: 2V is submodular if A A' V, and s V, F(A {s}) - F(A) F (A' {s}) - F(A’) “diminishing returns” Why does it matter? Max F s.tc(A) B, where F is monotoneGreedy algorithm gives a constant-factor approximation Can do lazy evaluation to speedup Maximize: ftot= iTiCi, Subject To: j, LoadjLj Special case of above problem with “partition-knapsack”
What about fmin? fmin = mini {Ci} is not submodular Hard to approximate without violating constraints! But, can get near-optimal, if we violate by a fixed factor Main idea: Define f’ = iC’iwhere C’i = min {Ci, T} Note that f’ = N * T, iff each Ci T Run binary search over T to find best solution (Each iteration runs greedy with no resource constraints) • Heuristic improvements: • Intelligent resource augmentation • Upgrade a few ingresses to add OD-pairs
Outline • Motivation • Problem Formulation • Algorithms and Heuristics • Evaluation
Total flow coverage cSamp-T (tuple+) gives near-ideal total flow coverage vs. cSamp
Minimum fractional coverage(with intelligent resource augmentation) Can get 75% of optimal performance with 1.5X total increase and a 5X max-per-router increase
Summary • cSamp for efficient flow monitoring • Network-wide coordination and optimization • But needs OD-pair identification • How to implement cSamp without OD-pair ids? • Leverage submodularity for total coverage • Targeted upgrades for minimum fractional coverage • cSamp-T makes cSamp’s benefits more immediately available