1. Coordinated Sampling sans Origin-Destination Identifiers: Algorithms and Analysis Vyas Sekar, Anupam Gupta, Michael K. Reiter, HuiZhang Carnegie Mellon University Univ. of North Carolina Chapel-Hill

2. Flow Monitoring is critical for effective Network Management Traffic Engineering Accounting Need high-fidelity measurements Worm Detection Network Forensics High flow coverage Provide network-wide goals Analyze new user apps ……. Respect resource constraints Botnet analysis Anomaly Detection

3. How do we meet the requirements? High flow coverage Flow Sampling Provide network-wide goals cSamp [NSDI’08] Network-Wide Coordination & Optimization Respect resource constraints

4. Network-wide coordination Sampling Manifest [1,5] [3,7] [7,9] [1,3] [1,2] [5,8] Assign non-overlapping ranges per OD-pair or path All routers configured with same hash function/key

5. Generating Sampling Manifests Objective: Max iεODPairsCoverageiTrafficiSubject to achieving maximum Mini εODPairs{ Coveragei} Inputs Linear Program OD-pair info Traffic, Path(routers) Output Sampling manifests Network-wide Optimization (@ NOC) {<OD-Pair,Hash-range>} per router Router constraints e.g., SRAM for flow records

6. cSamp algorithm on each router Sampling Manifest Flow memory OD Range [5,10] 2 1 [1,4] Red vs. Green? 2 1. Get OD-Pair from packet 2. Compute hash (flow = packet 5-tuple) 3. Look up hash-range for OD-pair from sampling manifest 4.Log if hash falls in range for this OD-pair

7. 1. Get OD-Pair from packet Why is this challenging? OD-pair identification might be ambiguous Multi-exit peers (and prefix aggregation) (Even with MPLS) How does cSamp overcome this? Ingresses compute and add this to packet headers Need to modify packet headers/add shim header Extra computation on ingresses May require overhauling routing infrastructure

8. Can we realize the benefits of cSamp without OD-pair identification? Use local information to make sampling decisions “Stitch” coverage across routers on a path

9. Outline • Background and Motivation • Problem Formulation • Algorithms and Heuristics • Evaluation

10. What local info can I get from packet and routing table? SamplingSpec Granularity at which sampling decisions are made R R1 R2 R3 How much to sample for this SamplingSpec? {Previous Hop, My Id, NextHop} SamplingAtom Discrete hash-ranges, select some to log

11. “Stitching” together coverage R1 R6 union = R3 R4 R5 R2 R7 union =

12. Problem Formulation SamplingAtom Coverage for path Pi Load on router Rj SamplingSpec Maximize: Total flow coverage: iTiCi Minimum fractional coverage: mini {Ci} Subject To:j, LoadjLj

13. Outline • Background and Motivation • Problem Formulation • Algorithms and Heuristics • Evaluation

14. NP-hard! Maximize: Total flow coverage: i TiCi Min. frac coverage: mini {Ci } Subject To: j, Loadj Lj Min: Hard to approximate! Total flow coverage: Submodular maximization with partition-knapsack  Efficient greedy algorithm is near-optimal Min. fractional flow coverage:  Need “resource augmentation”Intelligent resource augmentation Incrementally add OD-pair identifiers

15. Leveraging submodularity for ftot A function F: 2V is submodular if  A  A'  V, and s V, F(A  {s}) - F(A)  F (A'  {s}) - F(A’) “diminishing returns” Why does it matter? Max F s.tc(A) B, where F is monotoneGreedy algorithm gives a constant-factor approximation Can do lazy evaluation to speedup Maximize: ftot= iTiCi, Subject To: j, LoadjLj Special case of above problem with “partition-knapsack”

16. What about fmin? fmin = mini {Ci} is not submodular Hard to approximate without violating constraints! But, can get near-optimal, if we violate by a fixed factor Main idea: Define f’ = iC’iwhere C’i = min {Ci, T} Note that f’ = N * T, iff each Ci T Run binary search over T to find best solution (Each iteration runs greedy with no resource constraints) • Heuristic improvements: • Intelligent resource augmentation • Upgrade a few ingresses to add OD-pairs

17. Outline • Motivation • Problem Formulation • Algorithms and Heuristics • Evaluation

18. Total flow coverage cSamp-T (tuple+) gives near-ideal total flow coverage vs. cSamp

19. Minimum fractional coverage(with intelligent resource augmentation) Can get 75% of optimal performance with 1.5X total increase and a 5X max-per-router increase

20. Summary • cSamp for efficient flow monitoring • Network-wide coordination and optimization • But needs OD-pair identification • How to implement cSamp without OD-pair ids? • Leverage submodularity for total coverage • Targeted upgrades for minimum fractional coverage • cSamp-T makes cSamp’s benefits more immediately available