Network Measurement

Network Measurement PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

Motivation. service providers, service usersmonitoringanomaly detectiondebuggingtraffic engineeringpricing, peering, service level agreementsarchitecture designapplication design. Active measurements. active probe tools send stimulus (packets) into network; measure responsenetwork, transpor

Download Presentation

Network Measurement

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

1. Network Measurement/Management motivation measurement strategies passive sampling active network tomography

2. Motivation service providers, service users monitoring anomaly detection debugging traffic engineering pricing, peering, service level agreements architecture design application design

3. Active measurements active probe tools send stimulus (packets) into network; measure response network, transport, application layer probes can measure many things delay/loss topology/routing behavior bandwidth/throughput earliest tools use Internet Control Message Protocol (ICMP)

4. ping uses ICMP Echo capability C:\WINDOWS\Desktop>ping Reply from bytes=32 time=253ms TTL=240 Reply from bytes=32 time=231ms TTL=240 Reply from bytes=32 time=225ms TTL=240 Reply from bytes=32 time=214ms TTL=240 Ping statistics for packets: Sent = 4, Received = 4, Lost = 0 (0% loss), approximate round trip times in milliseconds: Minimum = 214ms, Maximum = 253ms, Average = 230ms

5. traceroute diagnostic tool in widespread use by users and providers finds outward path to given host, round trip times along path

6. Example: traceroute for n=1,2,…,nmax send pkt with TTL = n pkt dies at nth router router returns ICMP pkt with router address

7. traceroute example

8. Passive measurements Capture packet data as it passes by packet capture applications on hosts use packet capture filters (tcpdump) requires access to the wire promiscuous mode network ports to see other traffic flow-level, packet-level data on routers SNMP MIBs Cisco NetFlow hardware-based solutions Endace, Inc.’s DAG cards – OC12/48/192

9. Example from tcpdump 04:47:00.410393 > S 1616942532:1616942532(0) win 512 (ttl 64, id 47959) 04:47:03.409692 > S 1616942532:1616942532(0) win 32120 (ttl 64, id 47963) 04:47:03.489652 > S 3389387880:3389387880(0) ack 1616942533 win 31744 (ttl 52, id 27319) 04:47:03.489652 > . ack 1 win 32120 (DF) (ttl 64, id 47964) 04:47:03.489652 > P 1:67(66) ack 1 win 32120 (DF) (ttl 64, id 47965) 04:47:03.579607 > . ack 67 win 31744 (DF) (ttl 52, id 27469) 04:47:04.249539 > . 1:1461(1460) ack 67 win 31744 (DF) (ttl 52, id 28879) 04:47:04.249539 > . 1461:2921(1460) ack 67 win 31744 (DF) (ttl 52, id 28880) 04:47:04.259534 > . ack 2921 win 32120 (DF) (ttl 64, id 47968) 04:47:04.349489 > P 2921:4097(1176) ack 67 win 31744 (DF) (ttl 52, id 29032) 04:47:04.349489 > . 4097:5557(1460) ack 67 win 31744 (ttl 52, id 29033)

10. Passive IP flow measurement IP Flow defined as “unidirectional series of packets between source/dest IP/port pair over period of time” Identified by (IP protcol, src address, src port, dst address, dst port) exported by applications such as Cisco’s NetFlow

11. Netflow: example addin

12. Challenges flow observations are memory/processor intensive how to do flow observations at high speeds use sampling

13. Need for packet sampling keep cache of active flows for keys seen, but corresponding flow not yet terminated packet classification each arriving packet: cache lookup to match key if match: modify cache entry, e.g., increment counters, adjust timers else: instantiate new cache entry cache resources for high end routers memory: 1,000s of active flows speed: look up at line rate ? lots of fast memory

14. Packet sampling construct flows from sampled packet stream (e.g. 1 in N periodic) call these “packet sampled flows” reduce effective packet rate reduces cost: slower memory sufficient

15. Packet sampling Simple example: recover original packet rate sample packets with probability q measure rate of sampled traffic l(q) infer rate of original traffic l(q)/q

16. IP flow: set of packets with same 5-tuple

17. Original traffic

18. Packet sampling recovering original flow sizes not easy

19. Packet sampling in latest Cisco router

20. Original traffic

21. Flow sampling

22. Flow statistics from packet sampling measured flows set of packets with common property, observed in some time period common property: “key”: built from header fields (e.g. src/dst address, TCP/UDP ports) flow termination criteria interpacket timeout protocol signals (e.g. TCP FIN) ageing, flushing, …

23. Flow statistics from packet sampling (2) flow summaries reports of measured flows exported from routers flow key, flow packets/bytes, first/last packet time, router state inversion and inference recover properties of original flows from packet sampled flow statistics

24. Rate and #active flows: aggregate traffic rate and #active flows decreasing, eventually proportional to 1/N probability to at least one of p packets ? p/N for large N

25. Rate, #active flows: application application identified by port number rate of flow production can increase with N for some applications, eventually decreasing napster, ms-streaming, realaudio mean active flows decreases with N

26. Packet sampling 1 out of N What constitutes a flow? large T: less splitting: fewer flows observed, more active flows

27. Flow splitting under sampling sampling increases interpacket times flow splitting when interpacket time exceed interpacket timeout flows vulnerable to splitting: call these sparse flows with many packets, not too fast packet rate e.g. streaming, p2p applications Question: if increase T, as N increases: can we better maintain flow semantics?

28. Rates, #active flows: trade-offs www: mean flow length 6 pkts little flow splitting active flows linear in T, observed flows constant napster: mean flow length 455 pkts small N: big trade off between rate and #active flows large N: trade-off washes out (typically only 1 packet sampled)

29. Packet sampling no. “active” flows ~1/N reduction Big savings in memory size and speed observe ~ 1/N flows (large N)

30. Inferring original flow statistics from packet sampled flow statistics

31. Characteristics of interest motivation assume only packet sampled flow statistics available want to determine characteristics of original flows which? packet/byte rates arrival rate of original flows average # packets/ bytes per original flow why difficult? some flows are missed altogether trick: supplement with protocol level information, when available

32. Easy estimates original packet and bytes model: packets independently sampled with probability 1/N estimates: # original packets by Pest = N * # sampled packets # original bytes by Best = N * # sampled bytes properties (Bernoulli sampling): unbiased estimators: E[Pest] = P; E[Best] = B standard error bounds

33. Estimating number of TCP flows M # of original TCP flows with Cisco NetFlow, can detect (w high prob.) if sampled packet was SYN model (SYN flags in TCP flows are well-behaved) each TCP flow contains one SYN packet expect close adherence to model, modulo retransmits, packet drops experiments long flow traces: very rare not to see at least one SYN similar model for FIN packets not so accurate poor termination, SYN flood attacks estimation each SYN packet sampled with probability 1/N estimate: M1 = N * #{sampled flows with SYN flag set} properties: unbiased estimator of M = #{original TCP flows}

34. Estimating number of original TCP flows (2) estimator M1: uses only sampled SYN flows decrease estimator variance by using all flow statistics? yes - see reading estimate of mean packets per flow, bytes per flow packets: pest, 1 = Pest / M1 ; bytes: best,1 = Best / M1

35. Estimation Accuracy Restricted packet trace: select only packets in original TCP flows starting a SYN packet

36. Flow sampling flow metrics much easier 1 out of N flows sampled estimate for # of flows M = N*#flows sampled E[flow size, pkts], pest = ?i #pkts in flow i/# sampled flows E[flow size, bytes], best = ?i #bytes in flow i /# sampled flows flow size distribution easy to estimate

  • Login