1 / 16

Monitoring very high speed links

Monitoring very high speed links. Gianluca Iannaccone Sprint ATL joint work with: Christophe Diot – Sprint ATL Ian Graham – University of Waikato Nick McKeown – Stanford University. Packet-level trace collection: the DAG example.

michelle-mason
Download Presentation

Monitoring very high speed links

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Monitoring very high speed links Gianluca Iannaccone Sprint ATL joint work with: Christophe Diot – Sprint ATL Ian Graham – University of Waikato Nick McKeown – Stanford University

  2. Packet-level trace collection: the DAG example • Optical splitter to divert part of the signal toward the capture card. • The capture card stores a record with a timestamp for each packet. • The card uses the PCI bus to transfer the records to the host main memory. • It is used by the Sprint IP Monitoring Project. IMW 2001

  3. Challenges for next generation capture devices • PCI bus throughput: • PCI 66Mhz/64bit is already challenged at OC-48 speed. • Storage capacity and data management: • Several terabytes per trace collection. • Memory access speed. • Disk array speed: • OC-192 would require roughly 250Mbytes/sec (64 byte records, 300 byte average packet size). IMW 2001

  4. Challenges for next generation capture devices (cont’d) • Deployment of extensive monitoring facilities inside the routers: • Backplane speeds are in the order of hundreds of Gbps; • Stringent power and space constraints; • Switched backplanes reduce the advantage of being inside the router. IMW 2001

  5. Our Goals • Be capable of collecting packet at OC-192 speed with the current PCI technology. • Develop a 4:1 compression technique. • Minimize the loss of information due to compression. IMW 2001

  6. System requirements • Online processing required for compression. • At OC-192 speed (10Gbps), a new packet arrives every 240ns (300-byte packets). • We can assume large average packet sizes because we can buffer incoming packets (once timestamped). IMW 2001

  7. From packet traces to flow traces • Store information at the flow-level: • a flow is defined by the usual 5-tuple. • Flow information are shared among all the packets belonging to the same flow. • Store few information per packet: • packet arrival time; • size; • sequence numbers • … IMW 2001

  8. Flow and packet records TCP: 28 bytes/flow; 16 bytes/pkt; UDP: 20 bytes/flow; 12 bytes/pkt; Other: 16 bytes/flow; 12 bytes/pkt; IMW 2001

  9. Flow termination • Records of terminated flows are dumped to the host memory (and then to disk). • Flow termination relies on a timeout: • A short timeout will require less memory; • A long timeout will give better compression rates; • Anyway, the timeout does not affect the accuracy of the system because we can post-process the trace; IMW 2001

  10. Flow fragmentation • Long-lived flows can be a problem: • Large records and high bursts of traffic on the PCI bus; • We set a limit to memory occupancy for a flow record (with relative packet records): • If the memory limit is exceeded, the Record Number is increased and the record is stored. • When the flow terminates the entire record is stored to disk with the Last Record flag set. IMW 2001

  11. Memory required IMW 2001

  12. Preliminary performance study • We have built a simple emulator to compress a packet-level trace. • 5 different traces with different link utilizations and traffic patterns. • Compression rates ranging between 3.20 and 4.02 (compared to DAG3 packet traces). IMW 2001

  13. Hardware architecture • Steps required in trace collection: • process framing information; • timestamp the packet; • classify the packet; • update flow/packet record; • store terminated flows to disk (background); • All these steps can be pipelined: • Each step can be performed during the interval between two packet arrivals. IMW 2001

  14. Packet classification • May be very expensive in terms of computation time and resources: • One-to-one mapping between packets and flows. • But a simple hash function may be enough: • Requires a large amount of fast memory; • Collisions can be solved using a second hash function or a lookup trie. IMW 2001

  15. Conclusion • New compression techniques are necessary to monitor OC-192/OC-768 links. • We have proposed a novel scheme for flow-based trace collection. • We have shown how to achieve compression ratio of 4:1 without losing much information. IMW 2001

  16. Open issues and future work • Detailed cost analysis (chipsets, memory). • Packet classification performance. • Second stage of record compression. • Techniques to cope with traffic anomalies. • Sampling techniques. IMW 2001

More Related