1 / 96

SPP Version 1 Router Plans and Design

This document outlines the plans and design for SPP Version 2 router. It includes upcoming meetings, work items, and updates.

myrnao
Download Presentation

SPP Version 1 Router Plans and Design

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SPP Version 1RouterPlans and Design John DeHart

  2. SPP Versions • SPP Version 0: • What we used for SIGCOMM Paper • SPP Version 1: • Bare minimum we would need to release something to PlanetLab Users • SPP Version 2: • What we would REALLY like to release to PlanetLab users.

  3. Upcoming Meetings and Dates • techX Meetings: • 2/05/08: Status update • SPP V2 • SPP V1 • SPP V1 Control • NAT • SPP V1 next demo • ONL NPR • 2/12/08: ONL NP Router Distribution issues • 2/19/08: NAT Review • 2/26/08: SPP V1 Demo • 3/04/08: SPP V2 Review • Other dates: • 2/22/08: Department Visitors day for doctoral candidates • Clean up labs and machine rooms • Etc.

  4. Work Items (Updated 4/9/08) • SPP V2 • Design • SPP V1 • DONE: Get Init Scripts working again without Control • Remove what is not needed in init scripts • DONE: UDP Tunnel Traffic Generation

  5. 3/4/08: Fred Questions • Add info on Substrate only NPE Lookups • NPE Lookup Key: What is the 1 bit type field? • This differentiates between: • 0: Substrate only lookup: when GPE returns a packet with no reclassify • 1: Normal lookup • Slice ID: 15 bits, but we only use 12 bits, right? • Yes, it should be 12 bits and there should be 3 reserved bits. • Actually we are limiting it to 11 bits now. • Slides have been updated • Move Exception bits in Lookup  HF data • Implement LD bit in Lookup Result • What is the order of the lookup key ports ? • Slides have been updated • Lookup key fields, which come from Tunnel header and which come from encapsulated header. • First word is all Substrate Info • Rest is MN info • Why don’t we have tunnel rx IP Daddr and port number in key to fully map meta-interface. • Yes, good point. This has been raised before and we keep side stepping it. • We will use an Index in the key that selects 1 of 16 possible Rx IP DAddrs. • Slides updated but not implemented yet. • NPE Lookup Result: • What is H Flag? HIT • What is D Flag? DROP • 12 exception bits? • Yes, there are 12 exception bits but they are not part of the lookup result. They get carried over from the input ring from Parse to the output ring to HF. • MAC Address still there, but we don’t use it anymore right? • Done (removed from slides) • DONE: QParams: Have we added qlength in packets yet? • GPE VLAN issues seem to require that a different set of IP Addresses be used for each different VLAN. This might require us to put the NPE Src IP Address for packets going to the GPE in the GPE Info table and ignore what is in the Per Scheduler IP Src address table . • Can the data path do this? • Looks like it should be easy since substrate encap uses the gpe info struct to figure out which scheduler is to be used so it can read the right src_ip struct. So the data is all there at the right place already.

  6. L V 1b H V 1b T V 1b R S v 1b Unused (4b) QLength (Pkts) (28b) QM: QLength in Packets Flags: Length Valid Head Valid Tail Valid Current: QLength (Bytes) (32b) L V 1b H V 1b T V 1b Unused (29b) Threshold (Bytes) (32b) QLength (Bytes) (32b) Quantum (32b) Threshold (Bytes) (32b) Unused (32b) Quantum (32b) • QParams: Have we added qlength in packets yet? • No, I haven’t had time for that yet. QParams in SRAM QParams in Local Memory Flags(4b): Reserved Length Valid Head Valid Tail Valid Proposed: QLength (Pkts) (28b) QLength (Bytes) (32b) QLength (Bytes) (32b) Threshold (Bytes) (32b) Threshold (Bytes) (32b) Quantum (32b) Quantum (32b) QParams in Local Memory QParams in SRAM

  7. Rx IP DAddr in Key • Why don’t we have tunnel rx IP Daddr and port number in key to fully map meta-interface. • Yes, good point. This has been raised before and we keep side stepping it. To fix it for real we need to make the key larger and take the performance hit. • Actually, what we decided to do was to implement a small table (16 entries) of IP Addresses. The address that is matched by the incoming Rx IP DAddr will have its index used in the key as a 4-bit field. • This is now reflected in the slides but has not yet been implemented in the code.

  8. LD Bit in Lookup Result • Implement LD bit in Lookup Result

  9. November HW Test • First test in HW should happen in Nov. 2007 • Plan: • Retry SPP V0 demo on Dev. Chassis with new boards • Finish all three projects in Simulation: • LCI: Currently missing ICMP and NAT • LCE: • Currently missing NAT • KE and Lookup are nearing completion • HF MAC Address lookup fine tuning • Flow stats: FS2 nearing completion, needs archive thread and testing testing testing • Initialization scripts need work • NPE: • One more memory update needed for Substrate Encap (DRAM write) • Working on initialization scripts. • Testing • Test all three in simulation including initialization scripts • Convert *.ind initialization scripts to ‘cmd’ utility HW initialization scripts • Review *.ind and ‘cmd’ scripts with Fred and control group • TCAM utility for SPP V1? • Plan A: Use Jonathon’s test utility • Plan B: • Packet generation for HW test? • Use Traffic Generators? • Test all in HW

  10. November HW Test • Status (11/29/07): • DONE: Packets going through all three projects: • DONE: TCAM Utilities for all three projects seem to be working • Next Steps • DONE: NPE config/init (JDD and MLW) • Orchid (MLW) • DONE: V1 Testing (JDD) • This results in November HW Test milestone • HF MAC Address usage and Initialization cleanup (JDD) • LCI: DONE • LCE: DONE • NPE: Next • Move NPE to NPUA (JDD): DONE • We are still using the config we had to use because NPUA on the pre-production board was flakey. • Orchid Integration (MLW and JDD) On hold. • Performance testing (JDD and MLW) • Expand lookup filters to exercise all Schedulers • Need access to traffic generators or hosts with Charlie’s UDP Tunnel driver • Flow Stats testing (JDD, JM) • NAT (DMZ) • ICMP pkt handling (JDD) • LCI KeyExtract and LCE KeyExtract need to extract ICMP ID when Protocol==ICMP • Control Integration (JDD, FK, etc) In Progress • RLI Monitoring • V2: To begin January 2008 (JDD, MLW)

  11. Control Demo Notes • Fast Path changes needed: • QID changes : DONE • All three projects. Change to: • QM_ID (2b) • Sched_ID (3b) • QID (15b) • 32K Queues available with now implicit association with a QM or scheduler. • Blocks Affected: • QM: Format of Input, position of SchedID and QID • LCI: • Lookup: Format of output, change the format of TCAM result to report index in first word and results starting in second word so we can get a full 64 bit result • HF: Format of Input, Format of Output, position of VLAN in input data • PortSplitter: Format of Input, Format of Output, position of QM_ID in input data • NPE: • Lookup: Format of output • HF : Format of Input, Format of Output • SubstrateEncap : Format of Input, Format of Output, position of QM_ID in input data • LCE: • Lookup: Format of output , change the format of TCAM result to report index in first word and results starting in second word so we can get a full 64 bit result • HF: Format of Input, Format of Output, position of VLAN in input data • PortSplitter: Format of Input, Format of Output, position of QM_ID in input data • Initialization? • How to test?

  12. Control Demo Notes • Fast Path changes needed (continued): • LC HF MAC tables need to be dynamically read: DONE • LCI Needs: • Initialization of 1 SMAC Address (Fabric) • Table of per scheduler DMAC addresses (1 per scheduler) • LCE Needs: • Table of per scheduler DMAC/SMAC addresses (1 pair per scheduler) • To simplify things, lets have LCI and LCE use the same format table: • Dst MAC (8 bytes) • Src MAC (8 bytes) • NPE Substrate Encap GPE info table entries need to be moved out of Slice data space and into a separate table of their own indexed by VLAN. • DONE • MAC Address coming from NPE to Egress is wrong • FIXED • NPE Substrate Encap has hard coded DMac address. • Needs to read the MAC address from memory when it reads the IP Src address. • DONE • Remove the 8 bits of DMAC from Lookup result • DONE (no longer used) • Max Buffer Limit (todo notes) • Change JDD TCAM utilities to use DB IDs that Control uses. • DONE

  13. Notes • For NPE look at putting a limit on the number of outstanding buffers a Slice has at a time. • Add a counter to the Substrate Decap VLAN/Slice table. • When SD gets a packet, increment the counter for that Slice • When a buffer is freed have the generic buf_free code decrement the counter for that slice. • This will probably require recording the Slice ID in the buffer descriptor and having the buf_free code read the descriptor. • Look at using all 10 external interfaces on LC • Each interface that is used will be connected to different ports on the same router. • Thus the SPP node does not have to worry about participating in routing protocols in V1. • Use both fabric interfaces on GPEs • V1 will use just one of them • The one that is used will be associated with 1 external interface. • The interfaces from different GPEs may or may not be associated with different external interfaces. • There may be cases where GPEs share an IP Address • There may be cases where GPEs have different IP Addresses • We need to support both cases • Check on how we handle fragmentation • Add Ring specs to block diagram • Schedule for upcoming meetings: • 8/14: Charlie’s SIGCOMM talk and NAT • 8/21: Plugin Framework (Shakir) • 8/28: Flow Stats (JMM)

  14. SPP V1 Plans • SPP Version 1: • 1 5-Port NPE (still don’t use NPUB) • Support Multiple External IP Addresses • Switch Blade integration • 10GE Tx module integration • ARP: Probably not needed in V1 • NAT: • Flow Stats: Egress Traffic monitoring • MR Code Options • Anything new? • Control • Local Control • Booting NPU • Add/Remove Slices • MR Control • Add/Remove Routes • Node Manager • GPE • Multiple GPEs • NAT • SSH Forwarding • PLC integration • Main focus today will be on the LC: • Block/ME design • Lookups • Flow stats • ARP

  15. Cycle Budget (min eth packets) • To hit 5 Gb rate: • 76B per min IPv4 packet (64 min Eth + 12B IFS) • 1.4Ghz clock rate • 5 Gb/sec * 1B/8b * packet/76B = 8.22 Mp/sec • 1.4Gcycle/sec * 1 sec/ 8.22 Mp = 170.3 cycles per packet • compute budget: 170 cycles • latency budget: (threads*170) • 8 threads: 1360 cycles • To hit 10 Gb rate: • 76B per min IPv4 packet (64 min Eth + 12B IFS) • 1.4Ghz clock rate • 10 Gb/sec * 1B/8b * packet/76B = 16.44 Mp/sec • 1.4Gcycle/sec * 1 sec/ 16.44 Mp = 85.16 cycles per packet • compute budget: 85 cycles • latency budget: (threads*85) • 8 threads: 680 cycles

  16. Cycle Budget (IPv4 MN packets) • To hit 5 Gb rate: • 90B per min IPv4 packet (78 min IPv4MN + 12B IFS) • 1.4Ghz clock rate • 5 Gb/sec * 1B/8b * packet/90B = 6.94 Mp/sec • 1.4Gcycle/sec * 1 sec/ 6.94 Mp = 201.7 cycles per packet • compute budget: 201 cycles • latency budget: (threads*201) • 8 threads: 1608 cycles • To support 6.94 M pkts/sec • we can Read 28 Words and Write 28 Words per pkt per SRAM Bank • (200M/6.94M) = 28.818 • To hit 10 Gb rate: • 90B per min IPv4 packet (78 min IPv4MN + 12B IFS) • 1.4Ghz clock rate • 10 Gb/sec * 1B/8b * packet/90B = 13.88 Mp/sec • 1.4Gcycle/sec * 1 sec/ 13.88 Mp = 100.86 cycles per packet • compute budget: 100 cycles • latency budget: (threads*100) • 8 threads: 800 cycles • To support 13.88 M pkts/sec • we can Read 14 Words and Write 14 Words per pkt per SRAM Bank • (200M/13.88M) = 14.409

  17. Cycle Budget (Average Pkts) • To hit 5 Gb rate: • 218B per min IPv4 packet (200 avg IPv4MN + 12B IFS) • 1.4Ghz clock rate • 5 Gb/sec * 1B/8b * packet/218B = 2.87 Mp/sec • 1.4Gcycle/sec * 1 sec/ 2.87 Mp = 487.8 cycles per packet • compute budget: 487 cycles • latency budget: (threads*487) • 8 threads: 3896 cycles • To support 2.87 M pkts/sec • we can Read 69 Words and Write 69 Words per pkt per SRAM Bank • (200M/2.87M) = 69.686 • To hit 10 Gb rate: • 218B per min IPv4 packet (200 avg IPv4MN + 12B IFS) • 1.4Ghz clock rate • 10 Gb/sec * 1B/8b * packet/218B = 5.74 Mp/sec • 1.4Gcycle/sec * 1 sec/ 5.74 Mp = 243.9 cycles per packet • compute budget: 243 cycles • latency budget: (threads*243) • 8 threads: 1944 cycles • To support 5.74 M pkts/sec • we can Read 34 Words and Write 34 Words per pkt per SRAM Bank • (200M/5.74M) = 34.843

  18. SPP V1 ARP Notes • Statically configure the Ethernet Addr of next hop(s). • Don’t need ARP in V1. • LC uses scheme similar to ONL • LCE Lookup result contains Next Hop IP or NH Ethernet Addr. • If NH Ethernet Addr is present than update packet and send • If NH IP Addr present instead of NH Ethernet Addr then send to XScale • Need to define shim/descriptor for LCE to XScale • Physical Interface • NH IP Address • … • XScale will send ARP Broadcast on physical interface • LCI receives Unicast ARP Response from RTM Port • Sends to XScale indicating which physical interface recv’d on. • XScale updates filter table • If XScale has waiting packet, send to data path. • LCI receives ARP Broadcast from RTM port • Sends to XScale indicating which physical interface recv’d on. • XScale processes and sends ARP Response if needed. • ARP Entry Aging.

  19. SPP V1 ARP Interfaces • LCI to XScale Interface • LCI just needs to detect EtherType Field of ARP • Should be able to do this in Key Extract. • Code already there to detect ARP and send to XScale. • We may have to adjust for shim/descriptor to communicate additional info to XScale • LCE to XScale Interface • Needs to be Post Lookup and Pre QM. • Needs to update the Shim/Descriptor to send info to the XScale. • Hdr Format is probably the best place for this. • XScale to LCE Interface • Should be Queued to keep Port rate control sane. • Does it need to be a separate scratch ring or can it go directly into the QM input ring(s)?

  20. SPP V1 NAT Notes • NAT Notes moved to separate file: • SPP_V1_NAT_design.ppt • http://www.arl.wustl.edu/projects/techX/design/SPP/SPP_V1_NAT_design.ppt

  21. XScale NAT Scratch Rings SCR SCR SCR SCR SCR SCR SCR SCR NN NN NN NN NN NN NN SPP V1 LC Ingress(1x10Gb/s and 10x1Gb/s) R B U F R T M M S F Rx1 Rx2 Key Extract Lookup Hdr Format TCAM S W I T C H T B U F QM0 Scr2NN Port Splitter M S F 1x10G Tx2 1x10G Tx1 QM1 QM2 QM3 Stats (1 ME) SRAM1 SRAM3 SRAM2

  22. XScale NAT Scratch Rings SCR SCR SCR SCR SCR SCR SCR SCR SRAM SRAM SCR NN NN NN NN NN NN NN Freelist SPP V1 LC Egress with 1x10Gb/s Tx S W I T C H R B U F M S F Rx1 Rx2 Key Extract Lookup Hdr Format TCAM T B U F QM0 Port Splitter Flow Stats1 R T M M S F 1x10G Tx2 1x10G Tx1 QM1 QM2 QM3 Stats (1 ME) SRAM1 SRAM3 Flow Stats2 SRAM2 XScale Archive Records

  23. XScale NAT Scratch Rings SCR SCR SCR SCR SCR SCR SCR SCR SCR SRAM SRAM SCR SCR NN NN NN NN NN Freelist SPP V1 LC Egress with 10x1Gb/s Tx S W I T C H R B U F M S F Rx1 Rx2 Key Extract Lookup Hdr Format TCAM 5x1G Tx1 (P0-P4) T B U F QM0 Port Splitter Flow Stats1 R T M M S F QM1 5x1G Tx2 (P5-P9) QM2 QM3 Stats (1 ME) SRAM1 SRAM3 Flow Stats2 SRAM2 XScale Archive Records

  24. INGRESS Block Interfaces

  25. XScale NAT Scratch Rings SCR SCR SCR SCR SCR SCR SCR SCR NN NN NN NN NN NN NN SPP V1 LC Ingress(1x10Gb/s and 10x1Gb/s) R B U F R T M M S F Rx1 Rx2 Key Extract Lookup Hdr Format TCAM S W I T C H T B U F QM0 Scr2NN Port Splitter M S F 1x10G Tx2 1x10G Tx1 QM1 QM2 QM3 Stats (1 ME) SRAM1 SRAM3 SRAM2

  26. Notes on Frame vs. Pkt Lengths • RX reports Ethernet Frame Length • KE passes along IP Pkt length and Ethernet Hdr Length • HF uses Ethernet Hdr Length and Buffer Offset to find start of IP Pkt so it can put on new ethernet header. • HF passes along Ethernet Frame Length • TX needs Ethernet Frame Length which it gets from buffer descriptor Buffer Size • QM Dequeue gets length from buffer descriptor • Thus it will get Ethernet Frame Length just like TX • QM Enqueue gets a length from input ring which must agree with what QM Dequeue gets from buffer descriptor. • Thus: HF must pass Ethernet Frame length in output ring AND it must write it to buffer descriptor. • QM Link rates should include IFS, etc.

  27. XScale Rx Flags (8b) Buf Handle(24b) NAT Scratch Rings Eth. Frame Len (16b) Reserved (12b) Intf (4b) SCR SCR SCR SCR SCR SCR SCR SCR NN NN NN NN NN NN NN SPP V1 LC Ingress(1x10Gb/s and 10x1Gb/s) R B U F R T M M S F Rx1 Rx2 Key Extract Lookup Hdr Format TCAM S W I T C H T B U F QM0 Scr2NN Port Splitter M S F 1x10G Tx2 1x10G Tx1 QM1 QM2 QM3 Stats (1 ME) SRAM1 SRAM3 SRAM2

  28. XScale Rx Flags (8b) Buf Handle(24b) NAT Scratch Rings Eth. Frame Len (16b) Reserved (12b) Intf (4b) SCR SCR SCR SCR SCR SCR SCR SCR NN NN NN NN NN NN NN SPP V1 LC Ingress(1x10Gb/s and 10x1Gb/s) Flags (8b) Buf Handle(24b) IP Pkt Length (16b) Eth Hdr Len (8b) Rsv (4b) Intf (4b) Lookup Key IP DAddr (32b) Protocol (8b) UDP DPort (16b) Type (8b) IP Hdr 1st Word (32b) R B U F IP Hdr 2nd Word (32b) R T M M S F Rx1 Rx2 Key Extract Lookup Hdr Format TCAM S W I T C H T B U F QM0 Scr2NN Port Splitter M S F 1x10G Tx2 1x10G Tx1 QM1 QM2 QM3 Stats (1 ME) SRAM1 SRAM3 SRAM2

  29. Lookup Result Lookup Key SCR SCR SCR SCR SCR SCR SCR SCR NN NN NN NN NN NN NN SPP V1 LC Ingress XScale ICMP ERR Flags(8b) Rsvd 2b IE 1b T 1b U 1b I 1b N 1b H 1b NAT Scratch Rings Rsv 1b IE 1b TCP Flags 6b Hit NAT UDP TCP ICMP ICMP ERR R B U F R T M M S F Rx1 Rx2 Key Extract Lookup Hdr Format Flags (8b) Buf Handle(24b) Flags (8b) Buf Handle(24b) TCAM Rsv (4b) IP Pkt Length (16b) Eth Hdr Len (8b) Intf (4b) Eth Hdr Len (8b) Reserved (8b) IP Pkt Length (16b) IP DAddr (32b) VLAN (12b) QM 2b Sch 3b PerSchedQID (15b) Protocol (8b) TCP/UDP DPort Or ICMP ID (16b) ICMP Type (8b) Translated DPort/ID (16b) Stats Index (16b) S W I T C H Scr2NN T B U F QM0 IP SAddr (32b) Port Splitter IP Hdr 1st Word (32b) M S F IP Hdr 1st Word (32b) 1x10G Tx2 1x10G Tx1 IP Hdr Top 16 bits Of 2nd Word (16b) QM1 Original DPort/ID (16b) IP Hdr Top 16 bits Of 2nd Word (16b) TCP/UDP SPort (16b) QM2 QM3 Stats (1 ME) SRAM1 SRAM3 SRAM2

  30. H 1b Rsv 1b TCP Flags 6b VLAN (12b) QM 2b Sch 3b PerSchedQID (15b) U 1b A 1b P 1b R 1b S 1b F 1b Hit URG SYN ACK PSH RST FIN Hit NAT UDP TCP ICMP Translated DPort/ID (16b) Stats Index (16b) SCR SCR SCR SCR SCR SCR SCR SCR NN NN NN NN NN NN NN SPP V1 LC Ingress XScale Flags (8b) Reserved (8b) Buf Handle(24b) NAT Scratch Rings Rsv (4b) IP Pkt Length (16b) Eth Hdr Len (8b) Intf (4b) IP DAddr (32b) R B U F Protocol (8b) TCP/UDP DPort Or ICMP ID (16b) ICMP Type (8b) R T M M S F Rx1 Rx2 Key Extract Lookup Hdr Format IP_SAddr (32b) IP Hdr 1st Word (32b) IP Hdr Top 16 bits Of 2nd Word (16b) TCP/UDP SPort (16b) Flags (8b) Buf Handle(24b) TCAM TCAM Hit Index (32b) Eth Hdr Len (8b) Reserved (8b) IP Pkt Length (16b) ICMP ERR Rsvd 2b IE 1b T 1b U 1b I 1b N 1b H 1b S W I T C H Scr2NN T B U F QM0 Port Splitter IP Hdr 1st Word (32b) M S F 1x10G Tx2 1x10G Tx1 IP Hdr Top 16 bits Of 2nd Word (16b) QM1 Original DPort/ID (16b) QM2 QM3 Stats (1 ME) SRAM1 SRAM3 SRAM2

  31. XScale VLAN (12b) QM 2b Sch 3b PerSchedQID (15b) NAT Scratch Rings QM 2b Sch 3b PerSchedQID (15b) Reserved (12b) SCR SCR SCR SCR SCR SCR SCR SCR NN NN NN NN NN NN NN SPP V1 LC Ingress(1x10Gb/s and 10x1Gb/s) Flags (8b) Buf Handle(24b) Eth Hdr Len (8b) Reserved (8b) IP Pkt Length (16b) Translated DPort/ID (16b) Stats Index (16b) IP Hdr 1st Word (32b) R B U F IP Hdr 2nd Word (32b) R T M M S F Rx1 Rx2 Key Extract Lookup Hdr Format Reserved (8b) Buffer Handle(24b) TCAM Frame Length (16b) Stats Index (16b) S W I T C H T B U F QM0 Scr2NN Port Splitter M S F 1x10G Tx2 1x10G Tx1 QM1 QM2 QM3 Stats (1 ME) SRAM1 SRAM3 SRAM2

  32. XScale NAT Scratch Rings QM 2b QM 2b Sch 3b Sch 3b PerSchedQID (15b) PerSchedQID (15b) Reserved (12b) Reserved (12b) SCR SCR SCR SCR SCR SCR SCR SCR NN NN NN NN NN NN NN SPP V1 LC Ingress(1x10Gb/s and 10x1Gb/s) R B U F R T M M S F Rx1 Rx2 Key Extract Lookup Hdr Format Reserved (8b) Buffer Handle(24b) TCAM Frame Length (16b) Stats Index (16b) S W I T C H T B U F QM0 Port Splitter Scr2NN M S F 1x10G Tx2 1x10G Tx1 QM1 QM2 QM3 Reserved (8b) Buffer Handle(24b) Stats (1 ME) SRAM1 SRAM3 SRAM2 Frame Length (16b) Stats Index (16b)

  33. XScale NAT Scratch Rings QM 2b Sch 3b PerSchedQID (15b) Reserved (12b) V 1 Rsv (3b) Intf (4b) Buffer Handle(24b) SCR SCR SCR SCR SCR SCR SCR SCR NN NN NN NN NN NN NN SPP V1 LC Ingress(1x10Gb/s and 10x1Gb/s) R B U F R T M M S F Rx1 Rx2 Key Extract Lookup Hdr Format Reserved (8b) Buffer Handle(24b) Frame Length (16b) TCAM Stats Index (16b) S W I T C H T B U F QM0 Scr2NN Port Splitter M S F 1x10G Tx2 1x10G Tx1 QM1 QM2 QM3 Stats (1 ME) SRAM1 SRAM3 SRAM2

  34. XScale NAT Scratch Rings V 1 V 1 Rsv (3b) Rsv (3b) Intf (4b) Intf (4b) Buffer Handle(24b) Buffer Handle(24b) SCR SCR SCR SCR SCR SCR SCR SCR NN NN NN NN NN NN NN SPP V1 LC Ingress(1x10Gb/s and 10x1Gb/s) R B U F R T M M S F Rx1 Rx2 Key Extract Lookup Hdr Format TCAM S W I T C H T B U F QM0 Scr2NN Port Splitter M S F 1x10G Tx2 1x10G Tx1 QM1 QM2 QM3 Stats (1 ME) SRAM1 SRAM3 SRAM2

  35. XScale NAT Scratch Rings Opcode (4b) Data (12b) Stats Index (16b) SCR SCR SCR SCR SCR SCR SCR SCR NN NN NN NN NN NN NN SPP V1 LC Ingress(1x10Gb/s and 10x1Gb/s) R B U F R T M M S F Rx1 Rx2 Key Extract Lookup Hdr Format TCAM S W I T C H T B U F QM0 Scr2NN Port Splitter M S F 1x10G Tx2 1x10G Tx1 QM1 QM2 QM3 Stats (1 ME) SRAM1 SRAM3 SRAM2

  36. EGRESS Block Interfaces

  37. Egress Buffer Descriptor Buffer_Next (32b) LW0 Buffer_Size (16b) Offset (16b) LW1 Packet_Size (16b) Free_list 0000 (4b) Reserved (4b) Reserved (8b) LW2 Reserved (4b) SliceID (VLAN) (12b) Stats Index (16b) LW3 Reserved (16b) Reserved (8b) Reserved (4b) Reserved (4b) LW4 Reserved (32b) Reserved (4b) Reserved (4b) LW5 Reserved (16b) Reserved (16b) LW6 Packet_Next (32b) LW7

  38. XScale NAT Scratch Rings SCR SCR SCR SCR SCR SCR SCR SCR SRAM SRAM SCR NN NN NN NN NN NN NN Freelist SPP V1 LC Egress with 1x10Gb/s Tx ToXScale:43 From:XS:44 S W I T C H R B U F M S F ToHF:33 Rx1 Rx2 Key Extract Lookup Hdr Format From:LK:34 DROP:13 DROP:14 IN:F2 DROP:12 TCAM T B U F QM0 Port Splitter Flow Stats1 R T M M S F 1x10G Tx2 1x10G Tx1 QM1 QM2 QM3 Rcv: 25 Stats (1 ME) SRAM1 SRAM3 Flow Stats2 SRAM2 XScale Archive Records

  39. XScale NAT Scratch Rings SCR SCR SCR SCR SCR SCR SCR SCR SCR SRAM SRAM SCR SCR NN NN NN NN NN Freelist SPP V1 LC Egress with 10x1Gb/s Tx S W I T C H R B U F M S F Rx1 Rx2 Key Extract Lookup Hdr Format TCAM 5x1G Tx1 (P0-P4) T B U F QM0 Port Splitter Flow Stats1 R T M M S F QM1 5x1G Tx2 (P5-P9) QM2 QM3 Stats (1 ME) SRAM1 SRAM3 Flow Stats2 SRAM2 XScale Archive Records

  40. XScale NAT Scratch Rings Rx Flags (8b) Buf Handle(24b) Eth. Frame Len (16b) Reserved (12b) Port (4b) SCR SCR SCR SCR SCR SCR SCR SCR SCR SRAM SRAM SCR SCR NN NN NN NN NN Freelist SPP V1 LC Egress with 10x1Gb/s Tx S W I T C H R B U F M S F Rx1 Rx2 Key Extract Lookup Hdr Format TCAM 5x1G Tx1 (P0-P4) T B U F QM0 Port Splitter Flow Stats1 R T M M S F QM1 5x1G Tx2 (P5-P9) QM2 QM3 Stats (1 ME) SRAM1 SRAM3 Flow Stats2 SRAM2 XScale Archive Records

  41. XScale NAT Scratch Rings Rx Flags (8b) Buf Handle(24b) Eth. Frame Len (16b) Reserved (12b) Port (4b) Reserved (8b) Buf Handle(24b) IP Pkt Length (16b) Eth Hdr Len (8b) SrcMAC (8b) IP_SAddr (32b) IP Proto (8b) UDP SPort (16b) Type(8b) IP DAddr (32b) IP Hdr 1st Word (32b) SCR IP Hdr 2nd Word (32b) SCR SCR SCR SCR SCR SCR SCR SCR SRAM SRAM SCR SCR NN NN NN NN NN Freelist SPP V1 LC Egress with 10x1Gb/s Tx S W I T C H R B U F M S F Rx1 Rx2 Key Extract Lookup Hdr Format TCAM Lookup Key 5x1G Tx1 (P0-P4) T B U F QM0 Port Splitter Flow Stats1 R T M M S F QM1 5x1G Tx2 (P5-P9) QM2 QM3 Stats (1 ME) SRAM1 SRAM3 Flow Stats2 SRAM2 XScale Archive Records

  42. Reserved 5b I 1b N 1b H 1b VLAN (12b) QM 2b Sch 3b PerSchedQID (15b) Translated SPort(16b) Stats Index (16b) SCR SCR SCR SCR SCR SCR SCR SCR SRAM SRAM SCR SCR NN NN NN NN NN Freelist SPP V1 LC Egress with 10x1Gb/s Tx Flags (8b) Buf Handle(24b) H IP Pkt Length (16b) Eth Hdr Len (8b) Reserved (8b) XScale Lookup Result Flags(8b) Hit NAT Miss Scratch Ring NAT ICMP IP DAddr (32b) S W I T C H IP Hdr 1st Word (32b) R B U F IP Hdr Top 16 bits Of 2nd Word (16b) Rsv (4b) SliceID (12b) M S F Rx1 Rx2 Key Extract Lookup Hdr Format Reserved (8b) Buf Handle(24b) TCAM IP Pkt Length (16b) Eth Hdr Len (8b) SrcMAC (8b) IP_SAddr (32b) IP Proto (8b) UDP SPort (16b) Type(8b) 5x1G Tx1 (P0-P4) T B U F QM0 Port Splitter Flow Stats1 R T M M S F IP DAddr (32b) QM1 IP Hdr 1st Word (32b) 5x1G Tx2 (P5-P9) QM2 IP Hdr Top 16 bits Of 2nd Word (16b) Rsv (4b) SliceID (12b) QM3 NAT Pkt return Stats (1 ME) SRAM1 SRAM3 Flow Stats2 XScale SRAM2 XScale Archive Records

  43. Lookup Result VLAN (12b) QM 2b Sch 3b PerSchedQID (15b) Translated SPort(16b) Stats Index (16b) Hit NAT UDP TCP ICMP SCR SCR SCR SCR SCR SCR SCR SCR SCR SRAM SRAM SCR SCR NN NN NN NN NN Freelist Proposed Change: SPP V1 LC Egress H XScale ICMP ERR Flags(8b) Rsvd 2b IE 1b T 1b U 1b I 1b N 1b H 1b NAT Scratch Rings Rsv 1b IE 1b TCP Flags 6b ICMP ERR S W I T C H R B U F M S F Rx1 Rx2 Key Extract Lookup Hdr Format Flags (8b) Buf Handle(24b) Reserved (8b) Buf Handle(24b) IP Pkt Length (16b) TCAM Eth Hdr Len (8b) Reserved (8b) IP Pkt Length (16b) Eth Hdr Len (8b) SrcMAC (8b) IP_SAddr (32b) IP Proto (8b) TCP/UDP SPort Or ICMP ID (16b) ICMP Type (8b) 5x1G Tx1 (P0-P4) T B U F QM0 IP DAddr (32b) Port Splitter Flow Stats1 R T M M S F IP DAddr (32b) IP Hdr 1st Word (32b) QM1 IP Hdr 1st Word (32b) IP Hdr Top 16 bits Of 2nd Word (16b) Original SPort/ID (16b) 5x1G Tx2 (P5-P9) QM2 IP Hdr Top 16 bits Of 2nd Word (16b) Reserved (16b) TCP/UDP DPort (16b) QM3 NAT Pkt return Stats (1 ME) SRAM1 SRAM3 Flow Stats2 XScale SRAM2 XScale Archive Records

  44. H 1b Rsv 1b TCP Flags 6b VLAN (12b) QM 2b Sch 3b PerSchedQID (15b) U 1b A 1b P 1b R 1b S 1b F 1b Hit Translated SPort(16b) Stats Index (16b) URG SYN ACK PSH RST FIN Hit NAT UDP TCP ICMP SCR SCR SCR SCR SCR SCR SCR SCR SCR SCR SRAM SRAM SCR SCR NN NN NN NN NN Freelist Proposed Change: SPP V1 LC Egress Flags (8b) Buf Handle(24b) XScale IP Pkt Length (16b) Eth Hdr Len (8b) SrcMAC (8b) NAT Scratch Rings IP_SAddr (32b) IP Proto (8b) TCP/UDP SPort Or ICMP ID (16b) ICMP Type(8b) S W I T C H R B U F M S F IP_DAddr (32b) Rx1 Rx2 Key Extract Lookup Hdr Format IP Hdr 1st Word (32b) IP Hdr Top 16 bits Of 2nd Word (16b) TCP/UDP DPort (16b) Flags (8b) Buf Handle(24b) TCAM Hit Index (32b) TCAM IP Pkt Length (16b) Eth Hdr Len (8b) Reserved (8b) ICMP ERR Rsvd 2b T 1b U 1b I 1b N 1b H 1b 5x1G Tx1 (P0-P4) IE 1b T B U F QM0 Port Splitter IP DAddr (32b) Flow Stats1 R T M M S F IP Hdr 1st Word (32b) QM1 IP Hdr Top 16 bits Of 2nd Word (16b) Original SPort/ID (16b) 5x1G Tx2 (P5-P9) QM2 QM3 NAT Pkt return Stats (1 ME) SRAM1 SRAM3 Flow Stats2 XScale SRAM2 XScale Archive Records

  45. Lookup Result VLAN (12b) QM 2b Sch 3b PerSchedQID (15b) Reserved 3b T 1b U 1b I 1b N 1b H 1b Translated SPort(16b) Stats Index (16b) Hit NAT UDP TCP ICMP SCR SCR SCR SCR SCR SCR SCR SCR SCR SRAM SRAM SCR SCR NN NN NN NN NN Freelist NAT Changes: SPP V1 LC Egress H XScale Flags(8b) NAT Scratch Rings S W I T C H R B U F M S F Rx1 Rx2 Key Extract Lookup Hdr Format Flags (8b) Buf Handle(24b) Reserved (8b) Buf Handle(24b) IP Pkt Length (16b) TCAM Eth Hdr Len (8b) Reserved (8b) IP Pkt Length (16b) Eth Hdr Len (8b) SrcMAC (8b) IP_SAddr (32b) IP Proto (8b) TCP/UDP SPort Or ICMP ID (16b) ICMP Type (8b) 5x1G Tx1 (P0-P4) T B U F QM0 IP DAddr (32b) Port Splitter Flow Stats1 R T M M S F IP DAddr (32b) IP Hdr 1st Word (32b) QM1 IP Hdr 1st Word (32b) IP Hdr Top 16 bits Of 2nd Word (16b) Reserved (16b) 5x1G Tx2 (P5-P9) QM2 IP Hdr Top 16 bits Of 2nd Word (16b) Reserved (16b) QM3 Stats (1 ME) SRAM1 SRAM3 Flow Stats2 SRAM2 XScale Archive Records

  46. H 1b Rsv 1b TCP Flags 6b VLAN (12b) QM 2b Sch 3b PerSchedQID (15b) U 1b A 1b P 1b R 1b S 1b F 1b Hit Reserved 3b T 1b U 1b I 1b N 1b H 1b Translated SPort(16b) Stats Index (16b) URG SYN ACK PSH RST FIN Hit NAT UDP TCP ICMP Flags (8b) Buf Handle(24b) IP Pkt Length (16b) Eth Hdr Len (8b) SrcMAC (8b) IP_SAddr (32b) IP Proto (8b) TCP/UDP SPort Or ICMP ID (16b) ICMP Type(8b) IP_DAddr (32b) IP Hdr 1st Word (32b) SCR IP Hdr Top 16 bits Of 2nd Word (16b) TCP/UDP DPort (16b) SCR SCR SCR SCR SCR SCR SCR SCR SCR TCAM Hit Index (32b) SRAM SRAM SCR SCR NN NN NN NN NN Freelist NAT Change: SPP V1 LC Egress XScale NAT Scratch Rings S W I T C H R B U F M S F Rx1 Rx2 Key Extract Lookup Hdr Format Flags (8b) Buf Handle(24b) TCAM IP Pkt Length (16b) Eth Hdr Len (8b) Reserved (8b) 5x1G Tx1 (P0-P4) T B U F QM0 Port Splitter IP DAddr (32b) Flow Stats1 R T M M S F IP Hdr 1st Word (32b) QM1 IP Hdr Top 16 bits Of 2nd Word (16b) Reserved (16b) 5x1G Tx2 (P5-P9) QM2 QM3 Stats (1 ME) SRAM1 SRAM3 Flow Stats2 SRAM2 XScale Archive Records

  47. VLAN (12b) QM 2b Sch 3b PerSchedQID (15b) Translated SPort(16b) Stats Index (16b) Reserved (8b) Buf Handle(24b) IP Pkt Length (16b) Eth Hdr Len (8b) Reserved (8b) SCR SCR SCR SCR SCR SCR SCR SCR SCR SRAM SRAM SCR SCR NN NN NN NN NN Freelist SPP V1 LC Egress with 10x1Gb/s Tx XScale NAT MISS! NAT Miss Scratch Ring Flags (8b) S W I T C H R B U F M S F Rx1 Rx2 Key Extract Lookup Hdr Format Flags (8b) Buf Handle(24b) IP Pkt Length (16b) TCAM Eth Hdr Len (8b) Reserved (8b) 5x1G Tx1 (P0-P4) T B U F QM0 IP DAddr (32b) Port Splitter Flow Stats1 R T M M S F IP Hdr 1st Word (32b) QM1 IP Hdr Top 16 bits Of 2nd Word (16b) Rsv (4b) SliceID (12b) 5x1G Tx2 (P5-P9) QM2 QM3 NAT Pkt return Stats (1 ME) SRAM1 SRAM3 Flow Stats2 XScale SRAM2 XScale Archive Records

  48. XScale VLAN (12b) QM 2b Sch 3b PerSchedQID (15b) NAT Scratch Rings Translated SPort(16b) Stats Index (16b) QM 2b Sch 3b PerSchedQID (15b) Reserved (12b) SCR SCR SCR SCR SCR SCR SCR SCR SCR SRAM SRAM SCR SCR NN NN NN NN NN Freelist SPP V1 LC Egress with 10x1Gb/s Tx Flags (8b) Buf Handle(24b) IP Pkt Length (16b) Eth Hdr Len (8b) Reserved (8b) IP DAddr (32b) S W I T C H IP Hdr 1st Word (32b) R B U F IP Hdr Top 16 bits Of 2nd Word (16b) Rsv (4b) SliceID (12b) M S F Rx1 Rx2 Key Extract Lookup Hdr Format Reserved (8b) Buffer Handle(24b) TCAM Ethernet Frame Length (16b) Cntr Index (16b) 5x1G Tx1 (P0-P4) T B U F QM0 Port Splitter Flow Stats1 R T M M S F QM1 5x1G Tx2 (P5-P9) QM2 QM3 Stats (1 ME) SRAM1 SRAM3 Flow Stats2 SRAM2 XScale Archive Records

  49. XScale NAT Scratch Rings QM 2b QM 2b Sch 3b Sch 3b PerSchedQID (15b) PerSchedQID (15b) Reserved (12b) Reserved (12b) SCR SCR SCR SCR SCR SCR SCR SCR SCR SRAM SRAM SCR SCR NN NN NN NN NN Freelist SPP V1 LC Egress with 10x1Gb/s Tx S W I T C H R B U F M S F Rx1 Rx2 Key Extract Lookup Hdr Format Reserved (8b) Buffer Handle(24b) TCAM Ethernet Frame Length (16b) Cntr Index (16b) 5x1G Tx1 (P0-P4) T B U F QM0 Port Splitter Flow Stats1 R T M M S F QM1 5x1G Tx2 (P5-P9) QM2 QM3 Reserved (8b) Buffer Handle(24b) Stats (1 ME) SRAM1 SRAM3 Flow Stats2 Ethernet Frame Length (16b) Cntr Index (16b) SRAM2 XScale Archive Records

  50. XScale NAT Scratch Rings V 1 Rsv (3b) Intf (4b) Buffer Handle(24b) QM 2b Sch 3b PerSchedQID (15b) Reserved (12b) SCR SCR SCR SCR SCR SCR SCR SCR SCR SRAM SRAM SCR SCR NN NN NN NN NN Freelist SPP V1 LC Egress with 10x1Gb/s Tx S W I T C H R B U F M S F Reserved (8b) Buffer Handle(24b) Rx1 Rx2 Key Extract Lookup Hdr Format Ethernet Frame Length (16b) Cntr Index (16b) TCAM 5x1G Tx1 (P0-P4) T B U F QM0 Port Splitter Flow Stats1 R T M M S F QM1 5x1G Tx2 (P5-P9) QM2 QM3 Stats (1 ME) SRAM1 SRAM3 Flow Stats2 SRAM2 XScale Archive Records

More Related