1 / 16

Building a Reliable Onboard Network with Ethernet: A GSFC Prototype

Building a Reliable Onboard Network with Ethernet: A GSFC Prototype. Jane Marquart NASA/GSFC. Requirements. Critical real-time data must be delivered reliably onboard the spacecraft (target is 5 milliseconds). Ethernet/IP Onboard Network Bus. Protocol Options.

sheba
Download Presentation

Building a Reliable Onboard Network with Ethernet: A GSFC Prototype

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Building a Reliable Onboard Network with Ethernet: A GSFC Prototype Jane Marquart NASA/GSFC

  2. Requirements • Critical real-time data must be delivered • reliably onboard the spacecraft • (target is 5 milliseconds) • Ethernet/IP Onboard Network Bus Fourth Space Internet Workshop

  3. Protocol Options TCP works but doesn’t support reliability features required by typical flight software TCP UDP is a better fit, but requires reliability be implemented by the user UDP UDP + Reliability Where to implement reliability? Application layer or data link layer? Fourth Space Internet Workshop

  4. Reliability Trade • Application layer: • Rx & tx packet latency through the stack is highly variable and non-deterministic (regardless of direction) • Latency up to several milliseconds on a MCP750 @ 233mhz • Datalink layer: • avoids IP stack latency, but; • the issue is standardization and portability of a datalink layer solution. Fourth Space Internet Workshop

  5. IEEE 802.3 LLC • LLC – “Logical Link Control” offers a IEEE standardized datalink layer reliability protocol, adding a 3 byte header to the ethernet frame • Supplies 3 major types of service; • Type 1 (unreliable packet exchange, same as plain ethernet) • Type 2 (reliable, statefull, connection oriented; used by FDDI, Wireless, Token Ring) • Type 3 (reliable, stateless, connectionless) • Type 3 selected • Stateless but reliable • Well-defined (and simple) state machine • Suitable for a variety of physical layers • Packet overhead is 3 bytes Fourth Space Internet Workshop

  6. Data-link Layer Reliability: LLC • Ack/retry algorithm implemented within the NIC driver. • Latency & delay well defined since control is right at the hardware. • Widely supported, if not tolerated, by common OS IP stacks, routers, etc. • Well defined & supported but ad-hoc mapping onto standard ethernet framing. • Reliability protocol only useable between conformant drivers. Fourth Space Internet Workshop

  7. PDU Formats Standard ethernet frame w/ EthernetType >= 0x600 Ethernet frame header Packet data Default datalink pdu Ethernet destination (6 bytes) Ethernet source (6 bytes) Ethernet type (2 bytes) User data &fill if req’d EthernetType < 0x600, set to frame length indicating a 802.2 LLC pdu Ethernet frame header LLC header Packet data Reliable pdu (data packet) ethernet source (6 bytes) ethernet type (2 bytes) ethernet dest (6 bytes) user data & fill if req’d DSAP (1 byte) SSAP (1 byte) Control (1 byte) Destination SAP Source SAP Ethernet frame header LLC header Packet data Reliable pdu (ack packet) ethernet source (6 bytes) ethernet type (2 bytes) ethernet dest (6 bytes) No user data, Fill to min. valid pkt length DSAP (1 byte) SSAP (1 byte) Control (1 byte) ack pdu's swap DSAP & SSAP, along with ethernet source/destination so reply is sent to originating host. Fourth Space Internet Workshop

  8. Ethernet/IP Multi-node Testbed SWITCH 4 Mbps MEDIA CONVERTER 150kbits 50 kbits 200 kbits MEDIA CONVERTER 4 Mbps "Comm Card" Dell PC Flight NIC Serial NIC Driver Ethernet Hub LLC Router Serial IP OS Stack 1 kbits 150 kbits Channel Link Simulator Net Mgr Serial Subsys ‘A’ Coldfire Subsys ‘B’ PC Router CI CFDP SB TO ITOS GSE CFDP Fourth Space Internet Workshop

  9. NIC Driver Architecture IP stack TX packets RX packets Async Pkt Channel Queues Packet validation & LLC processing SOIS Independent Layer LLC ack/error packets Packet Scheduler SOIS dependent Data-link layer Packet retrieved Packet submitted to NIC TX done interrupt RX packets RX interrupt TX packets Flight NIC Eth0 Eth1 Fourth Space Internet Workshop

  10. Use Case Examples nominal pkt lost Pkt sent with reliable QOS Pkt ‘N’ sent with reliable QOS Driver returns ‘ACK’ Pkt lost ACK Time-out, Resend pkt ‘N’ Log error Receive ACK Done Pkt lost pkt drop ACK Time-out, Log error Pkt sent with reliable QOS Pkt lost Pkt N+1 sent with reliable QOS Driver returns ‘ACK’; logs error ACK Time-out, Resend pkt Driver returns ‘ACK’ Receive ACK Done Receive ACK Done ACK lost Pkt ‘N’ sent with reliable QOS Driver returns ‘ACK’ ACK drop Pkt sent with reliable QOS ACK lost ACK Time- out, Resend pkt, Log error Driver returns ‘ACK’ Driver returns ‘ACK’ Discards redundant pkt Logs error ACK lost ACK Time-out, Resend pkt, Log error ACK Time- out, Log error ACK lost Driver returns ‘ACK’ Discards redundant pkt Logs error Pkt N+1 sent with reliable QOS Driver returns ‘ACK’ Receive ACK Done Receive ACK Done Fourth Space Internet Workshop

  11. NIC/OS Transmit Path Measurements User app. opens socket to send packet (UDP Client) Switch Tx_Stack_to_ Staging_Buffer Tx DMA/PIO NIC receives pkt. NIC-NIC Stack Processes Packet Tx Stack_to_NIC Tx App_to_NIC Separate test with SmartBits Analyzer Metrics: 1.) Stack Process Time 2.) Driver copy from stack to Staging buffer 3.) DMA vs. Programmed I/O time 4.) Switch Latency Key: = timestamp marker (software) = logic analyzer timestamp = start receive markers Fourth Space Internet Workshop

  12. NIC/OS Receive Path Measurements Packet Recv’d Interrupt App receives pkt. Rx DMA/PIO Rx_receive_buffer _to_stack Stack Process Time (TBD) Rx_NIC_to_Stack Metrics: Key: 1.) DMA vs. Programmed I/O 2.) Copy time from staging buffer(receive_buffer) to stack = timestamp marker (software) = start receive marker Fourth Space Internet Workshop

  13. Preliminary Performance Results With LLC Without LLC Min. Width (us) Max Width (us) Avg Width (us) Std. Dev (us) Min. Width (us) Max Width (us) Avg Width (us) Std. Dev (us) TBD TBD TBD TBD 16.560 296.400 25.264 Rx_NIC_to_BUFF (PIO or DMA)* 39.073 Rx_BUFF_to_STACK* 1.500 3.360 2.525 0.481 TBD TBD TBD TBD RX_NIC_TO_STACK* 18.481 299.581 27.843 39.156 TBD TBD TBD TBD TX_STACK_ PROCESSING** TX_STACK_TO_ STAGING BUFFER* 1.260 14.880 TBD TBD TBD TBD TBD TBD TBD TBD TBD TBD TBD TBD 2.523 2.346 TX_BUFFER_TO_NIC (PIO or DMA)* 5.100 144.600 17.256 23.849 TBD TBD TBD TBD TX_STACK_TO_NIC* 7.740 159.421 20.064 TBD TBD TBD TBD 26.148 *Numbers are for Programmed I/O (which can be up to ~4 times slower than DMA), using TCP (ftp), MTU sizes vary between 64-1500 bytes ** Numbers are for Programmed I/O, UDP, MTU sizes between 64-1500 Fourth Space Internet Workshop

  14. Status • LLC working in lab (2 millisecond timeout) • Additional performance testing includes: • LLC • Bi-directional benchmark • Worst case footprint of LLC from CPU & network throughput cost • Fixed packet size (minimum and maximum) • Linux OS (should be worse due to virtual memory model) Fourth Space Internet Workshop

  15. Conclusion Preliminary tests indicate that: the onboard network architecture, using LLC for reliability, will meet the 5 millisecond requirement for packet transfer Fourth Space Internet Workshop

  16. The Team Software Greg Menke Art Ferrer Freemon Johnson Alan Cudmore Jonathan Wilmot Jane Marquart Hardware Mike Lin Scott Edfors For more information contact: Jane Marquart at jane.marquart@nasa.gov Fourth Space Internet Workshop

More Related