an ethernet accessible control infrastructure for rapid fpga development
Download
Skip this Video
Download Presentation
An Ethernet-Accessible Control Infrastructure for Rapid FPGA Development

Loading in 2 Seconds...

play fullscreen
1 / 28

Title - PowerPoint PPT Presentation


  • 273 Views
  • Uploaded on

An Ethernet-Accessible Control Infrastructure for Rapid FPGA Development. Andrew Heckerling, Thomas Anderson, Huy Nguyen, Greg Price, Sara Siegal, John Thomas . High Performance Embedded Computing Workshop 24 September 2008.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Title' - PamelaLan


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
an ethernet accessible control infrastructure for rapid fpga development

An Ethernet-Accessible Control Infrastructure for Rapid FPGA Development

Andrew Heckerling, Thomas Anderson,Huy Nguyen, Greg Price, Sara Siegal, John Thomas

High Performance Embedded Computing Workshop

24 September 2008

This work is sponsored by the Department of the Air Force, under Air Force Contract FA8721-05-C-0002. Opinions, interpretations, conclusions, and recommendations are those of the authors and are not necessarily endorsed by the United States Air Force.

outline
Outline
  • Introduction and Motivation
  • Container Infrastructure
    • Concept
    • Implementation
  • Example Application
  • Summary
rapid advanced processor in development rapid
Rapid Advanced Processor In Development (RAPID)

RAPID Tiles and IP Library

Control

IO

Known Good Designs

Capture

Form Factor Selection

Main features of RAPID:

  • Composable processor board
    • Custom processor composed from tiles extracted from known-good boards
      • Form factor highly flexible
    • Tiles accompanied with verified firmware / software for host computer interface
  • Co-design of boards and IPs
    • Use portable FPGA Container Infrastructure to develop functional IPs
      • Container has on-chip control infrastructure, off-chip memory access, and host computer interface
    • Surrogate board can be used while target board(s) being designed (custom) or purchased (COTS)

Sig. Proc.

Custom

Composable Processor Board

VME / VPX

MicroTCA

Design

COTS Boards

FPGA Container Infrastructure

motivation
Motivation

Goal:Quick Development: 7-12 Months

Airborne Radar System Demo

Receiver Array

4 Channels

20 MHz BW

ROSA II

System Computer

Back-endProcessor

RAPIDFront-endSignal Processor

Reduce system development time in half

outline5
Outline
  • Introduction and Motivation
  • Container Infrastructure
    • Concept
    • Implementation
  • Example Application
  • Summary
fpga processing application
FPGA Processing Application

ADC

DAC

Control

Processor

FPGA

FPGA

  • FPGA processes high-speed streaming data from various sources
  • Control processor initializes, monitors, and configures FPGA
  • Control Processor gains visibility into FPGA via Controller Core
  • Controller Core provides monitoring and control of memories and functional blocks
    • Set parameters, load memory, monitor status

FPGA

Data

Data

Processor

Processor

Container Infrastructure

Controller Core

Status

Block

1

Block

2

Block

3

Off-Chip Memory

fpga container infrastructure
FPGA Container Infrastructure

Computer

FPGA Board

FPGA

Regs

RAM

Real-time Application

Debug Utility

FunctionCore

Ports

Interface

C++ interface

GigE

Controller

RDMA Library

Bus

Container

  • FPGA Function Core development can be accelerated with infrastructure provided by Container: host computer interface, on-chip registers, ports, and memory interface
  • Real-time application or debug utility can access any address (registers, ports, and memories) on the FPGA
  • Message formatting and data transfer operations are supported through Remote Direct Memory Access (RDMA) library
outline8
Outline
  • Introduction and Motivation
  • Container Infrastructure
    • Concept
    • Implementation
  • Example Application
  • Summary
motivation for memory mapped control
Motivation for Memory-Mapped Control

Graphics

Device

Address

Interconnect

  • Memory-Mapped Control Means
    • Device control via simple reads/writes to specific addresses
    • Processor and interconnect not specific to any device
      • With proper software, processor can control any device
  • Container Infrastructure extends concept to FPGA control

Data

Ethernet

Device

General

Processor

FPGA

e.g. Processor Bus,

PCI, PCI-Express, SRIO

interconnect
Interconnect
  • Interconnect Choices
    • Ethernet, Serial RapidIO, PCI Express, etc.
  • Platform-Specific Considerations
    • MicroTCA has Gigabit Ethernet channel to all payload slots, separate from higher-speed data channels

FPGA Boards

MicroTCAChassis

HUB

“Fat Pipe”datachannel

Control

Processor

Gigabit Ethernet

  • Advantages of using Gigabit Ethernet
    • Ubiquitous
    • Wide industry support
    • Easy to program
memory mapped control protocol
Memory-Mapped Control Protocol

magic

version

node number

command

address

length

flags

message tracking id

data (optional)

  • Stateless “request/response” protocol
  • Reads and writes an address range (for accessing memory) or a constant address (for accessing FIFO ports)
  • Presently implemented on top of UDP and Ethernet

MessageFormat

0

4

8

12

16

20

24

28

memory mapped control on fpga
Memory-Mapped Control on FPGA

Example FPGA Address Space

  • Each device or core has an address within the FPGA
  • Control processor refers to these addresses when reading from or writing to the FPGA

0x0

Off-chip SDRAM

Read

0x1000

Write

Control

Processor

0x2000

Mode

0x2004

Status

0x2008

Temperature

real time application example
Real-Time Application Example

Computer

FPGA Board

FPGA

Regs

RAM

Real-time Application

Debug Utility

FunctionCore

Ports

Interface

Controller

RDMA Library

Bus

Container

  • Real-Time Application uses simple C++ methods to communicate with FPGA
  • C++ interface portable to other interconnects (SRIO, PCIe)

// Create an FPGA access object

FpgaUdpReadWrite fpga(“fpga-network-address”, FPGA_UDP_PORT);

// Send input data from myBuffer to the FPGA

fpga->write(FPGA_INPUT_DATA_ADDR, INPUT_DATA_LENGTH, myBuffer);

// Read back the output data

fpga->read(FPGA_OUTPUT_DATA_DDR, OUTPUT_DATA_LENGTH, myBuffer);

command line example
Command-Line Example

Computer

FPGA Board

FPGA

Regs

RAM

Real-time Application

Debug Utility

FunctionCore

Ports

Interface

Controller

RDMA Library

Bus

Container

  • Command-line and scripting interface provides debug access to FPGA container
  • Function core can be tested before final software is written

# Send input data to the FPGAw 192.168.0.2 1001 0x0 sample_input_data.bin

# One-second delay (in ms)

P 1000

# Read back the output datar 192.168.0.2 1234 0x10000000 0x8000 result_data.dat

integrated container system
Integrated Container System

Control

Message

Ethernet

PHY

Ethernet MAC

UDP Protocol Engine

Message Encoder /

Decoder

Message

Decoding

Address

Data

Command

Streaming DMA Controller

WISHBONE

BusInterface

Wishbone Master

WISHBONE Interconnect

Wishbone Slaves

Register File

Port Array

WISHBONE / Memory Bridge

Port 2n-1

…..

Port 0

Port 1

ControlPeripherals

Memory Controller

Mode

Status

“Sticky”

Status

Processing

Application

Lincoln Laboratory IP

message decoding
Message Decoding

Control

Message

GigE

  • Inside the FPGA, the control message is decoded into a memory-mapped read or write command
  • Can mix and match components to implement different protocols

PHY

(on-chip or off-chip)

Xilinx Embedded TEMAC

UDP

Protocol

Engine

Encoder /

Decoder

Address

Data

Command

Decoded Message

wishbone bus interface
WISHBONE Bus Interface

Decoded Message

  • Streaming DMA Controller (SDMAC) handles read/write commands by generating WISHBONE bus cycles
  • WISHBONE Interconnect routes transactions to destinations based on memory map
  • Transaction block sizes range from one word (four bytes) to 8k bytes

Command

Address

Data

Streaming DMA Controller

WISHBONEMaster

WISHBONE

Interconnect

WISHBONESlaves

= WISHBONE Bus Interface

wishbone bus
WISHBONE Bus

SYSCON

RST_I

CLK_I

ARD_O()

DAT_I()

DAT_O()

WE_O

SEL_O()

STB_O

ACK_I

CYC_O

TAGN_O

TAGN_I

RST_I

CLK_I

ARD_I()

DAT_I()

DAT_O()

WE_I

SEL_I()

STB_I

ACK_O

CYC_I

TAGN_I

TAGN_O

Wishbone Master

Wishbone Slave

User Defined

IP Core Master

IP Core Master

Crossbar Switch Interconnect

IP Core Slave

IP Core Slave

IP Core Slave

  • WISHBONE is a flexible, open-source bus for system-on-chip designs
    • Specifies a logical (not electrical) interface between IP peripherals
    • WISHBONE peripherals and interconnect hubs are available on the OpenCores web site

FPGA

Wishbone Slave

IP Core

Wishbone Slave

IP Core

Wishbone Master

IP Core

Wishbone Master

IP Core

Shared Bus Interconnect

Diagrams and specification: http://www.opencores.org/

control peripherals register file and port array
Each register has an address

Registers enable / disable features, trigger processes, report condition and events

Multiple registers can be used in any combination of types

Port Array translates memory-mapped WISHBONE operations to data streams

Useful for testing computational blocks that expect data to arrive in a FIFO-like fashion

Control Peripherals:Register File and Port Array

WISHBONE Slave Interface

Port Array

Register File

Port 2n-1

…..

Port 0

Port 1

“Sticky”

Status

Mode

Status

FIFOs for streaming data

control peripherals ddr2 sdram controller interface
Control Peripherals:DDR2 SDRAM Controller Interface

WISHBONE SlaveInterface

  • High-speed processing application and lower-speed WISHBONE interface share access to DDR2 memory
    • Used to preload data into external memory for use by the processing application or for debugging
  • Xilinx memory controller interfaces to memory

Memory Bridge

DDR2

(single module

or SODIMM)

Xilinx MIG DDR2 Controller

Processing

application

resource usage on virtex 5 sx95t
Resource Usage on Virtex-5 SX95T
  • Container infrastructure consumes 7-12% of Virtex-5 SX95T depending on functionality used
  • Resource usage is constant as FPGA size increases
outline22
Outline
  • Introduction and Motivation
  • Container Infrastructure
    • Concept
    • Implementation
  • Example Application
  • Summary
rosa ii front end rapid processor
ROSA II Front-End RAPID Processor
  • ROSA II
    • Open architecture for putting a radar system together quickly
    • Interfaces and protocols are defined for subsystems
  • Front-end Processor
    • Developed with RAPID process
    • Performs Digital IQ, FIR, and Adaptive Beamforming

RAPIDFront-endSignal Processor

Receiver Array

4 Channels

20 MHz BW

ROSA II

System Computer

Back-endProcessor

RAPID Processor

spanning over multiple boards

Packet

Forming

Control

Timing signals

ADC data

Sample Timing Control

Data path

DIQ

FIR

ABF

Packet

Forming

Processed data

ADC

Analog data

rapid front end processor system
RAPID Front-End Processor System

Computer

GigE MicroTCA Hub

  • Processor is mapped to a MicroTCA system
    • Separate Control and Datapath on one Hub card
    • GigE base channel 0 is used for system control (~1 Gb/sec)
    • Serial RapidIO fat-pipe is used for datapath (~10 Gb/sec)
  • Container Infrastructure allows access to each FPGA via Gigabit Ethernet
    • High observability and controllability

Board1

Board2

Board3

Board4

Distribution

Control

Control

Control

AD1

AD3

Analog

Analog

AD2

AD4

Data path

sFPDP

Timing

Timing

Data path

Data path

Data path

sRIO MicoTCA switch

development with fpga container infrastructure
Development with FPGA Container Infrastructure

FPGA

Regs

FunctionCore

Ports

Memory

Interface

Control

Bus

Container

  • Container provides host computer access and on-chip control structure
    • Helps development of custom function cores
    • Flexible script-based method for sending test data and reading back response
  • Facilitates system-level testing with multiple data-path source and destination options
    • Data-path source and destination set via mode registers
    • Raw data from memory, FIFO ports, or stream input. Processed result to memory, FIFO ports, or stream output

GigE

GigE

RegistersFifo ports

RegistersFifo ports

DDR2ctrl

DDR2ctrl

Control

Control

Fifo ports

Fifo ports

Analog

& Timing

sRIO

Synch

DIQ-FIR

Frame

Synch

ABF

Frame

sFPDP

sRIO

CPI

sRIO

Header off

Header off

sRIO

Switch

Weights

Header

Header

FPGA

FPGA

system level benefits
System-Level Benefits
  • Eases development of Function Cores
    • Script interpreter on host computer allows easy sending of test data and reading of results
    • Incremental system integration tests with multiple data sources and output destinations
    • Estimated saving in system integration test: 2 months
  • Enables development on surrogate system(s)
    • Highly portable Container Infrastructure allows early development
    • 2 month head start while waiting for COTS system (initial capability)
    • 6-9 month head start with custom boards (full capability)

Coredge RL20

RAPID Processor

Xilinx ML506

Initial Capability

Full Capability

summary
Summary
  • Presented a Container Infrastructure for FPGA development
    • Memory-mapped control protocol for accessing FPGA registers, FIFO ports, and external DDR2 memory
  • Container Infrastructure enables fast system development
    • Helps development of FPGA function cores
    • Facilitates incremental system integration
    • Allows early FPGA development on surrogate boards
  • Future work
    • Extend framework to non-Gigabit-Ethernet channels
    • Ensure high portability and interoperability with COTS boards
    • Extend the container concept for high-speed data co-processing
acknowledgements
Acknowledgements

RAPID Team

  • Ford Ennis
  • Michael Eskowitz
  • Albert Horst
  • George Lambert
  • Larry Retherford
  • Michael Vai

UDP protocol engine

  • Timothy Schiefelbein
ad