sift fpgas and forth n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
SIFT, FPGAs and Forth PowerPoint Presentation
Download Presentation
SIFT, FPGAs and Forth

Loading in 2 Seconds...

play fullscreen
1 / 24

SIFT, FPGAs and Forth - PowerPoint PPT Presentation


  • 790 Views
  • Uploaded on

SIFT, FPGAs and Forth. Dave Wyland Reasonable Machines. Overview. SIFT algorithm for Object Recognition SIFT: FPGA vs DSP Forth Machine in FPGA SIFT on the R1A1 - The R1A1 Robot Project SIFT and R1A1 Status Implications for Robotics and HBRC. SIFT – What Is It?.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'SIFT, FPGAs and Forth' - Ava


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
sift fpgas and forth
SIFT, FPGAs and Forth

Dave Wyland Reasonable Machines

© David C. Wyland 2006

overview
Overview
  • SIFT algorithm for Object Recognition
  • SIFT: FPGA vs DSP
  • Forth Machine in FPGA
  • SIFT on the R1A1 - The R1A1 Robot Project
  • SIFT and R1A1 Status
  • Implications for Robotics and HBRC

© David C. Wyland 2006

sift what is it
SIFT – What Is It?
  • An algorithm for object recognition
    • Developed by David Lowe of University of BC
  • Identifies objects using ordinary cameras (TV, webcam)
    • At any distance
    • For any orientation (pose)
    • For (almost) any lighting
    • With simple image distortions

© David C. Wyland 2006

sift what good is it
SIFT – What Good Is It?
  • Lets us recognize objects automatically
  • Gives bearing and approximate range with 1 camera
  • Gives pose of object: normal, tilted, etc.
  • Lets our robots see what we see
  • Lets us build robots to go to named objects, pick them up

© David C. Wyland 2006

sift how does it work
SIFT – How does it work?
  • Outline
    • Find keypoints using Difference of Gaussians
      • 2D Gaussian filters of image
    • Look at 16x16 patch around keypoint
    • Generate a 128 dimension feature vector per point
    • Record new or search for match among recorded

© David C. Wyland 2006

sift references
SIFT – References
  • Lowe, David, “Distinctive Image Features from Scale Invariant Keypoints,” International Journal of Computer Vision, 2004.
  • Google for SIFT, Scale Invariant Feature Transform, David Lowe

© David C. Wyland 2006

sift how much computation
SIFT – How Much Computation?
  • Steps in Algorithm
    • Keypoint finding: 49.6 x 10^6 pixel operations
    • Keypoint descriptor generation: 1.4 x 10^6 pix ops
    • Keypoint search: 100 ref objects: 32 x 10^6 compares

© David C. Wyland 2006

sift fpga vs dsp
SIFT – FPGA vs DSP
  • 600 MHz ADI Blackfin DSP vs Xilinx Spartan 3 FPGA
    • Time for DoG on 640x480 Pyramid, 3 planes/level
    • Blackfin: 12 frames/second at 600 MIPs
    • FPGA: 41 frames/second at 50 MHz

© David C. Wyland 2006

fpga characteristics
FPGA Characteristics

© David C. Wyland 2006

fpga housekeeping cpu
FPGA: Housekeeping CPU
  • Need a CPU function for housekeeping, supervisor
  • Plenty of resources in FPGA for a CPU
  • Proposal: 32-bit CPU running Forth => Forth machine
    • 32-bits to address at least 1Meg of memory
  • Added benefit: Use primitives for FPGA custom functions
    • Launch & sequence keypoint processing steps
    • Control complex I/O, etc.

© David C. Wyland 2006

forth machine in fpga
Forth Machine in FPGA
  • One 512x36 block RAM holds data and return stacks
    • 512 words => 8 sets of stacks for multitasking
  • Use internal block RAM and external SRAM
    • Use 2+ block RAMs = 4 Kbytes for fast primitives
    • Have >200 Kbytes of external SRAM available for code, variables
  • Boot load from/save to 32 Kbyte serial EEPROM
  • Ref: Stack Computers by Philip Koopman

© David C. Wyland 2006

forth machine support idea
Forth Machine Support Idea
  • Use browser in a PC/Laptop for an IDE
    • Provides a clean way for visualizing Forth
    • Small TCP/IP stack in FPGA to communicate with Net, browser
  • Use web page and HTML links for word display
    • Show colon defs as a list of links to the words in the def
    • Click on a link, and nest into sub word
    • Primitives show the Forth machine instructions

© David C. Wyland 2006

forth some controversial ideas
Forth: Some Controversial Ideas
  • Use Data Stack for parameters, not calculation
    • Data stack ops are arcane: RPN calculators have lost the war
    • Use local variables + a “frame pointer” FP = SP saved at entry
    • Input parameters have fixed locations relative to FP
    • Have ops that reference stack values rel to FP + local vars
  • Use fixed size return stack “frames” for local variables
    • Push 8 words instead of 1
    • 1 word for return addrs, 1 for loop count, 6 for local variables

© David C. Wyland 2006

sift on the r1a1
SIFT on the R1A1

David Wyland

Reasonable Machines

© David C. Wyland 2006

the r1a1 project
The R1A1 Project
  • A Prototype to Test a Robotic Architecture
    • Animal Training as a Robot Design Model
  • Platform: Zagros base, Mini-itx CPU, Lynxmotion arm, webcam
  • Test Task: Go get a named object and bring it back
  • David Wyland
  • Reasonable Machines

© David C. Wyland 2006

slide18
R1A1

© David C. Wyland 2006

r1a1 block diagram
R1A1 Block Diagram
  • David Wyland
  • Reasonable Machines

© David C. Wyland 2006

r1a1 status
R1A1 Status
  • SIFT algorithm running on a PC
    • Demo at HBRC meeting
  • SIFT Gaussian filter running in FPGA
  • R1A1 elements to be done
    • Platform control: drive motors, arm servos, camera control
    • Dialog mgr (alicebot), Task mgr (RAP), Behavior mgr

© David C. Wyland 2006

robot design references
Robot Design References
  • Wyland, David, “Autonomy Without Independence: Animal Training as a Robot Design Model,” 2nd Workshop on Radical Agent Concepts (WRAC), NASA Goddard Spaceflight Center, September 2005. To be published by Springer-Verlag
  • Wyland, David, “Reasonable Machines: Analogical Reasoning in Autonomous Agent Design,” 1st Workshop on Radical Agent Concepts, NASA Goddard Spaceflight Center, January 2002. Springer-Verlag

© David C. Wyland 2006

sift implications for robotics
SIFT: Implications for Robotics
  • Robots are currently blind and deaf
    • Try walking around with your eyes closed + earplugs
  • With SIFT, robots can move around and pick up objects
    • Work in changing environments
  • With SIFT, we can make robots just like in the movies
    • Walk, talk and do things for us

© David C. Wyland 2006

sift implications for hbrc
SIFT: Implications for HBRC
  • With a SIFT Camera, We can make robots that:
    • Walk around without bumping into things
    • Can go to named objects and pick them up
  • With a SIFT Camera, We Can Make Real Robots
    • Just like In the Movies (JLIM™)
  • With SIFT, we can make robots just like in the movies

© David C. Wyland 2006

that s all folks
That’s All, Folks

David Wyland

Reasonable Machines

© David C. Wyland 2006