Sift fpgas and forth
Download
1 / 24

- PowerPoint PPT Presentation


  • 759 Views
  • Uploaded on

SIFT, FPGAs and Forth. Dave Wyland Reasonable Machines. Overview. SIFT algorithm for Object Recognition SIFT: FPGA vs DSP Forth Machine in FPGA SIFT on the R1A1 - The R1A1 Robot Project SIFT and R1A1 Status Implications for Robotics and HBRC. SIFT – What Is It?.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about '' - Ava


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Sift fpgas and forth
SIFT, FPGAs and Forth

Dave Wyland Reasonable Machines

© David C. Wyland 2006


Overview
Overview

  • SIFT algorithm for Object Recognition

  • SIFT: FPGA vs DSP

  • Forth Machine in FPGA

  • SIFT on the R1A1 - The R1A1 Robot Project

  • SIFT and R1A1 Status

  • Implications for Robotics and HBRC

© David C. Wyland 2006


Sift what is it
SIFT – What Is It?

  • An algorithm for object recognition

    • Developed by David Lowe of University of BC

  • Identifies objects using ordinary cameras (TV, webcam)

    • At any distance

    • For any orientation (pose)

    • For (almost) any lighting

    • With simple image distortions

© David C. Wyland 2006


Sift what good is it
SIFT – What Good Is It?

  • Lets us recognize objects automatically

  • Gives bearing and approximate range with 1 camera

  • Gives pose of object: normal, tilted, etc.

  • Lets our robots see what we see

  • Lets us build robots to go to named objects, pick them up

© David C. Wyland 2006


Sift how does it work
SIFT – How does it work?

  • Outline

    • Find keypoints using Difference of Gaussians

      • 2D Gaussian filters of image

    • Look at 16x16 patch around keypoint

    • Generate a 128 dimension feature vector per point

    • Record new or search for match among recorded

© David C. Wyland 2006


Sift references
SIFT – References

  • Lowe, David, “Distinctive Image Features from Scale Invariant Keypoints,” International Journal of Computer Vision, 2004.

  • Google for SIFT, Scale Invariant Feature Transform, David Lowe

© David C. Wyland 2006


Sift how much computation
SIFT – How Much Computation?

  • Steps in Algorithm

    • Keypoint finding: 49.6 x 10^6 pixel operations

    • Keypoint descriptor generation: 1.4 x 10^6 pix ops

    • Keypoint search: 100 ref objects: 32 x 10^6 compares

© David C. Wyland 2006


Sift fpga vs dsp
SIFT – FPGA vs DSP

  • 600 MHz ADI Blackfin DSP vs Xilinx Spartan 3 FPGA

    • Time for DoG on 640x480 Pyramid, 3 planes/level

    • Blackfin: 12 frames/second at 600 MIPs

    • FPGA: 41 frames/second at 50 MHz

© David C. Wyland 2006


Fpga characteristics
FPGA Characteristics

© David C. Wyland 2006




Fpga housekeeping cpu
FPGA: Housekeeping CPU

  • Need a CPU function for housekeeping, supervisor

  • Plenty of resources in FPGA for a CPU

  • Proposal: 32-bit CPU running Forth => Forth machine

    • 32-bits to address at least 1Meg of memory

  • Added benefit: Use primitives for FPGA custom functions

    • Launch & sequence keypoint processing steps

    • Control complex I/O, etc.

© David C. Wyland 2006


Forth machine in fpga
Forth Machine in FPGA

  • One 512x36 block RAM holds data and return stacks

    • 512 words => 8 sets of stacks for multitasking

  • Use internal block RAM and external SRAM

    • Use 2+ block RAMs = 4 Kbytes for fast primitives

    • Have >200 Kbytes of external SRAM available for code, variables

  • Boot load from/save to 32 Kbyte serial EEPROM

  • Ref: Stack Computers by Philip Koopman

© David C. Wyland 2006


Forth machine support idea
Forth Machine Support Idea

  • Use browser in a PC/Laptop for an IDE

    • Provides a clean way for visualizing Forth

    • Small TCP/IP stack in FPGA to communicate with Net, browser

  • Use web page and HTML links for word display

    • Show colon defs as a list of links to the words in the def

    • Click on a link, and nest into sub word

    • Primitives show the Forth machine instructions

© David C. Wyland 2006


Forth some controversial ideas
Forth: Some Controversial Ideas

  • Use Data Stack for parameters, not calculation

    • Data stack ops are arcane: RPN calculators have lost the war

    • Use local variables + a “frame pointer” FP = SP saved at entry

    • Input parameters have fixed locations relative to FP

    • Have ops that reference stack values rel to FP + local vars

  • Use fixed size return stack “frames” for local variables

    • Push 8 words instead of 1

    • 1 word for return addrs, 1 for loop count, 6 for local variables

© David C. Wyland 2006


Sift on the r1a1
SIFT on the R1A1

David Wyland

Reasonable Machines

© David C. Wyland 2006


The r1a1 project
The R1A1 Project

  • A Prototype to Test a Robotic Architecture

    • Animal Training as a Robot Design Model

  • Platform: Zagros base, Mini-itx CPU, Lynxmotion arm, webcam

  • Test Task: Go get a named object and bring it back

  • David Wyland

  • Reasonable Machines

© David C. Wyland 2006


R1A1

© David C. Wyland 2006


R1a1 block diagram
R1A1 Block Diagram

  • David Wyland

  • Reasonable Machines

© David C. Wyland 2006


R1a1 status
R1A1 Status

  • SIFT algorithm running on a PC

    • Demo at HBRC meeting

  • SIFT Gaussian filter running in FPGA

  • R1A1 elements to be done

    • Platform control: drive motors, arm servos, camera control

    • Dialog mgr (alicebot), Task mgr (RAP), Behavior mgr

© David C. Wyland 2006


Robot design references
Robot Design References

  • Wyland, David, “Autonomy Without Independence: Animal Training as a Robot Design Model,” 2nd Workshop on Radical Agent Concepts (WRAC), NASA Goddard Spaceflight Center, September 2005. To be published by Springer-Verlag

  • Wyland, David, “Reasonable Machines: Analogical Reasoning in Autonomous Agent Design,” 1st Workshop on Radical Agent Concepts, NASA Goddard Spaceflight Center, January 2002. Springer-Verlag

© David C. Wyland 2006


Sift implications for robotics
SIFT: Implications for Robotics

  • Robots are currently blind and deaf

    • Try walking around with your eyes closed + earplugs

  • With SIFT, robots can move around and pick up objects

    • Work in changing environments

  • With SIFT, we can make robots just like in the movies

    • Walk, talk and do things for us

© David C. Wyland 2006


Sift implications for hbrc
SIFT: Implications for HBRC

  • With a SIFT Camera, We can make robots that:

    • Walk around without bumping into things

    • Can go to named objects and pick them up

  • With a SIFT Camera, We Can Make Real Robots

    • Just like In the Movies (JLIM™)

  • With SIFT, we can make robots just like in the movies

© David C. Wyland 2006


That s all folks
That’s All, Folks

David Wyland

Reasonable Machines

© David C. Wyland 2006


ad