1 / 20

Distortion Correction ECE 6276 Project Review

Distortion Correction ECE 6276 Project Review. Team 5: Basit Memon Foti Kacani Jason Haedt Jin Joo Lee Peter Karasev. Outline. Objective/Motivation Movement of Data C code Conversion to Catapult C Initial Results Current Issues Plans for Optimization Schedule

eithne
Download Presentation

Distortion Correction ECE 6276 Project Review

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Distortion CorrectionECE 6276Project Review Team 5: Basit Memon Foti Kacani Jason Haedt Jin Joo Lee Peter Karasev

  2. Outline • Objective/Motivation • Movement of Data • C code • Conversion to Catapult C • Initial Results • Current Issues • Plans for Optimization • Schedule • Meeting with Mike Bradley

  3. Objective Given a distorted image with known size and known lens distortion parameter, generate an undistorted image.

  4. Motivation – Why? • The formation of undistorted images can be described by a series of matrix multiplications • Distortion makes it very difficult to calibrate a camera to measure geometry (depth, size, orientation, etc) • Many applications in image processing and computer vision like structure estimation, image mosaicing, and ultimately vision-based control.

  5. Motivation contd.. Application: Measure motion and geometry Problem: Known geometry in the scene is warped, relationship between 3D and 2D points is nonlinear. Solution: Undo the distortion, so x2D = A * X3D

  6. Movement of Data Input Output Algorithm Input Buffer RAM on Chip 1 Pixel/Line Random Read LUT

  7. C Code Reference Design I/O scheme: Write to output in raster order, read input randomly (From buffer). New input put into buffer in raster order. Coordinate in buffer does not have closed form expression- approximate with power series created in matlab, store as lookup table. const double powercoeffs[256][5] = { 0.0001676486936455572,0.9984675943966409,0.004271662030922045,-0.004648466894072161,0.001731029514912795, 0.0001678607731736833,1.004564329741083,0.004318757090371574,-0.008746108062078511,0.001856610695386462,… Top-level Entry point- block of pixels in and out, reset, index of distortion level void undistort_ref( const unsigned char pixels_in[PIXELS], unsigned char pixels_out[PIXELS], bool reset, unsigned short kappa_idx ); if( (iid >= 0) && (jjd >= 0) && (iid < HEIGHT) && (jjd < WIDTH) ) pixels_out[ i_*WIDTH + j_ ] = pixels_in[ (iid*WIDTH) + jjd ]; else pixels_out[ i_*WIDTH + j_ ] = 0; Conditional Write operation (ensure in-bounds) // read next frame n = fread(pix_in, 1, PIXELS, fin); // do the work for this block undistort_ref( pix_in, pix_out, 0, kappa_idx); // write what we undistorted n = fwrite( pix_out, 1, PIXELS, fout ); Code verification- load binary input vectors, run, save, compare to matlab generated binary files

  8. Conversion to Catapult C Catapult C does not support dynamic memory allocation - use static arrays for memory - simplify C code structure to avoid pointers Any value that needs to be stored must be declared as static or could be synthesized out Top-level default instantiation assumes that input and output ports are wires, must explicitly declare memories Casting for printing is not automatically supported, must use .to_int() and .to_double() functions Had to simplify C code structure to not use as many pointers as concept is not as well defined in Catapult C Verify correctness in Visual Studio and Catapult environment

  9. Test Vector – Original Image

  10. Test Vector – Distorted Image

  11. Test Vectors – Recovered Image

  12. Initial Catapult C Synthesis (1/3)

  13. Initial Catapult C Synthesis (2/3)

  14. Initial Catapult C Synthesis (3/3)

  15. Current Problems • Top issue is Area – we can’t support large image sizes based on current algorithm implementation. • How do we achieve the ability to stream images in and then pass through our algorithm? Need to segment our algorithm and stream in portions of a larger image. • Need to allocate large arrays to RAMs, particularly our LUT.

  16. Project Timeline

  17. Plans for Optimization • Reformulate problem slightly to avoid expensive sqrt() operation (requires a new LUT) • Ensure that LUT is mapping to ROM and buffer is on-chip RAM, not I/O pins • Determine required bit-width for math operation result and use minimum • Zero-pad arrays to be a power of two (avoids extra MUX elements) • Avoid division- add lookup for radius values • Experiment with size of input blocks. rgb2yuv example seemed to indicate large blocks create low latency, high area designs. • Choose pipelining interval, loop to unroll. Unroll low level, pipeline top?

  18. Meeting with Mike Bradley • He showed us how to ensure that LUT is mapping to ROM in Catapult C GUI • Determined which for loops to unroll and which to pipeline • Discussed different types of technologies ASIC vs FPGAs

  19. Updated Catapult Synthesis -Catapult C no longer crashes for large input images -Significant improvement in area, latency, and throughput -Processes 1 pixel per clock cycle

  20. Questions? ?

More Related