1 / 21

Distortion Correction ECE 6276 Project Review

Distortion Correction ECE 6276 Project Review. Team 5: Basit Memon Foti Kacani Jason Haedt Jin Joo Lee Peter Karasev. Outline. Motivation Components Component Optimization Current Results Plans for Catapult C Schedule. Objective.

Download Presentation

Distortion Correction ECE 6276 Project Review

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Distortion CorrectionECE 6276Project Review Team 5: Basit Memon Foti Kacani Jason Haedt Jin Joo Lee Peter Karasev

  2. Outline • Motivation • Components • Component Optimization • Current Results • Plans for Catapult C • Schedule

  3. Objective Given a distorted image with known size and known lens distortion parameter, generate an undistorted image.

  4. Motivation – Why? • The formation of undistorted images can be described by a series of matrix multiplications • Distortion makes it very difficult to calibrate a camera to measure geometry (depth, size, orientation, etc) • Many applications in image processing and computer vision like structure estimation, image mosaicing, and ultimately vision-based control.

  5. Motivation contd.. Application: Measure motion and geometry Problem: Known geometry in the scene is warped, relationship between 3D and 2D points is nonlinear. Solution: Undo the distortion, so x2D = A * X3D

  6. Literature Review (I) K.T. Gribbon, C.T. Johnston, and D.G. Bailey, “A Real-time FPGA Implementation of a Barrel Distortion Correction Algorithm with Bilinear Interpolation” • Focus on reducing hardware complexity. • Uses LUTs to store mapping data. • No quantitative results provided. • Logic resource utilization on RC-100 is 51%

  7. Literature Review (II) • Qiang, L.; Allinson, N.M.;” FPGA Implementation of Pipelined Architecture for Optical Imaging Distortion Correction”, Signal Processing Systems Design and Implementation, 2006. SIPS '06. • Same algorithm as previous one • Implementation on a Xilinx FPGA XCS3 1000-4 uses 75% of the hardware multipliers. • Residual error of the undistorted image was 1.5% of the distorted image.

  8. Literature Review (III) • Hany Farid & Alin C. Popescu, “Blind Removal of Lens Distortion”, Journal of the Optical Society of America 2001 • For removal of distortion in absence of any calibration data. • Uses polyspectral analysis to detect higher order correlations in frequency domain which are proportional to the distortion. • Computationally intensive. • No quantitative results. • Accuracy is not comparable to those based on known distortion parameters.

  9. Components to Achieve Objective Component 1: Matlab forward distortion function • Verifies correctness of undistortion algorithm Component 2: Data ordering test bench • Order the C++ input stream from MATLAB generated data Component 3: Undistortion lookup table or lookup function • Initial prototype in MATLAB Component 4: Least squares interpolation lookup function • Initial prototype in MATLAB • Compare different techniques such as LUT vs NEAREST NEIGHBOR Component 5: Verification structure • Compare original C++ result to undisorted image in MATLAB MATLAB C++  Catapult C  MATLAB

  10. Component Optimization Questions • What size buffers do we need to compare against previous frames? •  For undistortion function can we compute them dynamically or do we need a pre-defined LUT? •  For finding fast least squares / linear system solver, compare speed cost vs. nearest neighbor and effect on output error

  11. Matlab Demo of Algorithm - Original

  12. Matlab Demo of Algorithm - Distorted

  13. Matlab Demo of Algorithm - Recovered

  14. Design Goals • Limited Buffers • 8 bits per sub-pixel (24 bits total) • Resolution (up to 640x480) • Concerned with geometry • Area, Throughput, Latency • Target a low-cost implementation that handles consumer video application pixel clocks of 165 MHz (Apprx 6 ns cycle time).

  15. Test Vectors The Line Test Original Image Distorted Image Recovered Image

  16. Plans for Catapult C Code • Use C/C++ and Algorithmic C data types to describe synthesizable hardware • Architectures (type of hardware interface (streaming buffers) ) • Constraints (Throughput, area, latency) • RTL generation and verification • Optimizations -Pipelining -Parallelism -Loop Unrolling -Scheduling -Streaming buffers/ Read & Write

  17. Project Timeline

  18. Project Risks Risk 1: Slowness of interpolation plan: the key goal of undistortion is “geometrical accuracy” which hold even if noise is injected. To mitigate this risk we have a forward mapping algorithm that is fast compared to any interpolation method but at the cost of missing pixels near the edges. Risk 2: Not enough storage space to keep a lookup table of coordinates. To mitigate this risk we could compute coordinates on the fly at the cost of math operations. Risk 3: Cannot find least squares method for FPGA. To mitigate this risk we could do nearest neighbor interpolation.

  19. Current Status • Straight lines are recovered with minimal injection of missing points or noise through our algorithm. • Add more quantitative results on the geometrical accuracy of recovered images than presented in previous results. • Generalize the coordinate mapping to make the implementation more robust to varying  distortion models and hence supporting more cameras “on the fly.”

  20. References • Yi Ma, Stefano Soatto, et al., “An Invitation to 3-D Vision” • Richard Hartley, Andrew Zisserman, “Multiple View Geometry in Computer Vision” • Edward M. Mikhail, James S. Bethel, J. Chris McGlone, “Introduction to Modern Photogrammetry”

  21. Questions? ?

More Related