1 / 24

Summary: A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms

Summary: A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms. Matthew Wilhelm CS5331 Mobile Robotics. Goal / Motivation. Provide means of quantitatively gauge progress in the field of Stereo Correspondence as well as judge the value of new approaches

Download Presentation

Summary: A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Summary: A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms Matthew Wilhelm CS5331 Mobile Robotics

  2. Goal / Motivation • Provide means of quantitatively gauge progress in the field of Stereo Correspondence as well as judge the value of new approaches • Novel publications will have to improve in some way on the performance of existing algorithms • Provide an update on the state of the art of the field

  3. Background / Theory • All vision algorithms make assumptions about physical world and camera • Stereo Algorithms commonly make the following assumptions • Lambertian surfaces – appearance does not vary with viewpoint • Piecewise-smooth surfaces • Camera Calibration and geometry

  4. Disparity • ~ difference between location of matching pixels • ≈ inverse depth • Various computation methods • Displayed as a disparity space image, close items will be brighter and far away items will be darker

  5. Taxonomy • A classification system for items based on their relationship to one another • Allows dissection and comparison of individual algorithm components and design decisions • Matching Cost Computation • Cost Aggregation • Disparity Computation / Optimization • Disparity Refinement • Existing algorithms are built of various implementations of above classifications

  6. Matching Cost Computation • Form initial Disparity Space Image • Many Methods including: • Squared Intensity Differences • Absolute Intensity Differences • Truncated Quadratics • Contaminated Gaussians • Normalized Cross-Correlation • Binary Features

  7. Cost Aggregation • Group similar costs in disparity space image in order to form objects • Again Many Different Methods Including: • Square Windows • Gaussian Convolution • Shiftable Windows • Adaptable Size Windows

  8. Disparity Computation / Optimization • Local Methods – majority of work done in previous two steps • For optimization simply choose at each pixel the disparity with the minimum cost value. • Uniqueness is only enforced on one image. • Global Methods – majority of work done in this stage • Energy Minimization – continuation, simulated annealing, highest confidence first, and mean field annealing • Max-Flow and Graph-Cut for special cases • Dynamic Programming – compute disparity for pair wise matching costs, using adjusting parameters • Cooperative Algorithms – models human stereo vision

  9. Disparity Refinement • Sub pixel disparity estimates used when rendering images for more appealing view results • Clean up mismatches via various methods • Not usually done for fast implementation such as robot navigation or tracking

  10. Implementation • Closely tied to Taxonomy given above • Author developed modular and portable C++ implementation of several stereo algorithms • Post processing steps to improve results not implemented, in order to compare methods directly. • Easily extendable to include other algorithms

  11. Implementation Details • Matching Cost Computation • Squared or absolute difference in color • Sub-pixel interpolation • Aggregation • Box Filter: separable moving average filter • Binomial Filter: separable finite impulse response filters • Optimization • Winner-take-all, dynamic programming, scanline optimization, simulated annealing, and graph cut • Refinement • Three aggregated matching cost values around the winning disparity are examined to compute the sub-pixel disparity estimate

  12. Evaluation • Allows for quantitative evaluation of stereo algorithms • Provides test bed for new and existing algorithms along with test data and results on the Web at http://vision.middlebury.edu/stereo/ • Allows for testing of individual components as divided in taxonomy

  13. Quality Metrics • RMS error – root-mean-squared value of difference between the computed disparity map and the ground truth map • Percentage of bad matching pixels – disparity error tolerance • Computed over whole image as well as three areas which usually cause problems: • Textureless regions – average intensity gradient to low • Occluded regions – mapped disparity lands at location covered by closer object • Depth discontinuity regions – neighboring disparities differ by to much

  14. Experiments • Authors perform several experiments to compare various algorithm components, again as divided in the taxonomy • Focus on common problem areas for stereo algorithms

  15. Experiments / Results • Matching Costs • Experiment 1: ran many tests with different matching cost truncation values found good results are 5-20 • Experiment 2: ran same test as above, but used a 9x9 min filter before truncation and found that no truncation performed best • Experiment 3: tested effects of matching cost and truncation on global algorithms, found that some truncation helped, and suggested use of SNR based parameter setting

  16. Experiments / Results • Aggregation • Experiment 4: Analyze affects of various aggregation techniques on local methods • Large amounts of aggregation are necessary in textureless regions • Shiftable windows perform best

  17. Experiments / Results • Disparity Computation /Optimization • Experiment 5: analyze smoothness parameter • Found that the optimal smoothness parameter varies greatly for each image pair • Future work includes parameter calculation techniques • Experiment 6: Focus on graph-cut optimization • While Birchfield-Tomasi’s method and gradient based smoothness cost improve performance of graph-cut algorithms, • Choosing the right parameters for threshold and penalty is difficult and image specific

  18. Experiments / Results • Sub-Pixel Estimations • Experiment 7: refine disparity maps via sub-pixel interpolation • As expected an unrefined DSI contains staircase error, where refined DSI is considerably better • Again, this step is often skipped in fast implementations

  19. Conclusion • The author provides a comparison of 20 stereo algorithms all of which are available in detail on website • Found that most algorithms perform about the same in so-called easy area’s and the differences arise in known problematic areas • One evaluation of algorithms that I thought would have been helpful was runtime comparisons, however the author was not concerned with this

  20. Questions? • Can you clarify what is being reference in Figure 1 (f) regarding the disparity levels as a slice? • A slice simply means that the DSI is 3D and the are keeping on of the 3 variables constant to produce a 2D image • Can you find references to using illumination along side the stereo depth analysis to further define the depth of objects? • I have searched some and did not see any papers however this does not mean that it is impossible. • Probably would be very helpful to have a illumination estimate prior to stereo evaluation • And of course, can you simplify the differences between each algorithm? • I think I have done a brief simplification, to go into more detail I would have to read each of the 132 referenced papers

  21. Questions?? • How were the stereo algorithms chosen in the paper? • The paper focuses on Dense Two-Frame Stereo Correspondence Algorithms • Common algorithms which needed to be compared were chosen to be implemented however the framework allows novel algorithms to be implemented as well • What is a stereo algorithm? • A stereo algorithm utilizes images from two cameras, similar to human vision (two eyes)

  22. Questions??? • In page 2, section 2.2, it was indicated that an unvalued disparity map is produced as output. What is “unvalued disparity map”? • It sais univalued • I think this means that there is a single value for the disparity at each pixel • On page 11 under the evaluation section they discuss that they use three different regions to check the algorithm over (texture less, occluded, and depth) how did they come up with these? • These are common problem areas for several different stereo algorithms

  23. Questions???? • Why did they down sample the images for testing (page 13)? • To normalize the motion of background objects to a few pixels per frame, to allow better results when matching and truly compare various algorithms quality • Why do they only evaluate, bad_pixels_nonocc, bad_pixels_textureless, and bad_pixels_discont ? • The also evaluate the whole image, but the do these areas separate as well to get an idea of how different algorithms perform in these known problem areas

  24. Additional Resources • References throughout the paper provide resources for various algorithms • Hartley and Zisserman: Multiple View Geometry in Computer Vision. • Middlebury website, an excellent source of papers and code related to stereo vision algorithms. • Sebastian Thrun, Wolfram Burgard and Dieter Fox: Probabilistic Robotics, MIT Press, 2005.

More Related