1 / 43

Adaptation from:

Adaptation from:. Prof. James M. Rehg, G.Tech. Public Library, Stereoscopic Looking Room, Chicago, by Phillips, 1923. Teesta suspension bridge-Darjeeling, India. Mark Twain at Pool Table", no date, UCR Museum of Photography.

Download Presentation

Adaptation from:

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Adaptation from: Prof. James M. Rehg, G.Tech Computer Vision : CISC 4/689

  2. Public Library, Stereoscopic Looking Room, Chicago, by Phillips, 1923 Computer Vision : CISC 4/689

  3. Teesta suspension bridge-Darjeeling, India Computer Vision : CISC 4/689

  4. Mark Twain at Pool Table", no date, UCR Museum of Photography Computer Vision : CISC 4/689

  5. Woman getting eye exam during immigration procedure at Ellis Island, c. 1905 - 1920, UCR Museum of Phography Computer Vision : CISC 4/689

  6. Stereo results • Data from University of Tsukuba Scene Ground truth (Seitz) Computer Vision : CISC 4/689

  7. Results with window correlation Window-based matching (best window size) Ground truth (Seitz) Computer Vision : CISC 4/689

  8. Results with better method • State of the art method • Boykov et al., Fast Approximate Energy Minimization via Graph Cuts, • International Conference on Computer Vision, September 1999. Ground truth Computer Vision : CISC 4/689 (Seitz)

  9. Stereo Constraint • Color constancy • The color of any world points remains constant from image to image • This assumption is true under Lambertian Model • In practice, given photometric camera calibration and typical scenes, color constancy holds well enough for most stereo algorithms. Computer Vision : CISC 4/689

  10. Stereo Constraint • Epipolar geometry • The epipolar geometry is the fundamental constraint in stereo. • Rectification aligns epipolar lines with scanlines Epipolar plane Epipolar line for p Computer Vision : CISC 4/689 Epipolar line for p’

  11. Stereo Constraint • Uniqueness and Continuity • Proposed by Marr&Poggio. • Each item from each image may be assigned at most one disparity value,” and the “disparity” varies smoothly almost everywhere. Computer Vision : CISC 4/689

  12. Problems with Window-based matching • Disparity within the window must be constant. • Bias the results towards frontal-parallel surfaces. • Blur across depth discontinuities. • Perform poorly in textureless regions. • Erroneous results in occluded regions Computer Vision : CISC 4/689

  13. Cooperative Stereo Algorithm • Based on two basic assumption by Marr and Poggio: • Uniqueness: at most a single unique match exists for each pixel. • Continuous: disparity values are generally continuous, i.e., smooth within a local neighborhood. Computer Vision : CISC 4/689

  14. Disparity Space Image (DSI) • The 3D disparity space has dimensions row r column c and disparity d. Each element (r, c, d) of the disparity space projects to the pixel (r, c) in the left image and to the (r, c + d) in the right image • DSI represents the confidence or likelihood of a particular match. Computer Vision : CISC 4/689

  15. Illustration of DSI (r, c) slices for different d (c, d) slice for r = 151 Computer Vision : CISC 4/689

  16. Definition Match value assigned to element (r, c, d) at iteration n Initial values computed from SSD or NCC Inhibition area for element (r, c, d) Local support area for element (r, c, d) Computer Vision : CISC 4/689

  17. Illustration of Inhibitory and Support Regions If d is disparity for (r,c), d -1 is for (r,c-1) And d+1 for (r,c+1), etc. Vertical inhibition is self explanatory. Diagonal basically means, r,c+1in left image cannot map to r,c+d in right image, i.e, it cannot have disparity of d-1. Similarly, r,c-1 in left image cannot map to r,c+d so it cannot have disparity of d+1. This satisfies uniqueness constraint. Computer Vision : CISC 4/689

  18. Iterative Updating DSI 1 2 • Questions to ponder: • What is typical alpha? • Do we include inhibition • region in numerator of Rn? 3 4 Computer Vision : CISC 4/689

  19. Explicit Detection of Occlusion Identify occlusions by examining the magnitude of the converged values in conjunction with the uniqueness constrain Computer Vision : CISC 4/689

  20. Summary of Cooperative Stereo • Prepare a 3D array, (r, c, d): (r, c) for each pixel in the reference image and d for the range of disparity. • Set initial match values using a function of image intensities, such as normalized correlation or SSD. • Iteratively update match values using (4) until the match values converge. • For each pixel (r, c), find the element (r, c, d) with the maximum match value. • If the maximum match value is higher than a threshold, output the disparity d, otherwise, declare a occlusion. Computer Vision : CISC 4/689

  21. Ordering Constraint • If an object a is left on an object b in the left image then object a will also appear to the left of object b in the right image Ordering constraint… …and its failure Computer Vision : CISC 4/689

  22. … Stereo Correspondences Left scanline Right scanline Match intensities sequentially between two scanlines Computer Vision : CISC 4/689

  23. … Match Match Match Left occlusion Right occlusion Stereo Correspondences Left scanline Right scanline Computer Vision : CISC 4/689

  24. Left Occluded Pixels Search Over Correspondences Three cases: • Sequential – cost of match • Left occluded – cost of no match • Right occluded – cost of no match Left scanline Right scanline Right occluded Pixels Computer Vision : CISC 4/689

  25. Left Occluded Pixels Standard 3-move Dynamic Programming for Stereo Dynamic programming yields the optimal path through grid. This is the best set of matches that satisfy the ordering constraint Left scanline Start Right scanline Right occluded Pixels Computer Vision : CISC 4/689 End

  26. Dynamic Programming • Efficient algorithm for solving sequential decision (optimal path) problems. 1 1 1 1 … 2 2 2 2 3 3 3 3 How many paths through this trellis? Computer Vision : CISC 4/689

  27. Dynamic Programming 1 1 1 States: 2 2 2 3 3 3 Suppose cost can be decomposed into stages: Computer Vision : CISC 4/689

  28. Dynamic Programming 1 1 1 2 2 2 3 3 3 Principle of Optimality for an n-stage assignment problem Computer Vision : CISC 4/689

  29. Dynamic Programming 1 1 1 2 2 2 3 3 3 Computer Vision : CISC 4/689

  30. Occluded Pixels Stereo Matching with Dynamic Programming Scan across grid computing optimal cost for each node given its upper-left neighbors.Backtrack from the terminal to get the optimal path. Left scanline Dis-occluded Pixels Right scanline Terminal Computer Vision : CISC 4/689

  31. Occluded Pixels Stereo Matching with Dynamic Programming Scan across grid computing optimal cost for each node given its upper-left neighbors.Backtrack from the terminal to get the optimal path. Left scanline Dis-occluded Pixels Right scanline Terminal Computer Vision : CISC 4/689

  32. Occluded Pixels Stereo Matching with Dynamic Programming Scan across grid computing optimal cost for each node given its upper-left neighbors.Backtrack from the terminal to get the optimal path. Left scanline Dis-occluded Pixels Right scanline Terminal Computer Vision : CISC 4/689

  33. Occluded Pixels Stereo Matching with Dynamic Programming Scan across grid computing optimal cost for each node given its upper-left neighbors.Backtrack from the terminal to get the optimal path. Left scanline Dis-occluded Pixels Right scanline Terminal Computer Vision : CISC 4/689

  34. Occluded Pixels Stereo Matching with Dynamic Programming Scan across grid computing optimal cost for each node given its upper-left neighbors.Backtrack from the terminal to get the optimal path. Left scanline Dis-occluded Pixels Right scanline Terminal Computer Vision : CISC 4/689

  35. Occluded Pixels Stereo Matching with Dynamic Programming Scan across grid computing optimal cost for each node given its upper-left neighbors.Backtrack from the terminal to get the optimal path. Left scanline Dis-occluded Pixels Right scanline Terminal Computer Vision : CISC 4/689

  36. Occluded Pixels Stereo Matching with Dynamic Programming Scan across grid computing optimal cost for each node given its upper-left neighbors.Backtrack from the terminal to get the optimal path. For each (i,j), look for what is Optimal to get to: would it be From (i-1,j-1), or (i,j-1), or (i-1,j-1)? Assign large values for left occlusion and right occlusion. Left scanline Dis-occluded Pixels Right scanline Terminal Computer Vision : CISC 4/689

  37. Stereo Matching with Dynamic Programming Pseudo-code describing how to calculate the optimal match Computer Vision : CISC 4/689

  38. Stereo Matching with Dynamic Programming Pseudo-code describing how to reconstruct the optimal path Computer Vision : CISC 4/689

  39. Results Local errors may be propagated along a scan-line and no inter scan-line consistency is enforced. Computer Vision : CISC 4/689

  40. Assumption Behind Segmentation-based Stereo • Depth discontinuity tend to correlate well with color edges • Disparity variation within a segment is small • Approximating the scene with piece-wise planar surfaces Computer Vision : CISC 4/689

  41. Segmentation-based stereo • Plane equation is fitted in each segment based on initial disparity estimation obtained SSD or Correlation • Global matching criteria: if a depth map is good, warping the reference image to the other view according to this depth will render an image that matches the real view • Optimization by iterative neighborhood depth hypothesizing Computer Vision : CISC 4/689

  42. Result Computer Vision : CISC 4/689

  43. Another Result Computer Vision : CISC 4/689

More Related