1 / 42

Stereo Computation using Iterative Graph-Cuts

Stereo Computation using Iterative Graph-Cuts. Vision Course - Weizmann Institute. The “Binocular” Stereo problem. “Right” Camera. “Left” Camera. Two views of the same scene from a slightly different point of view. The stereo problem.

morey
Download Presentation

Stereo Computation using Iterative Graph-Cuts

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Stereo ComputationusingIterative Graph-Cuts Vision Course - Weizmann Institute

  2. The “Binocular” Stereo problem “Right” Camera “Left” Camera Two views of the same scene from a slightly different point of view

  3. The stereo problem • Both images are very similar (like images that you see with your two eyes) • • Most of the pixels in the left image are present in the right image (except for few occlusions)

  4. Rectification • Image Reprojection • reproject image planes onto common plane parallel to baseline

  5. The stereo problem After rectification: all correspondences are along the same horizontal scan lines (pixels in one image simply shift horizontally in the other image)

  6. The relation between depth and disparities • Origin at midpoint between camera centers • Axes parallel to those of the two (rectified) cameras Depth and Disparity are inverse proportional

  7. The stereo problem • The horizontal shifts between the images are sometimes called: “disparities” • The Disparities are related to depth: Closer objects have larger disparities

  8. The stereo problem: compute the disparity map between two images

  9. Traditional Approaches • Matching small windows around each pixel • Each window is matched independently Modern approaches • Finding coherent correspondences for all pixels - “Graph cuts” - “Belief Propagation”

  10. Window-Based Approach • Compute a cost for each location • Location with the lowest cost wins

  11. General Problem : Ambiguity Left Right scanline

  12. Window-Based Approach Small Window Large Window blurred boundaries noisy in low texture areas

  13. Results with best window size(still not good enough) Window-based matching (best window size) Ground truth

  14. Graph Cuts Graph cuts Ground truth

  15. Max flow problem: Each edge is a “pipe” Find the largest flow F of “water” that can be sent from the “source” to the “sink” along the pipes Edge weights give the pipe’s capacity flow F “source” “sink” T S A graph with two terminals Maximum flow problem

  16. Min cut problem: Find the cheapest way to cut the edges so that the “source” is completely separated from the “sink” Edge weights now represent cutting “costs” a cut C “source” “sink” T S A graph with two terminals Minimum cut problem

  17. Max Flow = Min Cut: Maximum flow saturates the edges along the minimum cut. Ford and Fulkerson, 1962 Problem reduction! Ford and Fulkerson gave first polynomial time algorithm for globally optimal solution “source” “sink” T S A graph with two terminals Max flow/Min cut theorem

  18. Min-Cut: Important Rule No subset of the cut can also be a cut This is not a minimal cut

  19. Energy Minimization Using Iterative Graph cuts Fast Approximate Energy Minimization via Graph Cuts Yuri Boykov, Olga Veksler and Ramin ZabihPami 2001 More papers, code: http://www.cs.cornell.edu/~rdz/graphcuts.html

  20. To do better we need a better model of images • We can make reasonable assumptions about the surfaces in the world • Usually assume that the surfaces are smooth • Can pose the problem of finding the corresponding points as an energy (or cost) minimization: f - assignment neighboring pixels have similar disparities how well the pixels match up for different disparities

  21. To do better we need a better model of images • We can make reasonable assumptions about the surfaces in the world • Usually assume that the surfaces are smooth • Can pose the problem of finding the corresponding points as an energy (or cost) minimization: f - assignment p,q - pixels Data term is calculated for each pixels Smoothness is calculated on neighbor pixels

  22. Example for Smoothness terms • Quadratic • L1 • Truncated L1 • Potts model

  23. Constructing a Graph to Solve the Stereo Problem

  24. Constructing a Graph to Solve the Stereo Problem

  25. Constructing a Graph to Solve the Stereo Problem The labels of each pixel are the possible disparity values

  26. Constructing a Graph to Solve the Stereo Problem The labels of each pixel are the possible disparity values

  27. Smoothness term Relation between the Energy and the Graph labeling problem {fp=10} Data term 10 1 {fq=2} q p

  28. Smoothness term Relation between the Energy and the Graph labeling problem Dp(10) Data term 10 V(p,q)(1, 10) 1 q p

  29. Iterative graph-cuts • Use an iterative scheme to find a “good” local optimum of the energy function. • In each iteration: convert the original multi-label problem to a binary one, and solve it by finding a minimal graph-cut (max-flow). • The most popular scheme is the expansion move. • -expansion: set the label of each pixel to be either or the current label.

  30. Types of Moves A Single Pixel Move Problem: A lot of local minima

  31. Types of Moves Expansion Move Any pixel can change its label to alpha

  32. Types of Moves Expansion Move Claim (without proof): The difference between the optimal solution and the solution from the iterative expansion moves is bounded

  33. Energy Minimization Algorithm • Start with arbitrary labeling f • Set success = 0 • For each label • Find • If set and success =1 • If success =1 goto (2) • Return f

  34. Conditions on the Smoothness for using expansion moves: In other words: V should be a metric Note : The Quadratic smoothness is not a metric

  35. For each pair of vertices such that we add a ‘dummy’ vertex (together with the respective edges as shown in the table).

  36. The Relation between the cut and the Energy • Given a cut C, we define a labeling fc by: • The cost of a cut C is |C| = E(fC) (plus a constant) If the cut C separates p and If the cut C separates p and

  37. The Relation between the cut and the Energy The case

  38. The Relation between the cut and the Energy The case

  39. Conditions on the Smoothness for using expansion moves: In other words: V should be a metric

  40. Smoothness term Again, we can describe the problem using a graph Data term {fp=5} means – p belongs to the segment ’5’. V(p,q)(1, 10)– The penalty for assigning p and q to different segments (1 and 10 respectively). D(p)(5)– The penalty for assigning p to segment 5.

  41. The Data & Smoothness terms The penalty for assigning p and q to different segments should be high if the colors of pixels p and q are similar. For example: As a data term, we have only one constraint: That the user scribbles will be assigned with the correct label: For pixels inside user scribbles for the rest ofthe pixels

  42. Graph Cuts can be used for problems other than Stereo ! (segmentation, noise removal, image stitching, etc’).

More Related