Loading in 5 sec....

Stereo Matching & Energy MinimizationPowerPoint Presentation

Stereo Matching & Energy Minimization

- 78 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about ' Stereo Matching & Energy Minimization' - dana-sutton

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

### Stereo Matching &Energy Minimization

Vision for GraphicsCSE 590SS, Winter 2001Richard Szeliski

Stereo Matching

- What are some possible algorithms?
- match “features” and interpolate
- match edges and interpolate
- match all pixels with windows (coarse-fine)
- use optimization:
- iterative updating
- energy minimization (regularization, stochastic)
- dynamic programming
- graph algorithms

Vision for Graphics

Feature-based stereo

- Match “corner” (interest) points
- Interpolate complete solution

Vision for Graphics

Data interpolation

- Given a sparse set of 3D points, how do we interpolate to a full 3D surface?
- Scattered data interpolation [Nielson93]
- triangulate
- put onto a grid and fill (use pyramid?)
- place a kernel function over each data point
- minimize an energy function

Vision for Graphics

Relaxation

- Iteratively improve a solution by locally minimizing the energy: relax to solution
- Earliest application: WWII numerical simulations

zx,y

dx+1,y

dx,y

dx+1,y

Vision for Graphics

Relaxation

- Try solving this problem yourself:
- Make up a bunch of zx,y
- Guess best values for dx,y

Vision for Graphics

Relaxation

- How can we get the best solution?
- Differentiate energy function, set to 0

Vision for Graphics

Discrete optimization space

- What if you have discrete (e.g., binary) values?

0

0

0

0

0

1

1

1

1

1

Vision for Graphics

Dynamic programming

- Evaluate best cumulative cost at each pixel

0

0

0

0

0

1

1

1

1

1

Vision for Graphics

Dynamic programming

- Can we apply this trick in 2D as well?

dx-1,y

dx,y

dx-1,y-1

dx,y-1

No: dx,y-1 anddx-1,y may depend on different values of dx-1,y-1

Vision for Graphics

Neighborhood size (review)

- Smaller neighborhood: more details
- Larger neighborhood: fewer isolated mistakes
- w = 3 w = 20

Vision for Graphics

Plane sweep stereo

- Re-order (pixel / disparity) evaluation loopsfor every pixel, for every disparity for every disparity for every pixel compute cost compute cost

Vision for Graphics

Stereo matching framework

- For every disparity, compute raw matching costsWhy use a robust function?
- occlusions, other outliers

- Can also use alternative match criteria

Vision for Graphics

Stereo matching framework

- Aggregate costs spatially
- Here, we are using a box filter(efficient moving averageimplementation)
- Can also use weighted average,[non-linear] diffusion…

Vision for Graphics

Stereo matching framework

- Choose winning disparity at each pixel
- Can interpolate to sub-pixel accuracy

Vision for Graphics

Linear diffusion

- Average energy with neighbors + starting value
- window diffusion

Vision for Graphics

Graph cuts

- a-b swap
- expansion
modify smoothness penalty based on edges

compute best possible match within integer disparity

Vision for Graphics

Bayesian inference

- Formulate as statistical inference problem
- Prior model pP(d)
- Measurement model pM(IL, IR| d)
- Posterior model
- pM(d | IL, IR) pP(d) pM(IL, IR| d)
- Maximum a Posteriori (MAP estimate):
- maximize pM(d | IL, IR)

Vision for Graphics

Markov Random Field

- Probability distribution on disparity field d(x,y)
- Enforces smoothness or coherence on field

Vision for Graphics

Measurement model

- Likelihood of intensity correspondence
- Corresponds to Gaussian noise for quadratic r

Vision for Graphics

MAP estimate

- Maximize posterior likelihood
- Equivalent to regularization (energy minimization with smoothness constraints)

Vision for Graphics

Why Bayesian estimation?

- Principled way of determining cost function
- Explicit model of noise and prior knowledge
- Admits a wider variety of optimization algorithms:
- gradient descent (local minimization)
- stochastic optimization (Gibbs Sampler)
- mean-field optimization
- graph theoretic (actually deterministic) [Zabih]

Vision for Graphics

Mean-field interpretation

- Bayesian non-linear diffusion rule:
- update your probability distribution assuming your neighbors’ distributions are independent (valid for Markov chain)

- Equivalent to finding best factored approximation
- P(d|IL,IR) ~ Q(d) = iQi(di)

Vision for Graphics

Mean-field interpretation

- log MAP estimate
- -log P(d|IL,IR) = ijEij(di,dj) + iEi(di)
- = ijsiAijsi + ibisi
- Kullback-Leibler divergence
- DKL =H(Q) - dQ(d) log P(d)
- = ik qik log qik + ijsiAijsi + ibisi

Vision for Graphics

Mean-field interpretation

- minimize K-L divergence with
- k qik = 1
- update rule:
- qik exp[ - ( jaijkqj +bik )]
- = exp[- ( jlEij(di=k,dj=l)p(dj=l) + Ei(di=k))]

Vision for Graphics

Stereo with Non-Linear Diffusion

- Advantages:
- works very well in non-occluded regions

- Disadvantages:
- restricted to two images (not)
- gets confused in occluded regions
- can’t handle mixed pixels

Vision for Graphics

Summary

- Applications
- Image rectification
- Matching criteria
- Local algorithms (aggregation)
- area-based; iterative updating

- Optimization algorithms:
- energy (cost) formulation
- Markov Random Fields
- mean-field; dynamic programming;
- stochastic; graph algorithms

Vision for Graphics

More stereo…(next 2 lectures)

- Multi-image stereo
- Volumetric techniques
- Graph cuts
- Transparency
- Surfaces and level sets

Vision for Graphics

Bibliography

- See the references in the readings…
- Y. Boykov, O. Veksler, and Ramin Zabih, Fast Approximate Energy Minimization via Graph Cuts, Unpublished manuscript, 2000.
- A.F. Bobick and S.S. Intille. Large occlusion stereo. International Journal of Computer Vision, 33(3), September 1999. pp. 181-200
- D. Scharstein and R. Szeliski. Stereo matching with nonlinear diffusion. International Journal of Computer Vision, 28(2):155-174, July 1998
- R. Szeliski. Stereo algorithms and representations for image-based rendering. In British Machine Vision Conference (BMVC'99), volume 2, pages 314-328, Nottingham, England, September 1999.
- R. Szeliski and R. Zabih. An experimental comparison of stereo algorithms. In International Workshop on Vision Algorithms, pages 1-19, Kerkyra, Greece, September 1999.
- G. M. Nielson, Scattered Data Modeling, IEEE Computer Graphics and Applications, 13(1), January 1993, pp. 60-70.

Vision for Graphics

Download Presentation

Connecting to Server..