1 / 29

Learning Low-Level Vision William T. Freeman Egon C. Pasztor Owen T. Carmichael

Learning Low-Level Vision William T. Freeman Egon C. Pasztor Owen T. Carmichael. Model image and scene patches as nodes in a Markov network. image patches. scene patches. image. F ( x i , y i ). Y ( x i , x j ). scene. Network joint probability. 1. Õ. Õ. =. Y. F. y. P. (. x.

leyna
Download Presentation

Learning Low-Level Vision William T. Freeman Egon C. Pasztor Owen T. Carmichael

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Learning Low-Level Vision William T. Freeman Egon C. Pasztor Owen T. Carmichael

  2. Model image and scene patches as nodes in a Markov network image patches scene patches image F(xi, yi) Y(xi, xj) scene

  3. Network joint probability 1 Õ Õ = Y F y P ( x , y ) ( x , x ) ( x , ) i j i i Z , i j i scene Scene-scene compatibility function Image-scene compatibility function image neighboring scene nodes local observations

  4. image scene Super-resolution • Image: low resolution image • Scene: high resolution image ultimate goal...

  5. Representation Zoomed low-freq. Full freq. original True high freqs Low-band input (contrast normalized, PCA fitted) (to minimize the complexity of the relationships we have to learn, we remove the lowest frequencies from the input image, and normalize the local contrast level).

  6. Training images,~100,000 image/scene patch pairs Images from two Corel database categories: “giraffes” and “urban skyline”.

  7. Gather ~100,000 patches ... ... high freqs. low freqs. Training data samples (magnified)

  8. Nearest neighbor estimate Input low freqs. Estimated high freqs. ... ... high freqs. low freqs. Training data samples (magnified)

  9. y Image-scene compatibility function, F(xi, yi) Assume Gaussian noise takes you from observed image patch to synthetic sample: x

  10. d Scene-scene compatibility function, Y(xi, xj) Assume overlapped regions, d, of hi-res. patches differ by Gaussian observation noise: Uniqueness constraint, not smoothness.

  11. Form linking matrices between nodes scene samples at node xk F(xk, xj) Linking matrix: F(xk,xj)at samples 0.16 0.14 0.23 0.40 0.38 0.72 0.61 0.58 0.13 0.05 0.60 0.55 0.52 0.11 0.07 0.48 0.32 0.29 0.03 0.00 0.09 0.04 0.03 0.01 0.00 Local likelihoods are all 1 for the scene samples scene samples at node xj

  12. Markov network image patches F(xi, yi) scene patches Y(xi, xj)

  13. y1 x1 y2 x2 y3 x3 Derivation of belief propagation

  14. y1 y3 y2 x1 x3 x2 The posterior factorizes

  15. y1 y3 y2 x1 x3 x2 Propagation rules

  16. Belief, and message updates = i j i j

  17. Optimal solution in a chain or tree:Belief Propagation • “Do the right thing” Bayesian algorithm. • For Gaussian random variables over time: Kalman filter. • For hidden Markov models: forward/backward algorithm (and MAP variant is Viterbi).

  18. y1 y3 y2 x1 x3 x2 Y ( x , x ) 1 3 No factorization with loops!

  19. Justification for running belief propagation in networks with loops • Experimental results: • Error-correcting codes • Vision applications • Theoretical results: • For Gaussian processes, means are correct. • Large neighborhood local maximum for MAP. • Equivalent to Bethe approx. in statistical physics. Kschischang and Frey, 1998; McEliece et al., 1998 Freeman and Pasztor, 1999; Frey, 2000 Weiss and Freeman, 1999 Weiss and Freeman, 2000 Yedidia, Freeman, and Weiss, 2000

  20. VISTA--Vision by Image-Scene TrAining image patches scene patches image F(xi, yi) Y(xi, xj) scene

  21. Super-resolution application image patches F(xi, yi) scene patches Y(xi, xj)

  22. Belief Propagation After a few iterations of belief propagation, the algorithm selects spatially consistent high resolution interpretations for each low-resolution patch of the input image. Input Iter. 0 Iter. 1 Iter. 3

  23. Cubic spline zoom to 340x204 Max. likelihood zoom to 340x204 Zooming 2 octaves We apply the super-resolution algorithm recursively, zooming up 2 powers of 2, or a factor of 4 in each dimension. 85 x 51 input

  24. Generic training images Next, train on a generic set of training images. Using the same camera as for the test image, but a random collection of photographs.

  25. Original 70x70 Cubic Spline Markov net, training: generic True 280x280

  26. Training image

  27. Processed image

More Related