1 / 53

Some problems...

Some problems. Lens distortion. Uncalibrated structure and motion recovery assumes pinhole cameras Real cameras have real lenses How can we correct distortion , when original calibration is inaccessible?. Even small amounts of lens distortion can upset uncalibrated structure from motion

serena
Download Presentation

Some problems...

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Some problems...

  2. Lens distortion Uncalibrated structure and motion recovery assumes pinhole cameras Real cameras have real lenses How can we correct distortion, when original calibration is inaccessible?

  3. Even small amounts of lens distortion can upset uncalibrated structure from motion A single distortion parameter is enough for mapping and SFX accuracy Including the parameter kin the multiview relations changes the 8-point algorithm from You can solve such “Polynomial Eigenvalue Problems” This is as stable as computation of the Fundamental matrix, so you can use it all the time.

  4. E ven small amounts of lens distortion can upset uncalibrated structure from motion—

  5. A map-building problem • Input movie – relatively low distortion • Plan view: red is structure, blue is motion (a) (b)

  6. Effects of Distortion • Input movie – relatively low distortion • Recovered plan view, uncorrected distortion (a) (c)

  7. Distortion of image plane is conflated with focal length when the camera rotates [From: Tordoff & Murray, ICPR 2000] Does distortion do that?

  8. Distortion correction in man-made scenes

  9. Distortion correction in natural scenes [Farid and Popescu, ICCV 2001] • In natural images, distortion introduces correlations in frequency domain • Choose distortion parameters to minimize correlations in bispectrum • Less effective on man-made scenes....

  10. Distortion correction in multiple images Multiple views, static scene • Use motion and scene rigidity [Zhang, Stein, Sawhney, McLauchlan, ...] Advantages: • Applies to man-made or natural scenes Disadvantages: • Iterative solutions|require initial estimates

  11. A single distortion parameter is accurate enough for map-building and cinema post production—

  12. x:xeroxed noxious experimental artifax p:perfect pinhole perspective pure Modelling lens distortion p p x x Known Unknown

  13. Single-parameter models

  14. Single-parameter modelling power • Single-parameter model • Radial term only • Assumes distortion centre is at centre of image A one-parameter model suffices

  15. A direct solution for k

  16. Look at division model again

  17. A quick matlab session >> help polyeig POLYEIG Polynomial eigenvalue problem. [X,E] = POLYEIG(A0,A1,..,Ap) solves the polynomial eigenvalue problem of degree p: (A0 + lambda*A1 + ... + lambda^p*Ap)*x = 0. The input is [etc etc...] >>

  18. Algorithm

  19. T his is as stable as computation of the fundamental matrix, so you can use it all the time—

  20. Stable – small errorbars Biased – not centred on true value Performance: Synthetic data 0 -0.1 Computed l -0.2 -0.3 -0.4 0 0.2 0.4 0.6 0.8 1 Noise s (pixels)

  21. Best-fit line Analogy: Linear ellipse fitting True Fitted: 10 trials Data

  22. Performance: Synthetic data

  23. Performance: Real sequences

  24. 250 pairs Low distortion Linear estimate used to initialize nonlinear Number of inliers changes by [-25..49] 50 40 30 20 10 0 -0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3

  25. Conclusions

  26. In: magnifying glass moving over background Out: same magnifying glass, new background Environment matting

  27. Learn light-transport properties of complex optical elements Previously Ray tracing geometric models Calibrated acquisition Here Acquisition in situ Environment matting: why?

  28. Purely 2D-2D Optical element performs weighted sum of (image of) background at each pixel suffices for many interesting objects separate receptive field for each output pixel Environment matte is collection of all receptive fields—yes, it’s huge. Image formation model

  29. Image formation model

  30. Input: Mosaic: Step 1: Computing background Clean plate: Point tracks:

  31. Step 2: Computing w... Input:

  32. Computing w(x,y,u,v) at a single (x,y)

  33. Assume wi independent

  34. Composite over new background

  35. Input: Two images Moving camera Planar background - Need priors A more subtle example

  36. Window example

  37. Works well for non-translucent elements need to develop for diffuse Combination assumes independence ok for large movements: “an edge crosses the pixel” Need to develop for general backgrounds Discussion

  38. A Clustering Problem • Watch a movie, recover the cast list • Run face detector on every frame • Cluster faces • Problems • Face detector unreliable • Large lighting changes • Changes in expression • Clustering is difficult

  39. A sample sequence

  40. Detected faces

  41. Face positions

  42. Lighting correction

  43. Raw distance Clustering: pairwise distances

  44. Transform-invariant distance Clustering: pairwise distances

  45. Clusters: “tangent distance”

More Related