1 / 27

Lucas-Kanade Image Alignment

Lucas-Kanade Image Alignment. Slides from Iain Matthews. Applications of Image Alignment. Ubiquitous computer vision technique Tracking Registration of MRI/CT/PET. Generative Model for an Image. Parameterized model. shape. Parameters. Image. appearance. Fitting a Model to an Image.

bella
Download Presentation

Lucas-Kanade Image Alignment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Lucas-Kanade Image Alignment Slides from Iain Matthews

  2. Applications of Image Alignment • Ubiquitous computer vision technique • Tracking • Registration of MRI/CT/PET

  3. Generative Model for an Image • Parameterized model shape Parameters Image appearance

  4. Fitting a Model to an Image • What are the best model parameters to match an image? shape Parameters Image appearance • Nonlinear optimization problem

  5. Active Appearance Model Appearance Warp to reference Region of interest Landmarks Shape • Cootes, Edwards, Taylor, 1998

  6. Image Alignment Template, T(x) Warp, W(x;p+p) Warp, W(x;p) Image coordinates x = (x, y)T Warp parameters, p = (p1, p2, …, pn)T Image, I(x)

  7. Want to: Minimize the Error Template, T(x) Warped, I(W(x;p)) T(x) - I(W(x;p)) • Warp image to get compute

  8. How to: Minimize the Error - = Solution: solve for increments to current estimate, Minimise SSD with respect to p, Generally a nonlinear optimisation problem , … how can we solve this?

  9. Linearize For image alignment: Taylor series expansion, linearize function f about x0:

  10. Gradient Descent Solution Error Image Solution, Gradient Hessian Jacobian Least squares problem, solve for p

  11. Gradient Images Ix Iy • Compute image gradient W(x;p) W(x;p) I(W(x;p))

  12. Jacobian = 1 2 4 3 • Compute Jacobian Mesh parameterization 4 1 4 1 Warp, W(x;p) Template, T(x) Image, I(x) 3 2 3 2 Image coordinates x = (x, y)T Warp parameters, p = (p1, p2, …, pn)T = (dx1, dy1, …, dxn, dyn)T

  13. Lucas-Kanade Algorithm • Warp I with W(x;p)I(W(x;p)) • Compute error image T(x) - I(W(x;p)) • Warp gradient of I to compute I • Evaluate Jacobian • Compute Hessian • Compute p • Update parameters p  p + p - = 

  14. Fast Gradient Descent? • To reduce Hessian computation: • Make Jacobian simple (or constant) • Avoid computing gradients on I

  15. Shum-Szeliski Image Aligment T(x) T(x) W(x;p) W(x;p+p) • Compositional Alignment – Shum, Szeliski I(x) W(x;p)o W(x;p) W(x;p) W(x;p) I(W(x;p)) I(x) • Additive Image Alignment – Lucas, Kanade W(x;0 + p) = W(x;p)

  16. Compositional Image Alignment T(x) W(x;p)o W(x;p) W(x;p) W(x;p) I(W(x;p)) I(x) Minimise, Jacobian is constant, evaluated at (x, 0) “simple”.

  17. Compositional Algorithm • Warp I with W(x;p)I(W(x;p)) • Compute error image T(x) - I(W(x;p)) • Warp gradient of I to compute I • Evaluate Jacobian • Compute Hessian • Compute p • Update W(x;p) W(x;p)o W(x;p) - = 

  18. Inverse Compositional • Why compute updates on I? • Can we reverse the roles of the images? • Yes! [Baker, Matthews CMU-RI-TR-01-03] Proof that algorithms take the same steps (to first order)

  19. Inverse Compositional T(x) T(x) W(x;p)o W(x;p) W(x;p)o W(x;p)-1 W(x;p) W(x;p) W(x;p) W(x;p) I(W(x;p)) I(W(x;p)) I(x) I(x) • Forwards compositional • Inverse compositional

  20. Inverse Compositional • Minimise, • Solution • Update 

  21. Inverse Compositional • Jacobian is constant- evaluated at (x, 0) • Gradient of template is constant • Hessian is constant • Can pre-compute everything but error image!

  22. Inverse Compositional Algorithm • Warp I with W(x;p)I(W(x;p)) • Compute error image T(x) - I(W(x;p)) • Warp gradient of I to compute I • Evaluate Jacobian • Compute Hessian • Compute p • Update W(x;p) W(x;p)o W(x;p)-1 - = 

  23. Framework • Baker and Matthews 2003 Formulated framework, proved equivalence

  24. Example

  25. Reprise… what have we solved for?

  26. Lucas-Kanade Algorithm Criterion :

More Related