1 / 25

Space Time Tracking ECCV 2002

Space Time Tracking ECCV 2002. Lorenzo Torresani Christoph Bregler. Outline . Problem Background Structure from Motion Matrix Decomposition Non-Rigid Motion Estimation Non-Rigid Shapes Estimation Results. Problem.

avon
Download Presentation

Space Time Tracking ECCV 2002

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Space Time Tracking ECCV 2002 Lorenzo Torresani Christoph Bregler

  2. Outline • Problem • Background • Structure from Motion • Matrix Decomposition • Non-Rigid Motion Estimation • Non-Rigid Shapes Estimation • Results

  3. Problem “To track feature points on non-rigid objects without using any prior model”

  4. M N M N Rank of a Matrix Rank (A) = Number of linearly independent vectors in Rank (B) = Number of linearly independent vectors in For M x N the Rank of A ≤ min (M,N)

  5. Rank of a Matrix cont’d Rank C = ? Columns of C are Linear combination of Columns of A Rank C = N There are only N independent vectors in A

  6. VT n x n D n x n A m x n U m x n SVD • SVD for a matrix A writes A as a product of three matrices: • U • D • V • Every m x n matrix has a singular value decomposition U,V have orthogonal columns

  7. Frame 1 Frame 2 ………… Frame F Tomasi Kanade Structure from Motion • Given N 2D trajectories taken over F images, recover 3D structure and motion (Camera pose) • Assumption: • 3D Object is rigid • Orthographic Projection • Tracks can be computed using any standard tracker (KLT etc)

  8. Tomasi Kanade Structure from Motion cont’d • Assume a set of P 3D points on a rigid object (structure) S = [P1, P2 …….. PP ] • Orthographic Projection where (u,v) are image coordinates and M is orthographic projection matrix • Subtract mean of all u’s and v’s to center the world coordinate frame at the center of the object. • This will get rid of T in the above equation

  9. Tomasi Kanade Structure from Motion cont’d • 2D coordinates of N points over F images can be written in one matrix • W is called the measurement/tracking matrix Rank 3

  10. From W to R and S Force the rank of W to be 3 SVD

  11. Steps • Matrix Decomposition of W matrix for non-rigid objects • Estimate Motion Matrix using reliable set of points • Estimate shape basis (S) for all other feature points (unreliable)

  12. For Non Rigid Constraint

  13. S l1S1 l2S2 lKSK + + … + = S S S 1 2 K ……………… 3D Non Rigid Shape Model • Linear Combination of K Basis Shapes • Each basis shape is Si3 x P matrix describing P points Courtesy Christopher Bregler

  14. W Q M Matrix Decomposition • Project P points of shape S • Scaled Orthographic Projection • Move world coordinate to object centeroid (This will get rid of T)

  15. 2F x 3K 3K x P Tracking Matrix Complete 2D Tracks or Flow Rank of W 3K In Tomasi Kanade it was 3

  16. ? ? = known reliable tracks W Q’ M’ Non Rigid Motion Estimation • Since W isrank-deficient, Q can be estimated w/o the full availability of W • r <= 3K point tracks will span the space of the trajectories of all the points (as rank of W is r) Courtesy Christopher Bregler

  17. r = 9

  18. t=F . t=2 = 3D positions of point i for the K modes of deformation . t=1 . . . . . . frames wi : full trajectory mi Trajectory Constraint Q’ • Generate m trajectories (hypothesis) using Factored Sampling • Evaluate w by computing sum of square difference around point i. Courtesy Christopher Bregler

  19. Each column mi of unreliable M is computed as expected value of posterior.

  20. Results

More Related