1 / 48

3-D Computational Vision CSc 83029

3-D Computational Vision CSc 83029. Optical Flow & Motion The Factorization Method. Optical Flow & Motion. Finding the movement of scene objects from time-varying images. Motion Field Optical Flow Computing Optical Flow. Computing Time-to-Impact τ. v. l(t)=f L/D(t). L. l(t). D 0. f.

yvonne
Download Presentation

3-D Computational Vision CSc 83029

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 3-D Computational VisionCSc 83029 Optical Flow & Motion The Factorization Method CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  2. Optical Flow & Motion • Finding the movement of scene objects from time-varying images. • Motion Field • Optical Flow • Computing Optical Flow CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  3. CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  4. Computing Time-to-Impact τ v l(t)=f L/D(t) L l(t) D0 f D0-vt CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  5. Computing Time-to-Impact τ v l(t)=f L/D(t) L l(t) D0 f D0-vt l(t) / l’(t) = τ Quantities measured from image sequence. CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  6. Sudden change in viewing position/direction: Hard to compute motion field/optical flow. CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  7. Views from a sequence of spatially close viewpoints: Motion Field/ Optical Flow CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  8. Sub problems of Motion Analysis • Correspondence. • Reconstruction. • Segmentation. CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  9. Motion Field 2-D vector field of velocities of image points, induced by relative motion between the viewing camera and the observed points. Image plane f v dt ri p V dt ro P Scene Point Velocity V=dro/dt Image Velocity v =dri/dt CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  10. Motion Field 2-D vector field of velocities of image points, induced by relative motion between the viewing camera and the observed points. Image plane f v dt ri p V dt Perspective projection ro P Velocity of point P as a function of translation and rotation Motion field equations CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  11. Motion Field 2-D vector field of velocities of image points, induced by relative motion between the viewing camera and the observed points => SUM of 2 COMPONENTS: CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  12. Motion Field 2-D vector field of velocities of image points, induced by relative motion between the viewing camera and the observed points => SUM of 2 COMPONENTS: + CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  13. Special Case 1: Pure Translation 2-D vector field of velocities of image points, induced by relative motion between the viewing camera and the observed points => SUM of 2 COMPONENTS: + CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  14. Pure Translation: Radial Motion Field p0=(x0,y0) CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  15. Pure Translation: Radial Motion Field • Tz < 0 : FOCUS OF EXPANSION. • Tz > 0: FOCUS OF CONTRACTION. • Tz = 0: PARALLEL MOTION FIELD. • Vanishing point (epipole) p0. p0=(x0,y0) CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  16. Special Case 2: Moving Plane n P Plane moves: n,d are functions of time. Motion field=? CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  17. Special Case 2: Moving Plane n P Plane moves: n,d are functions of time. Motion field=? CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  18. Special Case 2: Moving Plane n P Plane moves: n,d are functions of time. Motion field=? Motion field: quadratic polynomial in (x,y,f) at any time t. The same motion field can be produced by 2 different planes undergoing 2 different 3-D motions. CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  19. The notion of Optical Flow Optical Flow: Estimation of the motion field from a sequence of images. CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  20. Optical Flow CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  21. Optical Flow (x+uδt,y+vδt) u=δx/δt v=δy/δt (x,y) t t+δt Image brightness constancy equation: E(x,y,t)=E(x+uδt, y+vδt, t+δt) or CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  22. The Aperture Problem The component of the motion field in the direction orthogonal to the spatial image gradient is not constrained by the image brightness constancy equation. Aperture problem Image brightness constancy equation: E(x,y,t)=E(x+uδt, y+vδt, t+δt) 1 constraint 2 unknowns CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  23. Optical Flow At each point we know dE/dx dE/dy and dE/dt. How can we obtain dx/dt and dy/dt? CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  24. Computing Optical Flow Assumption: The motion field is well approximated by a constant vector field within any small patch of the image plane. provides one constraint Each Solution is CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  25. Computing Optical Flow Assumption: The motion field is well approximated by a constant vector field within any small patch of the image plane. Q Minimize CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  26. Computing Optical Flow Assumption: The motion field is well approximated by a constant vector field within any small patch of the image plane. Q Minimize =>Solve the linear system CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  27. Computing Optical Flowglobal approach [Schunck and Horn 81] Minimize the error in the image brightness constancy constraint. CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  28. Computing Optical Flowglobal approach [Schunck and Horn 81] Minimize the error in the image brightness constancy constraint. Minimize the deviation from smoothness of the motion vectors. CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  29. Computing Optical Flowglobal approach [Schunck and Horn 81] Minimize the error in the image brightness constancy constraint. Minimize the deviation from smoothness of the motion vectors. Find solution by minimizing where lambda weights the smoothness term. CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  30. Optical Flow CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  31. Optical Flow CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  32. Optical Flow CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  33. Optical Flow CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  34. Optical Flow If the scene is planar the motion Is described by Using Solve for the 8 unknowns a, b, c, d, e, f, g and h. CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  35. B Tracking Rigid Bodies B Random Sampling Algorithm Step 1: Find corners Step 2: Search for correspondence Step 3: Randomly choose small set of matches. Step 4: Estimate F matrix Step 5: Find total number of matches close to epipolar lines Step 6: Go to step 3 Step 7: Choose F with largest number of matches CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  36. CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  37. Structure and Motion Recovery from Video 1. Use multiple image stream to compute the information about camera motion and 3D structure of the scene 2. Tracking image features over time Original sequence Tracked Features From Jana Kosecka CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  38. Structure and Motion Recovery from Video Computed model 3D coordinates of the feature points Original picture From Jana Kosecka CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  39. Factorization Method ji ki ii FRAMES: i=1…N CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  40. Factorization Method Pj ji ki (xij,yij) ii FRAMES: i=1…N World Points: j=1…n CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  41. Factorization Method Pj Z ji ki (xij,yij) Ti ii FRAMES: i=1…N World Points: j=1…n Y X World Reference Frame CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  42. ASSUMPTIONS: The camera model is orthographic! The positions of n image points have been tracked. Pj Z ji ki (xij,yij) Ti ii FRAMES: i=1…N World Points: j=1…n Y X World Reference Frame CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  43. Measurement Matrix 2*N (Frames) n points per frame Registered Measurement Matrix 2*N (Frames) n points per frame CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  44. Factorization Method World Points: j=1…n Pj 3D Centroid Z ji ki Ti ii FRAMES: i=1…N Y X 2D Centroid World Reference Frame CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  45. Rank Theorem 2*N (Frames) n points per frame R: “Rotation Matrix” S: Shape Matrix CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  46. Rank Theorem 2*N (Frames) n points per frame R: “Rotation Matrix” S: Shape Matrix Rank of is 3. CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  47. The algorithm 2*N (Frames) n points per frame Decompose into R and S. Is the decomposition unique? Translation estimation? CSc83029 – 3-D Computer Vision/ Ioannis Stamos

  48. CSc83029 – 3-D Computer Vision/ Ioannis Stamos

More Related