objective l.
Skip this Video
Loading SlideShow in 5 Seconds..
Objective PowerPoint Presentation
Download Presentation

Loading in 2 Seconds...

play fullscreen
1 / 43

Objective - PowerPoint PPT Presentation

Download Presentation
An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Objective 3-D Scene u’ u Study the mathematical relations between corresponding image points. “Corresponding”means originated from the same 3D point.

  2. Two-views geometryOutline • Background: Camera, Projection models • Necessary tools: A taste of projective geometry • Two view geometry: • Planar scene (homography ). • Non-planar scene (epipolar geometry). • 3D reconstruction (stereo).

  3. Perspective Projection • Origin (0,0,0)is the Focal center • X,Y (x,y) axis are along the image axis (height / width). • Z is depth = distance along the Optical axis • f – Focal length

  4. Coordinates in Projective Plane P2 Take R3 –{0,0,0} and look at scale equivalence class (rays/lines trough the origin). k(0,1,1) k(1,1,1) “Ideal point” k(0,0,1) k(1,0,1) k(x,y,0)

  5. 2D Projective Geometry: Basics • A point: • A line: we denote a line with a 3-vector • Line coordinates are homogenous • Points and lines are dual: p is on l if • Intersection of two lines/points

  6. Cross Product in matrix notation [ ]x Hartley & Zisserman p. 581

  7. 2D Projective Transformation • H is defined up to scale • 9 parameters • 8 degrees of freedom • Determined by 4 corresponding points how does H operate on lines? Hartley & Zisserman p. 32

  8. Two-views geometryOutline • Background: Camera, Projection • Necessary tools: A taste of projective geometry • Two view geometry: • Homography • Epipolar geometry, the essential matrix • Camera calibration, the fundamental matrix • 3D reconstruction from two views (Stereo algorithms)

  9. Two View Geometry • When a camera changes position and orientation, the scene moves rigidly relative to the camera 3-D Scene u’ u Rotation + translation

  10. Two View Geometry (simple cases) • In two cases this results in homography: • Camera rotates around its focal point • The scene is planar Then: • Point correspondence forms 1:1mapping • depth cannot be recovered

  11. Camera Rotation (R is 3x3 non-singular)

  12. Planar Scenes Scene • Intuitively A sequence of two perspectivities • Algebraically Need to show: Camera 2 Camera 1

  13. Summary: Two Views Related by Homography Two images are related by homography: • One to one mapping from p to p’ • H contains 8 degrees of freedom • Given correspondences, each point determines 2 equations • 4 points are required to recover H • Depth cannot be recovered

  14. The General Case: Epipolar Lines epipolar line

  15. Epipolar Plane P epipolar plane epipolar line epipolar line O O’ Baseline

  16. Epipole • Every plane through the baseline is an epipolar plane • It determines a pair of epipolar lines (one in each image) • Two systems of epipolar lines are obtained • Each system intersects in a point, the epipole • The epipole is the projection of the center of the other camera epipolar plane epipolar lines epipolar lines O’ O Baseline

  17. Example

  18. Epipolar Lines To define an epipolar plane, we define the plane through the two camera centers O and O’ and some point P. This can be written algebraically (in some world coordinates as follows: P epipolar plane epipolar line epipolar line O O’ Baseline

  19. Essential Matrix (algebraic constraint between corresponding image points) • Set world coordinates around the first camera • What to do with O’P? Every rotation changes the observed coordinate in the second image • We need to de-rotate to make the second image plane parallel to the first • Replacing by image points Other derivations Hartley & Zisserman p. 241

  20. Essential Matrix (cont.) • Denote this by: • Then • Define • E is called the “essential matrix”

  21. Properties of the Essential Matrix • E is homogeneous • Its (right and left) null spaces are the two epipoles • 9 parameters • Is linear E, • E can be recovered up to scale using 8 points. • Has rank 2. • The constraint detE=0 7 points suffices • In fact, there are only 5 degrees of freedom in E, • 3 for rotation • 2 for translation (up to scale), determined by epipole

  22. Camera Internal Parameters or Calibration matrix • Background The lens optical axis does not coincide with the sensor • We model this using a 3x3 matrix the Calibration matrix

  23. Camera Calibration matrix • The difference between ideal sensor ant the real one is modeled by a 3x3 matrix K: • (cx,cy) camera center, (ax,ay) pixel dimensions, b skew • We end with

  24. Fundamental Matrix • F, is the fundamental matrix.

  25. Properties of the Fundamental Matrix • F is homogeneous • Its (right and left) null spaces are the two epipoles • 9 parameters • Is linear F, • F can be recovered up to scale using 8 points. • Has rank 2. • The constraint detF=0 7 points suffices

  26. Epipolar Plane P l x X’ l’ e’ e O O’ Baseline Other derivations Hartley & Zisserman p. 223

  27. Two-views geometry Summary:

  28. Stereo Vision • Objective: 3D reconstruction • Input: 2 (or more) images taken with calibrated cameras • Output: 3D structure of scene • Steps: • Rectification • Matching • Depth estimation

  29. Rectification • Image Reprojection • reproject image planes onto common plane parallel to baseline • Notice, only focal point of camera really matters (Seitz)

  30. Rectification • Any stereo pair can be rectified by rotating and scaling the two image planes (=homography) • We will assume images have been rectified so • Image planes of cameras are parallel. • Focal points are at same height. • Focal lengths same. • Then, epipolar lines fall along the horizontal scan lines of the images

  31. Cyclopean Coordinates • Origin at midpoint between camera centers • Axes parallel to those of the two (rectified) cameras

  32. Disparity • The difference is called “disparity” • dis inversely related to Z: greater sensitivity to nearby points • dis directly related to b: sensitivity to small baseline

  33. Main Step: Correspondence Search • What to match? • Objects? More identifiable, but difficult to compute • Pixels? Easier to handle, but maybe ambiguous • Edges? • Collections of pixels (regions)?

  34. Random Dot Stereogram • Using random dot pairs Julesz showed that recognition is not needed for stereo

  35. Random Dot in Motion

  36. Finding Matches

  37. 1D Search • More efficient • Fewer false matches SSD error disparity

  38. Ordering

  39. Ordering

  40. Comparison of Stereo Algorithms D. Scharstein and R. Szeliski. "A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms," International Journal of Computer Vision,47 (2002), pp. 7-42. Scene Ground truth

  41. Results with window correlation Window-based matching (best window size) Ground truth

  42. Scharstein and Szeliski

  43. Graph Cuts (next class). Graph cuts Ground truth