1 / 35

CV: 3D sensing and calibration

CV: 3D sensing and calibration. Coordinate system changes; perspective transformation; Stereo and structured light. roadmap. using multiple cameras using structured light projector 3D transformations general perspective transformation justification of 3x4 camera model.

mgraves
Download Presentation

CV: 3D sensing and calibration

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CV: 3D sensing and calibration Coordinate system changes; perspective transformation; Stereo and structured light MSU CSE 803 Fall 2008 Stockman

  2. roadmap • using multiple cameras • using structured light projector • 3D transformations • general perspective transformation • justification of 3x4 camera model MSU CSE 803 Fall 2008 Stockman

  3. Four Coordinate frames W: world, C,D: cameras, M: object model Need to relate all to each other. MSU CSE 803 Fall 2008 Stockman

  4. Is there some object M That can be placed in some location That will create the two images that are observed? Discover/compute what object and what pose Can we recognize? MSU CSE 803 Fall 2008 Stockman

  5. Need to relate frames to compute • relate camera to world using rotations and translations • project world point into real image using projection • scale image point in real image plane to get pixel array coordinates MSU CSE 803 Fall 2008 Stockman

  6. Stereo configuration 2 corresponding image points enable the intersection of 2 rays in W MSU CSE 803 Fall 2008 Stockman

  7. Stereo computation MSU CSE 803 Fall 2008 Stockman

  8. Math for stereo computations • need to calibrate both cameras to W so that rays in x,y,z reference same space • need to have corresponding points • find point of closest approach of the two rays • (rays are too far apart  point correspondence error or crude calibration) MSU CSE 803 Fall 2008 Stockman

  9. Replace camera with projector Can calibrate a projector to W easily. Correspondence now means identifying marks. MSU CSE 803 Fall 2008 Stockman

  10. Advantages/disadvantages of structured light MSU CSE 803 Fall 2008 Stockman

  11. Grid projected on objects All grid intersects are integral MSU CSE 803 Fall 2008 Stockman

  12. Computing surface normals Surface normals have been computed and then added to the image (augmented reality) MSU CSE 803 Fall 2008 Stockman

  13. Relating coordinate frames • need to relate camera frame to world • need to rotate, translate, and scale coordinate systems • need to project world points to the image plane • all the above are modeled using 4x4 matrices and 1x4 points in homogeneous coordinates MSU CSE 803 Fall 2008 Stockman

  14. Translation of 3D point P Point in frame 1 Point in frame 2 Point in 3D parameters MSU CSE 803 Fall 2008 Stockman

  15. Scaling 3D point P MSU CSE 803 Fall 2008 Stockman

  16. Rotation of P about the X-axis MSU CSE 803 Fall 2008 Stockman

  17. Rotate P about the Y-axis MSU CSE 803 Fall 2008 Stockman

  18. Rotate P about the Z-axis Looks same as 2D rotation omitting row, col 4 MSU CSE 803 Fall 2008 Stockman

  19. Arbitrary rotation has orthonormal rows and columns MSU CSE 803 Fall 2008 Stockman

  20. Example: camera relative to world MSU CSE 803 Fall 2008 Stockman

  21. exercise • verify that the 3 x 3 rotation matrix is orthonormal by checking 6 dot products • invert the 3 x 3 rotation matrix • invert the 4 x 4 matrix • verify that the new matrix transforms points correctly from C to W MSU CSE 803 Fall 2008 Stockman

  22. Transformation “calculus”: notation accounts for transforms Destination frame W Denotes transformation W T M T transforms points from model frame to world frame. (Notation from John Craig, 1986) Origin frame M MSU CSE 803 Fall 2008 Stockman

  23. Apply transformations to points Point in model coordinates W W M P = T P M Point in world coordinates Transformation from model to world coordinates (instance transformation) MSU CSE 803 Fall 2008 Stockman

  24. Matrix algebra enables composition • Let M and N be 4 x 4 matrices and let P be a 4 x 1 point • M ( N P ) = ( M N ) P • we can transform P using N and then transform that by M, or we can multiply matrices M and N and then apply that to point P • matrix multiplication is associative (but not commutative) MSU CSE 803 Fall 2008 Stockman

  25. Composing transformations Projection parms. Parameters: rotation and translation cancel A A C T (w) T = T(p) W W C Two transformations are composed to get one transformation that maps points from the world frame to the frame A MSU CSE 803 Fall 2008 Stockman

  26. Deriving form of the camera matrix We have already described what the camera matrix does and what form it has; we now go through the steps to justify it MSU CSE 803 Fall 2008 Stockman

  27. Viewing model points M What’s in front of the camera? MSU CSE 803 Fall 2008 Stockman

  28. Math for the steps Camera C maps 3D points in world W to 2D pixels in image I MSU CSE 803 Fall 2008 Stockman

  29. Perspective transformation: camera origin at the center of projection This transformation uses same units in 3D as in the image plane MSU CSE 803 Fall 2008 Stockman

  30. Perspective projection: camera origin in the real image plane MSU CSE 803 Fall 2008 Stockman

  31. Rigid transformation for change of coordinate frame 3D coordinate frame of camera MSU CSE 803 Fall 2008 Stockman

  32. Relate camera frame to world frame MSU CSE 803 Fall 2008 Stockman

  33. Change scene units to pixels To get into XV or GIMP image coordinates! This is a 2D to 2D transformation. MSU CSE 803 Fall 2008 Stockman

  34. Final result • a 3x4 camera matrix • maps 2D image pixels to 3D rays • maps 3D rays to 2D image pixels • obtain matrix via calibration (easy) • obtain matrix via reasoning (hard) • do camera calibration exercise MSU CSE 803 Fall 2008 Stockman

  35. The camera model, or matrix, is 3 x 4 and maps a homogeneous point in the world to a homogeneous pixel in the image. The ‘1’ is used to model translation MSU CSE 803 Fall 2008 Stockman

More Related