3d reconstruction class 16 l.
Skip this Video
Loading SlideShow in 5 Seconds..
3D reconstruction Class 16 PowerPoint Presentation
Download Presentation
3D reconstruction Class 16

Loading in 2 Seconds...

  share
play fullscreen
1 / 63
Download Presentation

3D reconstruction Class 16 - PowerPoint PPT Presentation

Ava
238 Views
Download Presentation

3D reconstruction Class 16

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. 3D reconstruction Class 16

  2. 3D photography course schedule

  3. Papers http://www.unc.edu/courses/2004fall/comp/290b/089/papers/

  4. Ideas for a project?

  5. 3D modeling • Aligning range images • Pairwise • Globally • Surface reconstruction • Single range image • Merged (some slides from S. Rusinkiewicz, J. Ponce,…)

  6. Aligning 3D Data • If correct correspondences are known,it is possible to find correct relative rotation/translation

  7. Norm: Intermezzo: quaternions q is aquaternion, a 2R is itsrealpart, and 2R3 is itsimaginarypart. q = a +  Operations on quaternions: • Sum of quaternions: • Multiplication by a scalar: • Quaternion product: • Conjugate: ) Note:

  8. Intermezzo: quaternions and rotations Let R denote the rotation of angle  about the unit vector u. Define Then for any vector , Reciprocally, if q = a + ( b, c, d )T is a unit quaternion, the corresponding rotation matrix is:

  9. n n E =  | xi’ – R xi – t |2 . E =  | yi’ – R yi |2 i=1 i=1 n  ( xi’ – R xi – t ) . At a minimum: 0 = E/t = –2 Or.. t = x’ – R x. i=1 If yi = xi –x and yi’ = xi’ –x’, the error is (at a minimum): n n =  | yi’ q – qyi |2 =  | yi’ – q yi q|2|q|2 i=1 i=1 Linear least squares !! Estimate rigid transformation Problem: Find the rotation matrix R and the vector t that minimize Or E =

  10. Aligning 3D Data • How to find corresponding points? • Previous systems based on user input,feature matching, surface signatures, etc.

  11. Aligning 3D Data Alternative: assume closest points correspond to each other, compute the best transform…

  12. Aligning 3D Data … and iterate to find alignment Iterated Closest Points (ICP) [Besl & McKay 92] Converges if starting position “close enough“

  13. ICP Variants • Classic ICP algorithm not real-time • To improve speed: examine stages of ICP and evaluate proposed variants [Rusinkiewicz & Levoy, 3DIM 2001] • Selecting source points (from one or both meshes) • Matching to points in the other mesh • Weighting the correspondences • Rejecting certain (outlier) point pairs • Assigning an error metric to the current transform • Minimizing the error metric

  14. ICP Variant – Point-to-Plane Error Metric • Using point-to-plane distance instead of point-to-point lets flat regions slide along each other more easily [Chen & Medioni 91]

  15. Finding Corresponding Points • Finding closest point is most expensive stage of ICP • Brute force search – O(n) • Spatial data structure (e.g., k-d tree) – O(log n) • Voxel grid – O(1), but large constant, slow preprocessing

  16. Finding Corresponding Points • For range images, simply project point [Blais 95] • Constant-time, fast • Does not require precomputing a spatial data structure

  17. High-Speed ICP Algorithm • ICP algorithm with projection-based correspondences, point-to-plane matchingcan align meshes in a few tens of ms.(cf. over 1 sec. with closest-point) • [Rusinkiewicz & Levoy, 3DIM 2001]

  18. 3D Global Registration

  19. 3D Global registration The problem: • Given: n scans around an object • Goal: align them all • First attempt: ICP each scan to one other

  20. 3D Global registration • Want method for distributing accumulated error among all scans

  21. Approach #1: Avoid the Problem • In some cases have 1 scan that covers large part of surface (e.g., cylindrical scan) • Align all other scans to this “anchor” • Disadvantage: not always practical to obtain anchor scan

  22. Approach #2: The Greedy Solution • Align each new scan to all previous scans • Disadvantages: • Order dependent • Doesn’t spread out error

  23. Approach #3: Brute-Force Solution • While not converged: • For each scan: • For each point: • For every other scan • Find closest point • Minimize error w.r.t. transforms of all scans • Disadvantage: • Solve (np)(np) matrix equation, where n is number of scans and p is number of points per scan

  24. Approach #3a: Slightly Less Brute-Force • While not converged: • For each scan: • For each point: • For every other scan • Find closest point • Minimize error w.r.t. transform of this scan • Faster than previous method (matrices are pp)

  25. Graph Methods • Many globalreg algorithms create a graph of pairwise alignments between scans Scan 3 Scan 5 Scan 1 Scan 2 Scan 4 Scan 6

  26. Pulli’s Algorithm • Perform pairwise ICPs, record sample (e.g. 200) of corresponding points • For each scan, starting w. most connected • Align scan to existing set • While (change in error) > threshold • Align each scan to others • All alignments during globalreg phase use precomputed corresponding points

  27. Sharp et al. Algorithm • Perform pairwise ICPs, record only optimal rotation/translation for each • Decompose alignment graph into cycles • While (change in error) > tolerance • For each cycle: • Spread out error equally among all scans in the cycle • For each scan belonging to more than 1 cycle: • Assign average transform to scan

  28. Lu and Milios Algorithm • Perform pairwise ICPs, record optimal rotation/translation and covariance for each • Least squares simultaneous minimization of all errors (covariance-weighted) • Requires linearization of rotations • Worse than the ICP case, since don’t converge to (incremental rotation) = 0

  29. Open Questions in Global Registration • Best way to do correctly-weighted globalreg without linearizing rotations? • How to prevent bias (if many scans in one area, few scans in another)? • Robust outlier detection • For graph methods, pairwise ICP often automated • One bad ICP can throw off the entire model

  30. Bad ICP in Globalreg One bad ICP can throw off the entire model Correct Globalreg Globalreg Including Bad ICP

  31. Huber’s Automatic Modeling System • Start with unaligned cans • Generate candidate pairwise alignments using spin images • Incrementally add scans, checking consistency • ICP error in overlapping regions • Free space consistency • Occupied space consistency

  32. Spin Images • Johnson and Hebert • “Signature” that captures local shape • Similar shapes  similar spin images

  33. Computing Spin Images • Start with a point on a 3D model • Find (averaged) surface normal at that point • Define coordinate system centered at this point, oriented according to surface normal and two (arbitrary) tangents • Express other points (within some distance) in terms of the new coordinates

  34. Computing Spin Images • Compute histogram of locations of other points, in new coordinate system, ignoring rotation around normal:

  35. Computing Spin Images

  36. Spin Image Parameters • Size of neighborhood • Determines whether local or global shapeis captured • Big neighborhood: more discriminatory power • Small neighborhood: resistance to clutter • Size of bins in histogram: • Big bins: less sensitive to noise • Small bins: captures more detail, less storage

  37. Using Spin Images for Alignment Not robust

  38. Verification – Overlap Distance Views 1 and 2: 48%, 2.1 mm Views 1 and 9: 15%, 3.1 mm

  39. Verification – Visibility Consistency surfaces are close S2 blocks S1 S1 is not observed S1 S1 S2 S2 S1 S2 C1 C1 C2 C2 C1 C2 consistentsurfaces free spaceviolation occupied spaceviolation

  40. 14 15 13 10 9 3 6 6 12 11 top view 5 5 1 8 8 10 9 13 11 15 2 3 2 4 4 7 7 1 12 14 horizontal slice Automatically Modeling the Angel initial model(bad matches in red) connectivity afterlocal verification

  41. 4 4 6 6 7 7 5 5 8 8 10 10 9 9 13 13 11 11 15 15 3 3 2 2 1 1 12 12 14 14 Inferring New Pose Relations 30% overlap 1% overlap

  42. top view horizontal slice The Final Angel Model

  43. Problems With Reconstruction from Point Clouds

  44. Surface Reconstruction from Range Images • Often an easier problem than reconstruction from arbitrary point clouds • Implicit information about adjacency, connectivity • Roughly uniform spacing

  45. Surface Reconstruction From Range Images • First, construct surface from each range image • Then, merge resulting surfaces • Obtain average surface in overlapping regions • Control point density

  46. Range Image Tesselation • Given a range image, connect up the neighbors

  47. 3D surface model Depth image Texture image Triangle mesh Textured 3D Wireframe model

  48. Range Image Tesselation • Caveat #1: can’t be too aggressive • Introduce distance threshold for tesselation

  49. Range Image Tesselation • Caveat #2: Which way to triangulate? • Possible heuristics: • Shorter diagonal • Dihedral angle closer to 180 • Maximize smallest angle in both triangles • Always the same way (best triangle strips)

  50. Scan Merging Using Zippering • Turk & Levoy, 1994 • Erode geometry in overlapping areas • Stitch scans together along seam • Re-introduce all data • Weighted average