238 Views

Download Presentation
## 3D reconstruction Class 16

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Papers**http://www.unc.edu/courses/2004fall/comp/290b/089/papers/**3D modeling**• Aligning range images • Pairwise • Globally • Surface reconstruction • Single range image • Merged (some slides from S. Rusinkiewicz, J. Ponce,…)**Aligning 3D Data**• If correct correspondences are known,it is possible to find correct relative rotation/translation**Norm:**Intermezzo: quaternions q is aquaternion, a 2R is itsrealpart, and 2R3 is itsimaginarypart. q = a + Operations on quaternions: • Sum of quaternions: • Multiplication by a scalar: • Quaternion product: • Conjugate: ) Note:**Intermezzo: quaternions and rotations**Let R denote the rotation of angle about the unit vector u. Define Then for any vector , Reciprocally, if q = a + ( b, c, d )T is a unit quaternion, the corresponding rotation matrix is:**n**n E = | xi’ – R xi – t |2 . E = | yi’ – R yi |2 i=1 i=1 n ( xi’ – R xi – t ) . At a minimum: 0 = E/t = –2 Or.. t = x’ – R x. i=1 If yi = xi –x and yi’ = xi’ –x’, the error is (at a minimum): n n = | yi’ q – qyi |2 = | yi’ – q yi q|2|q|2 i=1 i=1 Linear least squares !! Estimate rigid transformation Problem: Find the rotation matrix R and the vector t that minimize Or E =**Aligning 3D Data**• How to find corresponding points? • Previous systems based on user input,feature matching, surface signatures, etc.**Aligning 3D Data**Alternative: assume closest points correspond to each other, compute the best transform…**Aligning 3D Data**… and iterate to find alignment Iterated Closest Points (ICP) [Besl & McKay 92] Converges if starting position “close enough“**ICP Variants**• Classic ICP algorithm not real-time • To improve speed: examine stages of ICP and evaluate proposed variants [Rusinkiewicz & Levoy, 3DIM 2001] • Selecting source points (from one or both meshes) • Matching to points in the other mesh • Weighting the correspondences • Rejecting certain (outlier) point pairs • Assigning an error metric to the current transform • Minimizing the error metric**ICP Variant – Point-to-Plane Error Metric**• Using point-to-plane distance instead of point-to-point lets flat regions slide along each other more easily [Chen & Medioni 91]**Finding Corresponding Points**• Finding closest point is most expensive stage of ICP • Brute force search – O(n) • Spatial data structure (e.g., k-d tree) – O(log n) • Voxel grid – O(1), but large constant, slow preprocessing**Finding Corresponding Points**• For range images, simply project point [Blais 95] • Constant-time, fast • Does not require precomputing a spatial data structure**High-Speed ICP Algorithm**• ICP algorithm with projection-based correspondences, point-to-plane matchingcan align meshes in a few tens of ms.(cf. over 1 sec. with closest-point) • [Rusinkiewicz & Levoy, 3DIM 2001]**3D Global registration**The problem: • Given: n scans around an object • Goal: align them all • First attempt: ICP each scan to one other**3D Global registration**• Want method for distributing accumulated error among all scans**Approach #1: Avoid the Problem**• In some cases have 1 scan that covers large part of surface (e.g., cylindrical scan) • Align all other scans to this “anchor” • Disadvantage: not always practical to obtain anchor scan**Approach #2: The Greedy Solution**• Align each new scan to all previous scans • Disadvantages: • Order dependent • Doesn’t spread out error**Approach #3: Brute-Force Solution**• While not converged: • For each scan: • For each point: • For every other scan • Find closest point • Minimize error w.r.t. transforms of all scans • Disadvantage: • Solve (np)(np) matrix equation, where n is number of scans and p is number of points per scan**Approach #3a: Slightly Less Brute-Force**• While not converged: • For each scan: • For each point: • For every other scan • Find closest point • Minimize error w.r.t. transform of this scan • Faster than previous method (matrices are pp)**Graph Methods**• Many globalreg algorithms create a graph of pairwise alignments between scans Scan 3 Scan 5 Scan 1 Scan 2 Scan 4 Scan 6**Pulli’s Algorithm**• Perform pairwise ICPs, record sample (e.g. 200) of corresponding points • For each scan, starting w. most connected • Align scan to existing set • While (change in error) > threshold • Align each scan to others • All alignments during globalreg phase use precomputed corresponding points**Sharp et al. Algorithm**• Perform pairwise ICPs, record only optimal rotation/translation for each • Decompose alignment graph into cycles • While (change in error) > tolerance • For each cycle: • Spread out error equally among all scans in the cycle • For each scan belonging to more than 1 cycle: • Assign average transform to scan**Lu and Milios Algorithm**• Perform pairwise ICPs, record optimal rotation/translation and covariance for each • Least squares simultaneous minimization of all errors (covariance-weighted) • Requires linearization of rotations • Worse than the ICP case, since don’t converge to (incremental rotation) = 0**Open Questions in Global Registration**• Best way to do correctly-weighted globalreg without linearizing rotations? • How to prevent bias (if many scans in one area, few scans in another)? • Robust outlier detection • For graph methods, pairwise ICP often automated • One bad ICP can throw off the entire model**Bad ICP in Globalreg**One bad ICP can throw off the entire model Correct Globalreg Globalreg Including Bad ICP**Huber’s Automatic Modeling System**• Start with unaligned cans • Generate candidate pairwise alignments using spin images • Incrementally add scans, checking consistency • ICP error in overlapping regions • Free space consistency • Occupied space consistency**Spin Images**• Johnson and Hebert • “Signature” that captures local shape • Similar shapes similar spin images**Computing Spin Images**• Start with a point on a 3D model • Find (averaged) surface normal at that point • Define coordinate system centered at this point, oriented according to surface normal and two (arbitrary) tangents • Express other points (within some distance) in terms of the new coordinates**Computing Spin Images**• Compute histogram of locations of other points, in new coordinate system, ignoring rotation around normal:**Spin Image Parameters**• Size of neighborhood • Determines whether local or global shapeis captured • Big neighborhood: more discriminatory power • Small neighborhood: resistance to clutter • Size of bins in histogram: • Big bins: less sensitive to noise • Small bins: captures more detail, less storage**Using Spin Images for Alignment**Not robust**Verification – Overlap Distance**Views 1 and 2: 48%, 2.1 mm Views 1 and 9: 15%, 3.1 mm**Verification – Visibility Consistency**surfaces are close S2 blocks S1 S1 is not observed S1 S1 S2 S2 S1 S2 C1 C1 C2 C2 C1 C2 consistentsurfaces free spaceviolation occupied spaceviolation**14**15 13 10 9 3 6 6 12 11 top view 5 5 1 8 8 10 9 13 11 15 2 3 2 4 4 7 7 1 12 14 horizontal slice Automatically Modeling the Angel initial model(bad matches in red) connectivity afterlocal verification**4**4 6 6 7 7 5 5 8 8 10 10 9 9 13 13 11 11 15 15 3 3 2 2 1 1 12 12 14 14 Inferring New Pose Relations 30% overlap 1% overlap**top view**horizontal slice The Final Angel Model**Surface Reconstruction from Range Images**• Often an easier problem than reconstruction from arbitrary point clouds • Implicit information about adjacency, connectivity • Roughly uniform spacing**Surface Reconstruction From Range Images**• First, construct surface from each range image • Then, merge resulting surfaces • Obtain average surface in overlapping regions • Control point density**Range Image Tesselation**• Given a range image, connect up the neighbors**3D surface model**Depth image Texture image Triangle mesh Textured 3D Wireframe model**Range Image Tesselation**• Caveat #1: can’t be too aggressive • Introduce distance threshold for tesselation**Range Image Tesselation**• Caveat #2: Which way to triangulate? • Possible heuristics: • Shorter diagonal • Dihedral angle closer to 180 • Maximize smallest angle in both triangles • Always the same way (best triangle strips)**Scan Merging Using Zippering**• Turk & Levoy, 1994 • Erode geometry in overlapping areas • Stitch scans together along seam • Re-introduce all data • Weighted average