Mesh Zippering from Range Images

1. Mesh Zippering from Range Images Avneesh Sud COMP 258, Fall 2000

2. What Is a Range Image? • An mxn grid of distances (range points) that describe a surface in Cartesian (height field) or cylindrical coordinates. • A range scanner senses 3D positions on an objects surface and returns an array of distance values. Avneesh Sud, COMP258

3. Creating a single 3D model • Acquisition : Capture multiple range images • Registration : A rigid transformation that brings points of one range image into alignment with portions of a surface it shares with another range image • Integration : Create a single surface rep from sample points in multiple images Avneesh Sud, COMP258

4. Acquisition: Range Scanners • Tactile Sensing • Computer Vision Techniques (stereo, shading, texture, motion) • Laser, radar, sonar rangefinders (time-of-flight, triangulation) • Turk and Levoy use a triangulation based laser rangefinder. Avneesh Sud, COMP258

5. Structured Light Range Scanner Optical Triangulation • Sensor view direction is off-axis from source of light • Center of image pulse maps to a center of the laser, giving a range value Avneesh Sud, COMP258

6. Creating Triangle Meshes • Four adjacent points on a range image are joined to form 0,1 or 2 triangles. • The shorter diagonal must be longer than a distance threshold. Avneesh Sud, COMP258

7. Registration • Wada et al : Matching distinctive facets from convex hulls • Champleboux et al : Octree-spline rep of distances to object surface • Chen & Medioni : Correspondence of points and nearby tangent planes • Turk & Levoy : Adapted Iterated Closest Point Algorithm (Besl & McKay) Avneesh Sud, COMP258

8. Integration • Unorganised Points • Boissanat : Delaunay Triangulation in 3-space • Edelsbrunner &Mucke : “alpha-shapes” • Hoppe et al : implicit surface reconstruction • Structured Data • Implicit functions • Parametric Approaches Avneesh Sud, COMP258

9. Registration : Turk & Levoy • Original ICP requires every point on one surface have a corresponding point on the other surface • Distance/Boundary constraint on pairs of points Avneesh Sud, COMP258

10. Registration Algorithm • Find the nearest position on mesh A to each vertex of mesh B • Discard pairs of points which are too far apart • Eliminate pairs in which either point is on a boundary • Find the rigid transformation that minimizes a weighted least-squared distance • Iterate till convergence • Perform ICP on a more detailed mesh in the hierarchy Avneesh Sud, COMP258

11. Integration : Mesh Zippering • Remove overlapping portions of the meshes • Clip one mesh against another • Remove small triangles introduced in clipping • Fine tuning using “consensus geometry” Avneesh Sud, COMP258

12. Integration : Removing Redundant Triangles • Remove redundant triangles on boundary of mesh A and mesh B till both meshes remain unchanged • A triangle is redundant if its vertices lie within certain tolerance distance of the other mesh Mesh B Mesh A Avneesh Sud, COMP258

13. Integration : Mesh Clipping • New vertices added at intersection of meshes • Portion inside the boundary of other mesh discarded • Boundary is re-triangulated Avneesh Sud, COMP258

14. Integration: Mesh Clipping • In 3-space meshes may not exactly intersect • “Hack” : Thicken boundary of one mesh Avneesh Sud, COMP258

15. Integration: Removing Small Triangles • Small triangles introduced during re-triangulation may be undesirable • Remove a triangle’s vertex and use constrained triangulation to fill the hole • New vertices introduced during clipping are preferentially deleted Avneesh Sud, COMP258

16. Example Left (top and bottom): Meshes created from two range images of a telephone. Red denotes locations of high confidence and blue shows low confidence. Note the low confidence at the edges to account for false edge extensions. Top middle: The two meshes (colored red and white) after alignment. Bottom middle: Close­up of aligned meshes that shows a jagged ridge of triangles that is the false edge extension of the white mesh at a corner. Top right: The meshes after redundant surface removal. Bottom right: The meshes after zippering Avneesh Sud, COMP258

17. Consensus Geometry: Fine Tuning • Aim : Incorporate all information from each range image • Vertices are moved in the direction of the surface normal to a consensus position given by a weighted average of positions from original range images • Find a local approximation to the surface normal N at vertex V • Intersect each mesh with the line passing through V along N • Consensus position of V is the average of all points of intersection near V Avneesh Sud, COMP258

18. Example : Dinosaur 14 range images, 360k triangles Avneesh Sud, COMP258

19. Registration: Champleboux • Addresses sensor calibration and 3D object localization • Solves localization as an energy minimization • Uses an “octree spline” data structure for rapidly computing distances from a surface with low storage overhead Avneesh Sud, COMP258

20. Octree Spline Distance Rep • Octree decomposition of the surface points, with a continuous approximating distance function to the surface • Neighbors differ in size by atmost a factor of 2 • Distances d(q,S) is computed for each corner point of a node • Distance d(q,S) of any point can be computed from trilinear interpolation of 8 corner points of the terminal node point q lies in • Intuition: Higher accuracy near the surface Avneesh Sud, COMP258

21. Localization Problem Formulation • Known : surface S in Refmodel, Mp points qi in Refsensor • To determine : 6DOF transform matrix T(p) such that • Each point qi is transformed to point ri on surface S • ri = T qi • Every distance d(ri,S) = 0 • Least Squares Energy minimization on error function E(p) Avneesh Sud, COMP258

22. Example: Localization Localization of 2 range images of a phantom head Avneesh Sud, COMP258

23. Curless & Levoy • Voxelize space, compute a cumulative signed distance function D(x) and cumulative distance function W(x) for each voxel • Extract isosurface corresponding to D(x)=0 Avneesh Sud, COMP258

24. Curless & Levoy: Functions Avneesh Sud, COMP258

25. Curless & Levoy : Functions Avneesh Sud, COMP258

26. Curless and Levoy: Algorithm • Voxelize space, initialize to “unseen.” • Tesselate scanned image, weighting each vertex by a confidence. • Cast ray from sensor through each voxel near the surface, updating distance function value for voxel corners. • Follow lines of sight back from the observed surface and mark corresponding voxels as empty (space carving). • Perform isosurface extraction at zero-crossing of signed distance function. • Post-filter the mesh to smooth it near “hole-filled” surfaces. Avneesh Sud, COMP258

27. Curliss & Levoy: Sampling Avneesh Sud, COMP258

28. Curliss & Levoy: Holes Avneesh Sud, COMP258

29. Curless & Levoy : Example Avneesh Sud, COMP258

30. Curless & Levoy : Example Avneesh Sud, COMP258

31. Bibliography • Turk, Greg and Mark Levoy “Zippered Polygon Meshes from Range Images” SIGGRAPH 1994 • Besl, Paul J and Neil D McKay “A Method for Registration of 3-D Shapes” PAMI, 14(2):239-256 1992 • Champleboux, G. , S. Lavalle, R. Szeleski and L. Brunie “From accurate imaging sensor calibration to accurate model-based 3-D object localization CVPR 1992 • Chen, Yang and Gerard Medioni “Surface Description of Objects from Multiple Range Images” IUW 1994 Avneesh Sud, COMP258

32. Bibliography • Hoppe, Hughes, Tony DeRose, Tom Duchamp, John McDonald, Werner Stuetzle “Surface Reconstruction from Unorganised Points” SIGGRAPH 1992 • Curless, Brian and Marc Levoy “A Volumetric Method for Building Complex Models from Range Images” SIGGRAPH 1996 • Reed, Michael K., “Solid Model Acquisition from Range Images” Phd thesis, Columbia Univ., 1998 Avneesh Sud, COMP258