Context-based Surface Completion

1. Context-based Surface Completion Andrei Sharf, Marc Alexa, Daniel Cohen-Or

2. Introduction Holes in models: • Imperfect range scanned data: • Complex objects with non-visible regions. • Misalignment of multiple-views depth image scans. • Material reflections. • Surface editing operations.

3. Motivation Smooth filling is sufficient: • Small holes. • Smooth surfaces. Smooth filling is insufficient : • Surfaces containing fine geometric detail. • Topology of hole is more complex than a disk.

4. Context-based Completion Complete the missing region with patches that conform with its context Smooth Context-based

5. Method Import patches with matching context from the surface itself : • Analyze surface characteristics. • Find best matching patch. • Fit imported patch to boundary.

6. Related Work • Example-based image completion: [Drori et al. 2003; Jia and tang 2003; Sun et al. 2003;] • Texture synthesis: [Efros and Leung 1999; Efros and Freeman 2001;Wei and Levoy 2000; Ying et al. 2001;] • Smooth surface completion: [Curless and Levoy 1996; Davis et al. 2002; Ilic and Fua 2003; Verdera et al. 2003; Liepa 2003;] • Model-based surface reconstruction [Savchenko et al. 2002] • Curve analogies[Hertzmann et al. 2002] Drori et al. 03 Wei and Levoi 00 Liepa 03 Hertzmann et al. 02

7. Moving to 3D Surfaces Images have a regular spatial structure domain. Problems in 3D: • Topology and geometry of missing region. • Fitting a patch to the boundary of the missing region. • Definition of similarity of shapes. • Definition of a surface patch.

8. Algorithm • Given an incomplete shape • Create initial spatial subdivision • For each cell • Compute a local shape representation. • Compute a shape signature. • For each empty cell: • Find matching nonempty cell ω’. • Copy patch of ω’ into ω. • Subdivide cells and repeat • Until completed region matches its neighborhood.

9. Algorithm • Given an incomplete shape • Create initial spatial subdivision • For each cell • Compute a local shape representation. • Compute a shape signature. • For each empty cell: • Find matching nonempty cell ω’. • Copy patch of ω’ into ω. • Subdivide cells and repeat • Until completed region matches its neighborhood.

10. Shape Representation • Shape sampled point-set • Range scan output • Easy to merge • Octree hierarchy on top of the point set • Implicit surface approximation by fitting polynomials [Ohtake et al. 2003]: • General quadric: • Bivariate quadratic: • Edge or corner fitting

11. Shape Analysis A local signature of a cell consists: • Implicit shape characteristics: • Signed distance • Normal variation • Detail amount: • Depth in octree

12. Algorithm • Given an incomplete shape • Create initial spatial subdivision • For each cell • Compute a local shape representation. • Compute a shape signature. • For each empty cell: • Find matching nonempty cell ω’. • Copy patch of ω’ into ω. • Subdivide cells and repeat • Until completed region matches its neighborhood.

13. Missing Region Automatic identification: • Intersect shape approximation with cells. • Empty cell that intersects surface is part of missing region. Empty cell definition can alter due to refinement of missing region.  

14. Matching Find most similar non-empty cell: Distance metrics: • dc : Signature distance inside cells. • da: Signature distance of adjacent cells. • dl : Difference in amount of detail.

15. Matching-Candidate Set • Non-empty cells of the same size • Symmetry rotations of cells(/2, , ...) • Rotation of all point-set with some angle (/4, /3, ...)

16. Transferring Points ICP: • Copy points into empty cell. • Find closest point correspondence • Align points rigidly • Align points non-rigidly using polynomial form.

17. Algorithm • Given an incomplete shape • Create initial spatial subdivision • For each cell • Compute a local shape representation. • Compute a shape signature. • For each empty cell: • Find matching nonempty cell ω’. • Copy patch of ω’ into ω. • Subdivide cells and repeat • Until completed region matches its neighborhood.

18. David’s Hair Original Down-sampled Smooth completion Context-based completion

19. Completion Process Original Final result Initial approximation

20. Manual Editing of Bunny Model

21. Manual Editing of Knot Model

22. Scan of “Youth” Statue (Rear View) Original Smooth Result

23. Scan of Human Bone Original Smooth Result

24. Limitations: Semantics

25. Sample Rate vs. Detail Frequency

26. Summary A fully automatic method to complete a missing region in a surface from its context. • Completed patches geometrically conform with neighborhood. • Incremental scale-space framework for finer approximation of the unknown region.

27. Future Work • Explore with other spatial hierarchies. • Enlarge the search space of examples by building a class based example set. • Couple method with image completion methods for texture completion. • Rotation and translation invariant signature

28. Acknowledgments: • David Levin and Olga Sorkine • Israel science foundation; Israeli ministry of science • Digital Michelangelo project 3D model repository; Stanford 3D scanning repository; Darmstadt university of technology; Imager computer graphics laboratory of the university of British Columbia

29. Thank you