1 / 104

Combining Photometric and Geometric Constraints

Combining Photometric and Geometric Constraints. Yael Moses IDC, Herzliya. Joint work with Ilan Shimshoni and Michael Lindenbaum, the Technion. Problem 1:. Recover the 3D shape of a general smooth surface from a set of calibrated images. Problem 2:.

hadley-fowler
Download Presentation

Combining Photometric and Geometric Constraints

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Combining Photometric and Geometric Constraints Yael Moses IDC, Herzliya Joint work with Ilan Shimshoni and Michael Lindenbaum, the Technion Y. Moses

  2. Problem 1: • Recover the 3D shape of a general smooth surface from a set of calibrated images Y. Moses

  3. Problem 2: Recover the 3D shape of a smooth bilaterally symmetric object from a single image. Y. Moses

  4. Shape Recovery • Geometry: Stereo • Photometry: • Shape from shading • Photometric stereo Main problems: Calibrations and Correspondence Y. Moses

  5. 3D Shape Recovery Photometry: • Shape from shading • Photometric stereo Geometry: • Stereo • Structure from motion Y. Moses

  6. Geometric Stereo • 2 different images • Known camera parameters • Known correspondence + + Y. Moses

  7. Photometric Stereo • 3D shape recovery: surface normals from two or more images taken from the same viewpoint Y. Moses

  8. Three images Photometric Stereo Solution: Matrix notation Y. Moses

  9. Photometric Stereo Main Limitation: Correspondence is obtained by a fixed viewpoint • 3D shape recovery (surface normals) Two or more images taken from the same viewpoint Y. Moses

  10. Overview • Combining photometric and geometric stereo: • Symmetric surface, single image • Non symmetric: 3 images • Mono-Geometric stereo • Mono-Photometric stereo • Experimental results. Y. Moses

  11. The input • Smooth featureless surface • Taken under different viewpoints • Illuminated by different light sources • The Problem: • Recover the 3D shape from a set of calibrated images Y. Moses

  12. n n  * • Perspective projection Assumptions • Three or more images • Given correspondence the normals can be computed (e.g., Lambertian, distant point light source …) * Y. Moses

  13. Our method Combines photometric and geometric stereo We make use of: • Given Correspondence: • Can compute a normal • Can compute the 3D point Y. Moses

  14. Basic Method Given Correspondence Y. Moses

  15. First Order Surface Approximation Y. Moses

  16. First Order Surface Approximation Y. Moses

  17. P() = (1 - )O1+ P, N(P() - P) = 0 First Order Surface Approximation Y. Moses

  18. First Order Surface Approximation Y. Moses

  19. New Correspondence Y. Moses

  20. New Surface Approximation Y. Moses

  21. Dense Correspondence Y. Moses

  22. Basic Propagation Y. Moses

  23. Basic Propagation Y. Moses

  24. Basic method: First Order • Given correspondence pi and L Pand n • Given P andn T • Given P, T andMi  a new correspondence qi Y. Moses

  25. Extensions • Using more than three images • Propagation: • Using multi-neighbours • Smart propagation • Second error approximation • Error correction: • Based on local continuity • Other assumptions on the surface Y. Moses

  26. Multi-neighbors Propagation Y. Moses

  27. Smart Propagation Y. Moses

  28. Second Order: a Sphere (P-P())(N+N)=0 N P() P N+N N Y. Moses

  29. Second Order Approximation Y. Moses

  30. Second Order Approximation Y. Moses

  31. Using more than three images • Reduce noise of the photometric stereo • Avoid shadowed pixels • Detect “bad pixels” • Noise • Shadows • Violation of assumptions on the surface Y. Moses

  32. Smart Propagation Y. Moses

  33. Error correction The compatibility of the local 3D shape can be used to correct errors of: • Correspondence • Camera parameters • Illumination parameters Y. Moses

  34. Score • Continuity: • Shape • Normals • Albedo • The consistency of 3D points locations and the computed normals: • General case: full triangulation • Local constraints Y. Moses

  35. Extensions • Using more than three images • Propagation: • Using multi-neighbours • Smart propagation • Second error approximation • Error correction: • Based on local continuity • Other assumptions on the surface Y. Moses

  36. Real Images • Camera calibration • Light calibration • Direction • Intensity • Ambient Y. Moses

  37. Error correction + multi-neighbor5 Images Y. Moses

  38. 5pp 5nn 5pn 3pp 3nn Y. Moses

  39. Y. Moses

  40. Y. Moses

  41. Y. Moses

  42. Y. Moses

  43. Y. Moses

  44. Detected Correspondence Y. Moses

  45. Error correction + multi-neighbord Multi-neighbors Basic scheme (3 images) Error correction no multi-neighbors Y. Moses

  46. Synthetic Images New Images Y. Moses

  47. Ground truth Basic scheme Multi-neighbors Error correction Sec a Y. Moses

  48. Ground truth Basic scheme Multi-neighbors Error correction Sec b Y. Moses

  49. Ground truth Basic scheme Multi-neighbors Error correction Sec c Y. Moses

  50. Ground truth Basic scheme Multi-neighbors Error correction Sec d Ground truth Basic scheme Multi-neighbors approx. Error correction Y. Moses

More Related