Structured Light

1. Structured Light --- some recent progress Bo Fu University of Kentucky

2. Paper List • 1. Structured Light 3D Scanning in the Presence of Global Illumination. (CVPR 11’) [systematic error introduced by ill lighting condition] Extension: A Practical Approach to 3D Scanning in the Presence of Inter-reflections, Subsurface Scattering and Defocus. (IJCV 12’) • 2. Implementing High Resolution Structured Light by Exploiting Projector Blur. (WACV 12’) [increase depth resolution] • 3. Vision Processing for Real-time 3D Data Acquisition Based on Coded Structured Light. (IEEE Trans. on Image Processing 2008) [reduce acquisition time]

3. Structured Light 3D Scanning in the Presence of Global Illumination Mohit Gupta, etc Carnegie Mellon University

4. Introduction • Motivation One important assumption of most structured light techniques does not always hold: scene points receive illumination only directly from the light source.

5. Introduction • Main idea Design patterns that modulate global illumination and prevent the errors at capture time itself. Min-SW XOR-02 Gray code

6. Errors due to Global Illumination Short range effect: sub-surface scattering, defocus Long range effect: inter-reflection, diffusion

7. 1. inter-reflection error

8. 1. inter-reflection error • solution

9. 2. sub-surface scattering error

10. 2. sub-surface scattering error • solution

11. Error formulation , only with global illumination : irradiances under pattern P : irradiances under inverse pattern Lid : direct component Lig : global component β: fraction of the global component under pattern P

12. Error formulation , with global illumination and projector defocus α: projector defocus fraction

13. Error formulation • Correct binarization: Note: without global illumination (Lg =0), defocus (α = 1), this condition automatically holds

14. Error formulation • Long range effect diffuse and specular inter-reflection Consequence: low frequency decode error. Since the low frequency pattern correspond to the higher-order bits, this results in a large error in the recovered shape.

15. Error formulation • Short-range effect sub-surface scattering and defocus Consequence: loss of depth resolution

16. Pattern for error prevention • How to design pattern to prevent both long range effect and short range effect? pattern with only high frequencies can prevent long range effect. pattern with only low frequencies can prevent short range effect. It is possible to design code with only high frequency patterns, while for short range effect, patterns with large minimum stripe-width can be designed.

17. Pattern for error prevention • 1. For long range effect Model the binarization process as a scene-dependent function from the set of binary projected patterns : Since low frequency pattern is not suitable for long range effect, we can decompose it into two high frequency patterns with logical XOR

18. Pattern for error prevention • 1. For long range effect Gray code XOR Base pattern

19. Pattern for error prevention • 1. For long range effect

20. Pattern for error prevention • 2. For short range effect Design codes with large minimum stripe-width (min-SW) well studied in combinatorial mathematics [1]. [1] binary gray codes with long bit runs. The electronic journal of combinatorics. 2003

21. Pattern for error prevention • 2. Ensemble of codes for general scenes using four codes optimized for different effects: XOR-04 and XOR-02 for long range effect Gray code with maximum min-SW and Gray code for short range effect Rule: if any two agree within a small threshold, that value is returned as true depth,.

22. Experiment • Please refer to the paper (IJCV preferred)

23. Implementing High Resolution Structured Light by Exploiting Projector Blur CamilloTaylor University of Pennsylvania

24. Introduction • Motivation 1. With standard structured light decoding schemes, one is limited by the resolution of the projector. The quantization of the projector ultimately limits the accuracy of the reconstruction.

25. Introduction • Motivation 2. Growing disparity between the resolution of the image sensor and the resolution of the projector systems. 1600*1200 VS 640*480

26. Introduction • Main idea By exploit the blur induced by the optics of the projector, subpixel correspondences between camera frame and projector frame can be established Major comparison: Li Zhang, Shree Nayar. Projection Defocus Analysis for Scene Capture and Image Display. (TOG 06’)

27. Approach The effective irradiance that a pixel in the projector contributes to a point in the scene is related to the displacement between the projection of that scene point on the projector frame and the center of the pixel. This falloff is modeled as a Gaussian I: observed scene radiance measured at a pixel in the camera f: BRDF at the scene point E: irradiance supplied to the corresponding scene point by the projector

28. Approach k: stripe index σ: width of the blur kernel at point in the scene δ: projection of scene point in the projector frame (with subpixel precision) k δ

29. Approach I0: scene irradiance due to ambient illumination (known) δ : floating point offset between -0.5 and 7.5

30. Experiment For more result, please refer to original paper

31. Vision Processing for Real-time 3D Data Acquisition Based on Coded Structured Light S. Y. Chen, etc City University of Hong Kong

32. Motivation • Conventional structured light system can not be applied to moving surfaces since multiple patterns must be projected.

33. Main idea • Grid solid pattern for 3d reconstruction with fast matching strategies.

34. Thank you