1 / 26

Interactive BRDF Estimation for Mixed-Reality Applications

Interactive BRDF Estimation for Mixed-Reality Applications. Martin Knecht, Georg Tanzmeister, Christoph Traxler, Michael Wimmer Institute of Computer Graphics and Algorithms Vienna University of Technology. Motivation. Goal of our mixed reality framework

shlomo
Download Presentation

Interactive BRDF Estimation for Mixed-Reality Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Interactive BRDF Estimation for Mixed-Reality Applications Martin Knecht, Georg Tanzmeister, Christoph Traxler, Michael Wimmer Institute of Computer Graphics and Algorithms Vienna University of Technology

  2. Motivation • Goal of our mixed reality framework • Light interaction between real and virtual objects  Materials of real objects must be known Martin Knecht

  3. Problem Statement • Material estimation should not need any preprocessing  Use Kinect sensor and fish-eye lense camera for data acquisition • Should run at interactive framerates  Use GPU wherever possible • Should estimate Phong parameters Used in mixed reality framework Martin Knecht

  4. Estimation Pipeline • Similar to pipeline of Zheng et al. 2009 Martin Knecht

  5. Input Data for Estimation Martin Knecht

  6. Highlight Removal Martin Knecht

  7. Highlight Removal Martin Knecht

  8. Diffuse Reflectance Estimation Martin Knecht

  9. Diffuse Reflectance Estimation Inverse shading: Martin Knecht

  10. Clustering Martin Knecht

  11. Clustering 1/5 • Assumption: similar color  same material • Same material  same specular parameters • Clustering executed on the diffuse estimation • Novel hybrid CPU/GPU K-Means 1) Initialize cluster centers 2) Assign pixel to nearest cluster center 3) Calculate new cluster centers 4) Repeat steps 2 & 3 Martin Knecht

  12. Clustering 2/5 • 1) Initialize cluster centers • Random cluster centers • Exploit temporal coherence • Reuse of cluster centers of previous frame Martin Knecht

  13. Clustering 3/5 • 2) Assign pixel to nearest cluster center RGBC1 RGBC2 ... Cluster Shader ... Cluster 1 Cluster 2 Cluster 6 Bitmask 1 Bitmask 2 ... Martin Knecht

  14. Clustering 4/5 • 3) Calculate new cluster centers • 1x1 Mipmap is the average over all pixel • New cluster center: TRGBD 1x1 RGBD T* 1x1 Bitmask Martin Knecht

  15. Clustering 5/5 • 4) Repeat steps 2 & 3 • Repeat until no pixel changes cluster  Standard stopping criteria too conservative • Max. 20 iterations • Check variance change of distances Martin Knecht

  16. Clustering 5/5 Martin Knecht

  17. Specular Reflectance Estimation Martin Knecht

  18. Specular Reflectance Estimation • Done on a per cluster basis  same material • CPU based nonlinear function solver • Variables: • Specular parameter • Light positions • Evaluation of objective function done on GPU • Similar mipmap method used as for clustering Martin Knecht

  19. Specular Reflectance Estimation Martin Knecht

  20. Results - Estimation + Diffuse component Specular component Phong shaded image Martin Knecht

  21. Results – Timings • BRDF estimation runs at ~2.8 fps • Two tasks with major impact  K-Means clustering  Specular estimation < 0.5 % ~ 11 % ~ 88,5 % Martin Knecht

  22. Differential Instant Radiosity Martin Knecht

  23. Results – Mixed Reality Integration Martin Knecht

  24. Limitations • Kinect sensor does not work everywhere • Bright objects are discarded from estimation • Shadows are not considered • No estimation of optimal amount of clusters • No integration of data over time • Simplifications lower quality of estimation Martin Knecht

  25. Conclusion & Future work • BRDF estimation without any preprocessing • Hybrid CPU/GPU K-Means implementation • Runs at interactive framerates • Future work • Improve speed  specular estimation • Improve quality BRDF estimation • Exploit temporal coherence more often Martin Knecht

  26. Thank you for your attention! Supported by grand from the FFG-Austrian Research Promotion Agency under the Program “FIT-IT Visual Computing” Project Nr.: 820916

More Related