1 / 17

High Quality Novel View Synthesis Based on Low Resolution Depth Image and High Resolution Color Image

This paper presents a hybrid camera system that combines a high-resolution color camera with a low-resolution depth camera to generate high-quality high-resolution depth images. The proposed depth processing algorithm includes depth acquisition, preliminary depth image generation, depth map refinement, and virtual view synthesis. Experimental results show the effectiveness of the system.

lhuff
Download Presentation

High Quality Novel View Synthesis Based on Low Resolution Depth Image and High Resolution Color Image

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. ICPR/WDIA-2012 High Quality Novel View Synthesis Based on Low Resolution Depth Image and High Resolution Color Image Jui-Chiu Chiang, Zheng-Feng Liu, and Wen-Nung Lie National Chung Cheng University Taiwan

  2. Outline • Introduction • Our hybrid camera system • The proposed depth processing algorithm • Experimental results • Conclusion

  3. Color + Depth in 3DTV/FTV • Depth image, in combination with DIBR (Depth Image-Based Rendering) technique, is important in multi-view 3DTV/FTV applications. Depth image Colorimage DIBR view9 view8 view7 view6 view5 view4 view3 view2 view1

  4. Depth acquisition • From stereo matching via disparity estimation • Unreliable for textureless regions • Computationally intensive • From active photo-electrical sensing • Lower resolution • Higher acquisition speed and accuracy Kinect SR4000 ToF

  5. Hybrid camera systems • High-resolution color cameras + Low-resolution depth camera (mis-alignment between their optical axis) • [12]: Use warped data of depth camera to refine ROI disparity of the dual-eye camera • Proposed: one color image is used to refine the warped (to both sides) and up-sampled depth images

  6. Processing diagram of our system

  7. Generation of preliminary high-resolution depth image • Purpose: to generate high-resolution depth image at the left and right sides Calibrated camera parameters

  8. Pre-processing of the depth map • 33 median filter • Low resolution to high resolution depth warping • Forward warping with a block size of 55 pixels • Post-processing by morphological operation • Morphological opening

  9. Generation of high quality high-resolution depth image • Color-image-guided depth map refinement

  10. Binarizing the warped depth image to have foreground/background classification • Color segmentation by mean-shift algorithm • Label each segment as FG or BG • Refine the corresponding depth values according to the segment categorization • Bilateral filtering is performed for the edge parts of the modified depth image

  11. Experiments – system configurations • Color image: 640x480 • Depth image: 176x144 (SR4000) • Intel Core 2 Duo Q6600 2.33 GHz, and DDR2 800 4GB

  12. Preliminary high-resolution depth image After depth warping Traditional depth warping After morphological operation

  13. High-quality high-resolution depth image After correction Mean-shift Incorrect pixels Bilateral filtering

  14. A series of generated depth maps

  15. Virtual view synthesis based on various depth maps Hole filled Bilateral filtering on edges Bilateral filtering on edges + Gaussian Bilateral filtering

  16. Speed performance

  17. Conclusion • A hybrid camera system consisting of high-resolution color camera and low-resolution depth camera is proposed • A high-quality high-resolution depth images can be obtained for the left and right eyes – multi-view video plus depth (N=2 channels). • Future work • Combine Kinect device and stereo camera to get better depth image and cheaper cost

More Related