1 / 21

Depth Image-Based Temporal Error Concealment for 3-D Video Transmission

Depth Image-Based Temporal Error Concealment for 3-D Video Transmission. Yunqiang Liu, Jin Wang, and Huanhuan Zhang IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 20, NO. 4, APRIL 2010 Professor: Jar - Ferr Yang Presenter: Jen - Hung Yeh. Outline. Introduction

anila
Download Presentation

Depth Image-Based Temporal Error Concealment for 3-D Video Transmission

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Depth Image-Based Temporal Error Concealment for3-D Video Transmission Yunqiang Liu, Jin Wang, and Huanhuan Zhang IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 20, NO. 4, APRIL 2010 Professor: Jar - Ferr Yang Presenter: Jen - Hung Yeh

  2. Outline • Introduction • Depth Image-Based Temporal Error Concealment • A. Error Concealment for Depth Stream • B. Error Concealment for 2-D Video Stream • Homogeneous MB • Boundary MB • Simulation Results • Conclusion

  3. Introduction(1/2) • 3-D television (3DTV) system has recently received increasing attention and is believed to be the next logical development. • In DIBR-based 3-D video application, it requires transmitting 2-D video and its corresponding depth information. • In the video transmission over unreliable channels such as wireless or internet, the transmission errors, such as packet losses and bit errors, are inevitable.

  4. Introduction(2/2) • Therefore , there is a need for good error concealment (EC) algorithms to compensate the large drop in received video quality. • It explores the correlation between the 2-D video stream and the depth stream and proposes a novel temporal error concealment approach by taking advantage of the depth information.

  5. Depth Image-Based Temporal Error Concealment • A. Error Concealment for Depth Stream • B. Error Concealment for 2-D Video Stream

  6. Error Concealment for Depth Stream(1/2) • The MV from the corresponding 2-D frame can be taken as the recovered MV for the corrupted MB in the depth stream. • In case of the error occurring at the same location in the depth stream and the video stream, the DMVE algorithm is performed in the depth sequence to recover the lost MV.

  7. Error Concealment for Depth Stream(2/2) • The depth map does not contain any texture information; moreover, it cannot distinguish the different objects at the same distance with the camera, even if they have relative motion. • It is not appropriate to take directly the MV of depth map as the recovered MV for 2-D video.

  8. Error Concealment for 2-D Video Stream • In the following steps, we will onlyconsider the situation that the error occurs in the 2-D video. 8-category histogram for the depth value The lost MBs Belong to two categories & The difference between their average depth values>T1 NO YES Homogeneous MB Boundary MB

  9. Homogeneous MB Find MV candidates of the MBs around the The MV of the corresponding MB in depth map Filter these MVs candidate The referenced depth block should coincide with that of the . MVi <T2 YES NO The MV with minimum discard

  10. Homogeneous MB • (SAD represents sum of absolute differences) is calculated to measure the similarity between the two depth MBs • For 3-D video , we introduce a new matching criterion with consideration of the corresponding depth, which is defined as

  11. Boundary MB • It concludes four steps as follows: • Step 1: MB Segmentation: • Step 2: MV Candidates’ Initialization: • Step 3: MV Selection: • Step 4: Motion Compensation:

  12. Step 1: MB Segmentation: The lost MB The pixels with the large depth value NO YES Background • Foreground We take the foreground part as an example to describe the approach.

  13. Step 2: MV Candidates’ Initialization: Foreground Background • If the depth value of the grid is close to the average depth of the foreground area, both the MV of the grid in 2-D video and depth map are chosen as the MV candidate for the EC of foreground area. • And the collocated MV in the reference frame is also taken as the MV candidate. The neighbors of the foreground area Strong correlation with the lost foreground part

  14. Step 3: MV Selection: • Similar with the recovery of homogeneous MB, the matching criterion considers both the neighbor information and the corresponding depth. • The matching criterion is defined as:

  15. Step 3: MV Selection: • measures the temporal correlation of the 2-D video , which is defined as follows: • measures the similarity between the current depth MB and its referenced MB , which is defined as:

  16. Step 4: Motion Compensation: • The foreground area in the lost MB is recovered using the corresponding area in the reference frame according the recovered MV. The background area is recovered with a similar process.

  17. Simulation Results(1/4) depth map 2-D video

  18. Simulation Results(2/4) Fig. 2. Comparison of PSNR for each frame with 5% packet loss. (a) Little Girl. (b) Mobile Phone. (c) Orbi.

  19. Simulation Results(3/4) Homogeneous MB Boundary MB

  20. Simulation Results(4/4) Homogeneous MB Boundary MB

  21. Conclusion • A depth image-based error concealment approach exploits the correlations between the 2-D video and its corresponding depth map to recover the lost blocks. • This strategy makes it preserve the temporal and spatial consistency on both 2-D video and depth information.

More Related