1 / 19

Ai-Mei Huang , Student Member, IEEE, and Truong Nguyen, Fellow, IEEE

Correlation-Based Motion Vector Processing With Adaptive Interpolation Scheme for Motion-Compensated Frame Interpolation. Ai-Mei Huang , Student Member, IEEE, and Truong Nguyen, Fellow, IEEE. I. Introduction.

nile
Download Presentation

Ai-Mei Huang , Student Member, IEEE, and Truong Nguyen, Fellow, IEEE

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Correlation-Based Motion Vector Processing With Adaptive Interpolation Scheme for Motion-Compensated Frame Interpolation Ai-Mei Huang, Student Member, IEEE, and Truong Nguyen, Fellow, IEEE

  2. I. Introduction • The application of motion-compensated frame interpolation (MCFI) techniques to increase video frame rate at playback has gained significant for the last decade. • This is because MCFI improves temporal resolution by interpolating extra frames and can be used to reduce motion jerkiness for video applications.

  3. I. Introduction • MCFI requires motion information between two frames, which can be either re-estimated at the decoder or retrieved directly from the received bitstreams. Depending on available resources of the devices. • Unfortunately, the received MVs or re-estimated MVs simply using block matching algorithm (BMA) are often unreliable for frame interpolation. • Directly employing these MVs usually results in unpleasant artifacts such as blockiness , ghost artifacts and deformed structures in the interpolated frames.

  4. I. Introduction • In the literature, many motion estimation algorithms performed at the decoder have been proposed for MCFI in order to obtain true motion. • A hierarchical BMA ,where three different window sizes are used to search for true MVs. • By considering the motion distribution on object boundaries, image segmentation techniques are employed to further refine the estimated motion vector field (MVF).

  5. II. Challenges in motion-compensated frame interpolation • In MCFI, the interpolated frame, , is often obtained by one of the following two different methods: • The interpolated frame can be produced by motion compensating from and along the motion trajectory. If a block-based MCFI is used, holes and overlapped regions frequently appear in the interpolated frame.

  6. II. Challenges in MCFI • Simply takes the MVs of the co-located blocks and divides them by two to form forward and backward MVs. • This method can also be referred to as bidirectional MCFI approach. (Vx, Vy)

  7. II. A. Co-Loacated Motion Vectors • Even though these MVs may represent true motion for blocks in , they may not represent the motion of their co-located blocks in .

  8. II. B. Irregular Motion Vectors • MVF between two frames is supposed to be smooth, except at the motion boundaries and occlusion areas.

  9. II. C. Video Occlusions • Areas where new objects appear or existing objects disappear can be referred to as video occlusions. • The visual artifacts caused by occlusion cannot be removed completely even though we have the correct MVs for the moving object.

  10. III. Motion vector analysis for motion-compensated frame interpolation A. Motion Vector Classification • Let denote the MV of each 8*8 block, , we classify into three different reliability levels, unreliable due to high residual energy (L1) , unreliable due to low inter-MV correlation (L2), possibly unreliable (L3).

  11. III . A. Motion Vector Classification • In order to detect the irregular MVs that have low residual energy, we calculate the correlation index of each MV to all its available adjacent MVs. • d is usually higher than other areas if the local movement is relatively large. Reduce the sensitivity from the motion magnitude values, the correlation index is defined as the magnitude variance in the local neighborhood.

  12. III . A. Motion Vector Classification • unreliable due to high residual energy (L1) • unreliable due to low inter-MV correlation (L2) • possibly unreliable (L3).

  13. III . B. Macroblock Merging Map for Motion Vector Processing • We have suggested that unreliable MVs should be grouped into larger blocks for MV correction. • MVs of L1 and L2 are identified due to different reasons, they should not be merged together. • MVs are considered similar if their angular distance, , and Euclidian distance, d, are less than predefined thresholds, , and , respectively.

  14. IV. Correlation-based motion vector processingusing bidrectional prediction difference

  15. IV. Correlation-based motion vector processingusing bidrectional prediction difference • A. Motion Vector Selection • Absolute bidirectional prediction difference (ABPD)

  16. IV. Correlation-based motion vector processingusing bidrectional prediction difference • B. Motion Vector Averaging Based on MV Correlation

  17. Result

  18. Result

  19. Result

More Related