Enhanced Frame Rate Up-Conversion Method for UHD Video

1. Enhanced Frame Rate Up-Conversion Methodfor UHD Video Tae-ShickWang;   Kang-Sun Choi;   Hyung-Seok Jang;   Morales, A.W.;   Sung-JeaKo; IEEE Transactions on Consumer Electronics, Vol. 56, No. 2, May 2010   

2. Introduction(1/2) • Motion blur caused by inherent characteristic of LCD • FRUC • MCFI consists of ME and MCI.

3. Introduction(2/2) • The motion vector field(MVF) is estimated between two successive frames. • MVF should represent the actual object motion • Several ME methods for MCFI have been proposed to find the true motion based on BMA • Multi-size block matching algorithm • Bi-directional Motion Compensate Interpolation(MCI)

4. Motion analysis for UHD video(1/2) • Enlarge SR can span regions which belong to another object but are more similar to the current block

5. Motion analysis for UHD video(2/2) • Full search, sort all MVs for each block in UHD • Increasing order of SAD • Search range(SR) should be kept as small as possible if a reliable MV is initially given • Large block size gives a more correct motion

6. Proposed ME algorithm • The blocks with similar motion are merged to an entity. • Introduce a segment-based ME algorithm • Divide the frame into several segments • Estimate the motion for each segment • Large region can be obtained more reliably fn: current frame

7. Block-based segmentation • Classify each block into three patterns: • edge, plane, and texture Segments are obtained by merging adjacent blocks with an identical pattern

8. Gradient calculation • For each block, the gradient is calculated by using the Sobel operator [14]. If large=>significant pixel Gradient image gradient [14] R. C. Gonzalez and R.E. Woods, Digital Image Processing, 2nd Edition, Prentice Hall, New Jersey, 2002.

9. Gradient Direction Histogram Largest number in the histogram # of significant pixels gy gx = r

10. Segmentation result MVF of fn-1 : k-th segment from fn

11. Segment selection • Find segments whose motion can be obtained accurately. • Temporal consistency of the segment • MV of the segment with high reliability • Large segment is more reliably, • The pattern of segment is plane or texture • has dominant MV pass through • Dominant MV : over 70% MVs are identical

12. Efficient true motion estimation(1/3) • Full search with low computation complexity • SSAD (subsample SAD) as matching criterion # of blocks in the segment => sub-sample rate σ

13. Efficient true motion estimation(2/3) • Determine the search range • Reliable block (RB) : SSAD < 5(B/σ)^2 • => the MV’s similarity to the actual motion of the segment => search range • Search range size LSR in the segment: ε =8

14. Efficient true motion estimation(3/3) • Three stage ME Refine dominant MV within LSR Refine the received MV (if average SSAD < 5(B/σ)^2) and within LSR No initial MV within max SR

15. Experimental result • Three UHD video sequence: • Toy and Calendar, Table Setting, and Tractor • Compare Ha’s [2] and Huang’s [10] methods [2] T. Ha, S. Lee, and J. Kim, "Motion compensated frame interpolation by new block-based motion estimation algorithm," IEEE Trans. Consumer Electron., vol. 50, no. 2, pp. 752-759, May 2004. [10] A.-M. Huang and T. Nguyen, "A multistage motion vector processing method for motion-compensated frame interpolation," IEEE Trans. Image Process., vol. 17, no. 5, pp. 694-708, May 2008.

16. Subjective comparison

17. Objective comparison • Improve quality for the interpolated frame by 2~3 dB • Reduce the computation load. RC: relative complexity

18. Conclusion • The block-based segmentation method was confirmed to produce meaningful segment information with low complexity. • The proposed method can also be successfully employed for various applications including • De-interlacing • View interpolation for multi-view video.