1 / 22

MRF-BASED TRUE MOTION ESTIMATION USING H.264 DECODING INFORMATION

Yung-Lin Huang, Yi- Nung Liu, and Shao-Yi Chien Media IC and System Lab Graduate Institute of Networking and Multimedia National Taiwan University Signal Processing Systems (SIPS), 2010 IEEE. MRF-BASED TRUE MOTION ESTIMATION USING H.264 DECODING INFORMATION. Outline. Introduction

finola
Download Presentation

MRF-BASED TRUE MOTION ESTIMATION USING H.264 DECODING INFORMATION

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Yung-Lin Huang, Yi-NungLiu, and Shao-Yi Chien Media IC and System Lab Graduate Institute of Networking and Multimedia National Taiwan University Signal Processing Systems (SIPS), 2010 IEEE MRF-BASED TRUE MOTION ESTIMATION USING H.264 DECODING INFORMATION

  2. Outline • Introduction • Markov Random Field • Motion Vector Analysis • Motion Vector Pre-processing • Predictor Selection • Simplified Belief Propagation • Experimental Results • Conclusion

  3. Introduction (1/4) • Instead of heuristic approaches, TMEcan be formulated as apixel-labeling problem:

  4. Introduction (2/4) • Markov Random Field : • Given an undirected graph G = (V, E) • A set of random variables(label)X = (Xv)v ∈ V • Markov properties:

  5. Introduction (3/4) • Assigning each pixel a label, can be justified in terms of maximum a-posterior estimationof a MRF model: posterior ∝ likelihood * prior • With negative log probabilities, where the max-product becomes a min-sum. • The max-product algorithm can be used to find an approximate minimum cost labeling of energy functions. Ed (the data term) & Es(the smoothness term)

  6. Introduction (4/4) • The cost energy function of a Markov Random Field model to estimate the optimal labels { lp } of corresponding pixels: • Ed: the data term that measures the penalty between the labels and the data • Es: the smoothness term that penalizes the coherencebetween labels • P: the set of all pixels • N: the 4-nearest neighbor pixels

  7. Motion Vector Analysis(1/3) • Optical flow datasets are used here because the ground truth (GT) MV maps are provided:

  8. Motion Vector Analysis(2/3) • Check the existence of true MV by similarity: • The existence of TMV: • W,H: the width, height of the test sequence • In Fig. 4. Both THxand THyare set to 1, and PSR ranges from 0 to 64.

  9. Motion Vector Analysis(3/3) • The ME strategy(FastFS, FS or EPZS) haslittle effect on the experimental results. • There are still MVs with true motion trajectoryin the H.264-coded MV field.

  10. Proposed Algorithm

  11. Motion Vector Pre-processing • In the proposed algorithm, the block size is fixedin each scale, so the MVs of variable block sizes must be splitand merged. • The block merging method takes not only the macroblocktypes (from H.264) but also neighboring MVs into consideration. • Although the global optimization might modify these bad MVs, the pre-processing costs less efforts.

  12. Proposed Algorithm

  13. Predictor Selection(1/2) • In Fig. 4, the probability that true MV exists is highwith enough PSR. • We choose PSR=32, when the block size is 16, the range of ±32 pixels ±2 blocks. • The strategy of predictor selection and the MRF model of the proposed algorithm are shown in Fig. 5(a). Nine predictors are selected.

  14. Predictor Selection(2/2)

  15. Proposed Algorithm

  16. Simplified Belief Propagation(1/3) • The multi-scale concept from [7],instead of pixel-based operation, 4x4 blockis taken as the smallest unit. • The belief propagation isoperated from the highest scale (16x16 block) tothelowest scale (4x4 block). • ft: video frame at time t

  17. Simplified Belief Propagation(2/3) • The basic concept of belief propagation is to perform message passingoperation iterativelyand approximate global minimum by local messages. • Each pixel requires O(k2)computation for full-searchcandidates. • The proposedalgorithm requires only O(k)computation with predictor selection [7] for each pixel.

  18. Simplified Belief Propagation(3/3) [7] Pedro F. Felzenszwalb and Daniel P. Huttenlocher, “Efficient belief propagation for early vision,” Int. J. Comput.Vision, vol. 70, no. 1, pp. 41–54, 2006. • Loopy Belief Propagation approach for MRF: • Messages with the truncated linear model: Time complexity: O(nk2T) n: the number of pixels k: the number of possible labels T:the number of iterations Time complexity: O(k) n: the number of pixels k: the number of possible labels T:the number of iterations

  19. Experimental Results • Frame Rate Up Conversion compare with: Bidirectional overlapped block motion estimation (OBME), MV field smoothing with median filter. • Proposed algorithm has higher PSNR aboutthe camera motion video (mobile calendar) because of the global MV field optimization. • OBME requires full search(FS) with an enlarged search range. • The proposed algorithm has relative lower computational complexity.

  20. Experimental Results • Motion Vector Field:

  21. Experimental Results • Motion Vector Field:

  22. Conclusion • In this paper, a MRF-based true motion estimation obtained from H.264/AVC scheme is proposed. • The MV field of H.264/AVC is optimizedusing belief propagation efficiently. • In the future works, more reusable decoding information and hardware implementation will be involved.

More Related