- 50 Views
- Uploaded on
- Presentation posted in: General

Efficient Reference Frame Selector for H.264

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Efficient Reference Frame Selector for H.264

Tien-Ying Kuo, Hsin-Ju Lu

IEEE CSVT 2008

Introduction

H.264 Inter-Coding

Proposed Method

Experimental Results

Conclusions

H.264 features for inter-frame coding

Variable block size motion compensation

Subpixel motion estimation

Multiple reference frame motion compensation

N - 5

Current frame N

N - 2

N - 1

Why do Multiple reference frame help predictions ?

Uncovered background

Non-integer pixel displacement

Lighting change

Camera shaking

Noises in the source signal

Noise effect

4

Sampling/noise effect of MobileCalendar

(N-1) Ref block

(N-2) Ref block

Original block

(N-2) Residual block

(N-1) Residual block

MSE = 32.4

MSE = 203.8

5

Drawback of MRF-ME

High computational complexity

In order to reduce complexity

Reduce search points

Continuous tracking technique

Early stop criteria

6

A simple and effective method of selecting proper reference frames in MRF-ME.

It enables working with any existing ME algorithms.

Experimental results demonstrate the effectiveness of proposed algorithm.

7

Variable block size

Macroblock partition : 16*16, 16*8, 8*16, 8*8

Submacroblock partition : 8*8, 8*4, 4*8, 4*4

16*16

16*8

8*8

8*16

8*8

8*4

4*8

4*4

8

Reference parameter REF

Signal to which frame referred

Coded only once for each submacroblock partition

All subblock smaller than 8*8 in the same subpartition must refer to the same frame

9

Rate-distortion cost of each possible partition

REF : Reference parameter

m( REF ) : The motion vector in the reference frame

p ( REF ) : The predicted motion vector from the neighbors

s : Original video signal

c : Coded video signal

R : Rate function of motion vectors

SA(T)D : SAD or SATD

10

Reference frame selector

Treating 8*8 block as a minimal unit

Making the selection using a mode 8*8 motion search

11

Flow chart

Input a MB and perform four 8*8 block motion search

on reference frame t-1

Variance of four MVs of 8*8 blocks ≦TH?

No

Yes

Enter 2nd stage

First Stage

Perform motion estimation

on reference frame t-1

Output results

12

Flow chart

i = 2

Perform four 8*8 block motion search of the MB on

the reference frame t-i

No

Is i ≧ N?

i + 1

Yes

Is ref-frame t-k reffered by

one of the four 8*8 blocks?

Yes

No

Second Stage

Drop the non-referred ref frame t-k

Drop the non-referred ref frame t-k

No

Is k ≧ N?

Yes

Perform motion estimation

on each valid reference frame

13

Output results

First stage

Disjoint 16*16 macroblock into four 8*8 blocks

Motion search on ref-frame t-1 and check threshold

4 MVs and

corresponding

minimal R-D cost

Reference frame t - 1

Current Frame t

14

Motion vectors

R-D cost

Check x and y components of V t-1

Less than a small threshold ?

Encoder terminate early by designating only the

previous frame as reference frame

Early stop criteria

15

The design of threshold depends upon the computational capacity of the encoder

Larger threshold, lower complexity, but worse coding efficiency

Determine upper bound of the coding efficiency and the worst case of complexity

16

Second stage

Motion search of mode 8*8 should be tested on all of the remaining reference frames

N : The maximal number of reference frames

17

For a given block index i, we let k be set as

The frame selector sets the ref-frame n-k as the valid, qualified reference frame

Block i has the best motion vector with the lowest cost,

by referring to reference frame n-k

18

Drop unqualified frames

N = 5

Frame t-4

Frame t-3

Frame t-2

Frame t-5

Current frame t

19

Verifying the effectiveness of frame selector

Measuring the hit rate by comparing retained frames with the actual frames used via exhaustive search method (hit rate = 88% ~ 95%)

20

Even with the high hit rate, we expect that frame selector can drop as many frames as possible.

The false alarm ranges from 13%~32%

21

Reference frame usage of the proposed frame selector for various video sequence

22

Why using mode 8*8 but not other modes in frame selection?

Hit rate

False alarm

Motion estimation time spending

23

Analysis of hit rate and false alarm using different modes on various sequence

24

The motion estimation time spending on a MB

Judging from the hit rate, falsealarm and complexity

8*8 is the best choice

25

Reference software : JM 9.2

Intel Pentium 4 3.0 GHz with 512MB RAM

Encoder parameters

26

Using Fast Full Search (FFS) and Fast Motion Estimation (FME) in JM 9.2

Comparing the results with a frame selection method of Li.[1]

[1] X. Li, E. Q. Li, and Y. K. Chen, “Fast multi-frame motion estimation algorithm with adaptive search strategies in H.264 ,”in Proc. IEEE Int.

Conf. Acoust., Speech, Signal Process., May 2004, vol.3, pp.369–372.

27

R-D curve comparison (Foreman)

28

R-D curve comparison (Mobile)

29

Using BDPSNR and BDBR to measure the performance difference between the methods

Calculates average PSNR and bitrate distance between two RD curves of two method, respectively.

30

Discussing the computational complexity

31

Speed up of SAD and SATD calculations

32

Number of MBs references in each reference frame (Mobile)

33

Number of MBs references in each reference frame (Foreman)

34

An efficient reference frame selector is proposed for the h.264 encoder to deal with the complexity issue pertaining to MRFME.

The experimental results demonstrate that the proposed algorithm can reduce significantly the complexity of ME at the encoder end, while keeping almost the same R-D performance as FFS.

35