A New Diamond Search Algorithm for Fast Block-Matching Motion Estimation

1. A New Diamond Search Algorithm for Fast Block-Matching Motion Estimation Shan Zhu and Kai-Kuang Ma IEEE TRANSACTIONS ON IMAGE PROCESSION, VOL. 9, NO. 2, FEB 2000 First Appearance at ICICS 97’ International Conference on, Information, Communication and Single Processing MPEG-4 video verification mode. Ver 14.0

2. Motion Estimation (22,20) (32,16) (-10,4) Current frame Referenced frame Macro block 16*16 31*31

3. TSS (Three step search algorithm) -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 Proc. NTC81, pp. C9.6.1-9.6.5, New Orleans, Lam Nov./Dec. 1981.

4. NTSS(new 3-step search) algorithm Decision 1: minimum at the search window center ? Decision 2: minimum at one neighbor of center ? -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 1st step of NTSS 17 checking points Decision 1 Decision 2 F T F T MV=0 2nd step of NTSS 3 or 5 checking points 2nd and 3rd step of NTSS (same as in TSS) IEEE Transation On Video Technology And Circuit System, Aug, 1994 (Express Letter)

5. 4SS(Four Step Search Algorithm) -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 IEEE Transation On Video Technology And Circuit System, June, 1996

6. BBGDS(block-based gradient descent search algorithm) IEEE Transation On Video Technology And Circuit System, Aug, 1996

7. Observation • The error surface is usually not monotonic, multiple local minimum points generally exist in the search window especially for those image sequences with large motion content. • BBGD trapped into a local minimum • TSS is most likely to mislead the search path to a wrong direction and hence misses the optimum point.

8. Observation • The distribution of the global minimum points in real-world video sequences is centered at the position of zero motion. • NTSS loses the regularity and simplicity of TSS • 4SS use a moderate search pattern • Both NTSS and 4SS utilize the overlapping of checking points between adjacent search steps to reduce the computational complexity further. • NTSS, 4SS still requires to test 17 checking points for a stationary block.

9. Observation

10. Observation • Motion vector distribution probabilities within the search window. • About 52.76%~98.70% of the motion vectors are enclosed in a circular support with a radium of 2 pels and centered on the position of zero motion. • The block displacement of real-world video sequences could be in any direction, but mainly in horizontal and vertical directions.

11. Diamond Search Algorithm • To employ two search patterns Large Diamond Search Pattern (LDSP) Small Diamond Search Pattern (SDSP)

12. Step 1 • The initial LDSP is centered at the origin of the search window, and the 9 checking points of LDSP are tested. If the minimum block distortion (MBD) point calculated is located at the center position, go to Step3; otherwise, go to Step 2.

13. Three cases of checking-point overlapping in LDSP Case1: the corner point LDSP → LDSP Initial Case2: the edge point LDSP → LDSP Case3: the center point LDSP → SDSP

14. Step 2 • The MBD point found in the previous search step is re-positioned as the center point to form a new LDSP. If the new MBD point obtained is located at the center position, go to Step3; otherwise, recursively repeat this step.

15. Step 3 • Switch the search pattern from LDSP to SDSP. The MBD point found in this step is the final solution of the motion vector which points to the vest matching block.

16. Search path example 9+5+3+3+4 = 24 24 search points in total Leads to the motion vector (-4,-2) in five search steps – four times LDSP and one time SDSP at the final step.

17. Comments on DS Algorithm Implementation • The checking points outside the search window are truncated. • DS algorithm doesn’t restrict the number of search steps essentially. • Search looping situation • Use Simple Tie-break algorithm the convergence of DS. • The checking points are partially overlapped between adjacent steps.

18. Simulation Results NTSS NTSS DS 4SS DS FS 4SS FS MSE comparison of DS, 4SS, NTSS, and FS for “Caltrain” sequence when (a) Frame distance = 1 and (b) frame distance = 2

19. Simulation Results NTSS NTSS 4SS 4SS DS DS Comparison of the average number of search points applying DS, 4SS, and NTSS to “Caltrain” sequence individually when (a) Frame distance = 1 and (b) frame distance = 2

20. Difference in motion content • For the image sequence with small-motion content, such as “talking-head” sequences (e.g., “Claire”), DS, 4SS, BBGDS and NTSS algorithms achieve close MSE performance as expected. • For moderate to large motion image sequences, DS, 4SS and NTSS maintain close performance while the BBGDS degrades distinctly. • Therefore, the BBGDS’ MSE performance is not stable and highly depends on the motion content, although BBGDS constantly demands the smallest number of search points.

21. Why does the DS algorithm work so well? • 2 pels in horizontal and vertical directions and 1 pels in each diagonal direction. • For large motion blocks • Not so easy to be trapped into a local minimum point as BBGDS would do and can find the global minimum point using relatively few search points. • For quasistationary or stationary blocks • Will be fewer than that of the 4SS. • The compact shape of the search patterns increases the possibility of finding the global minimum point located inside the search pattern.

22. Hybrid Block Matching with PCDB List • NTSS + DS + BBGD • Real world image sequences has wide range of motion content. That is Large, moderate, and small motion. • Observation on PCDB List • From fast motion down to small motions • Relationship among neighbor blocks?