1 / 30

Forward-Backward Correlation for Template-Based Tracking

Forward-Backward Correlation for Template-Based Tracking. Xiao Wang ECE Dept. Clemson University. Introduction. Object tracking: An important computer vision problem Security and surveillance Medical therapy Retail space instrumentation Video abstraction Traffic Management

deirdre
Download Presentation

Forward-Backward Correlation for Template-Based Tracking

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Forward-Backward Correlation for Template-Based Tracking Xiao Wang ECE Dept. Clemson University

  2. Introduction • Object tracking: An important computer vision problem • Security and surveillance • Medical therapy • Retail space instrumentation • Video abstraction • Traffic Management • Video editing • Template-Based Tracking • A classic technique • Idea of template-based tracker

  3. Related Work • Jepson et al. Robust Online Appearance Models for Visual Tracking, CVPR 2001 • Ho et al. Visual Tracking Using Learned Subspaces, CVPR 2004 • Davis et al. Tracking Rigid Motion using a Compact-Structure Constraint, ICCV 1999 • Avidan et al. Ensemble Tracking, CVPR 2005

  4. Backward Correlation Module Gradient Module Update Template Overview of the Approach Forward Correlation Module Textured Background? Next Frame No Yes

  5. Template-Based Tracking • Template Selection: first frame vs. previous frame • Motion Model: • Similarity transformation scaling displacement

  6. Template-Based Tracking • Cross Correlation: SSD reference image displacement search image

  7. Template-Based Tracking • Similarity measure: s(Δx, Δy) • Correlation Coefficient: c(Δx, Δy) Mean of template Mean of image region

  8. Forward Correlation • Forward Correlation: • Reference frame: previous frame • Goal: find transformation vector (dx, dy, α) • Approach: cross-correlation Put into correlation coefficient framework • Template Update:

  9. Forward Correlation Out-of-plane rotation • Drifting Problem: • Forward correlation approximates rotation with translation. • Forward correlation does not check the reliability of the template. • We need a mechanism to question the assumption of forward correlation. Previous frame Current Frame

  10. Backward Correlation • Consider our problem as motion segmentation • Goal of motion segmentation • Why is motion segmentation of video sequences difficult? • Under-constrained • Occlusion & Disocclusion • Image noise • A two-step procedure: • Determine the motion vectors associate with each pixel or feature point. • Group pixels or feature points that perform common motion.

  11. Backward Correlation • Kanade-Lucas-Tomasi (KLT) feature tracker • Idea: minimize the dissimilarity of feature windows in two images • Assumption: mutual correspondence

  12. Backward Correlation • Now consider the dissimilarity under the template window. • Decompose the template window into 2 partitions: foreground background • Rewrite dissimilarity as: low high

  13. Backward Correlation • Background is moving at a different velocity than the foreground. • Foreground pixels have similar velocity and generate low SSD error. • Correlation between background pixels using foreground velocity generates high SSD error. • Goal: group foreground pixels which are moving at similar velocities Reference frame I(x) Difference image D(x)=[I(x)-J(x+d)]2 Current image J(x)

  14. Backward Correlation • Formulations for backward correlation Set of template candidates Correlation coefficient (likelihood)

  15. Untextured Backgrounds • Limitation of backward correlation: • Fails if background has little texture. • Why? --- Examine the assumption. • Backward correlation has no reason to prefer the foreground to the background which is untextured. low Also low if untextured

  16. Untextured Backgrounds • Likelihood of backward correlation: textured vs. untextured Foreground Template containing background pixels Textured background Untextured background

  17. Gradient Module • Motivation: • Seek a module focusing on the boundary of the target being tracked. • An edge-based segmentation problem. • Prior information: an ellipse model. • Gradient Module: Unit vector normal at pixel i Intensity gradient

  18. Combining Modules • Gradient module and backward correlation module have orthogonal failure modes. • Textured or Untextured? • Use sum of the gradient magnitude of the neighborhood region. • Combination of forward correlation module and backward correlation module is straightforward. • Combination of forward correlation module and gradient module requires the normalization of the matching scores.

  19. Combining Modules • Normalize the matching score (likelihood): • Finial state is decided by:

  20. Adaptive Scale • Vary the scale by ± 10 percent during search process. • Filter the result to avoid oversensitive scale adaptation. • Comaniciu et al. Kernel-based object tracking,TPAMI 2003 Size of the best state given by the alg. Size of the object in the previous frame

  21. Experimental Results:Cluttered Background • Traditional template-based tracker slides off target:

  22. Experimental Results:Cluttered Background • Our algorithm remains locked onto target:

  23. Experimental Results:Cluttered Background • Tracking error plot: Our algorithm (blue, solid) vs. traditional template-based tracker (red, dashed) Error in x direction Error in y direction

  24. Experimental Results:Untextured Background • Tracking results of traditional template-based tracker:

  25. Experimental Results:Untextured Background • Tracking results of our algorithm:

  26. Experimental Results:Occlusion

  27. Experimental Results:Tracking a vehicle

  28. Conclusion • Presented an extension to template-based tracking. • Achieved robustness to out-of-plane rotation. • Effective tracking in both textured and untextured environment. • Remaining challenges: • Robustness when scale changes. • Use motion discontinuities to improve performance. • Analysis of parameter sensitivity for untextured backgrounds.

  29. Questions?

  30. Thank You!

More Related