310 likes | 550 Views
Forward-Backward Correlation for Template-Based Tracking. Xiao Wang ECE Dept. Clemson University. Introduction. Object tracking: An important computer vision problem Security and surveillance Medical therapy Retail space instrumentation Video abstraction Traffic Management
E N D
Forward-Backward Correlation for Template-Based Tracking Xiao Wang ECE Dept. Clemson University
Introduction • Object tracking: An important computer vision problem • Security and surveillance • Medical therapy • Retail space instrumentation • Video abstraction • Traffic Management • Video editing • Template-Based Tracking • A classic technique • Idea of template-based tracker
Related Work • Jepson et al. Robust Online Appearance Models for Visual Tracking, CVPR 2001 • Ho et al. Visual Tracking Using Learned Subspaces, CVPR 2004 • Davis et al. Tracking Rigid Motion using a Compact-Structure Constraint, ICCV 1999 • Avidan et al. Ensemble Tracking, CVPR 2005
Backward Correlation Module Gradient Module Update Template Overview of the Approach Forward Correlation Module Textured Background? Next Frame No Yes
Template-Based Tracking • Template Selection: first frame vs. previous frame • Motion Model: • Similarity transformation scaling displacement
Template-Based Tracking • Cross Correlation: SSD reference image displacement search image
Template-Based Tracking • Similarity measure: s(Δx, Δy) • Correlation Coefficient: c(Δx, Δy) Mean of template Mean of image region
Forward Correlation • Forward Correlation: • Reference frame: previous frame • Goal: find transformation vector (dx, dy, α) • Approach: cross-correlation Put into correlation coefficient framework • Template Update:
Forward Correlation Out-of-plane rotation • Drifting Problem: • Forward correlation approximates rotation with translation. • Forward correlation does not check the reliability of the template. • We need a mechanism to question the assumption of forward correlation. Previous frame Current Frame
Backward Correlation • Consider our problem as motion segmentation • Goal of motion segmentation • Why is motion segmentation of video sequences difficult? • Under-constrained • Occlusion & Disocclusion • Image noise • A two-step procedure: • Determine the motion vectors associate with each pixel or feature point. • Group pixels or feature points that perform common motion.
Backward Correlation • Kanade-Lucas-Tomasi (KLT) feature tracker • Idea: minimize the dissimilarity of feature windows in two images • Assumption: mutual correspondence
Backward Correlation • Now consider the dissimilarity under the template window. • Decompose the template window into 2 partitions: foreground background • Rewrite dissimilarity as: low high
Backward Correlation • Background is moving at a different velocity than the foreground. • Foreground pixels have similar velocity and generate low SSD error. • Correlation between background pixels using foreground velocity generates high SSD error. • Goal: group foreground pixels which are moving at similar velocities Reference frame I(x) Difference image D(x)=[I(x)-J(x+d)]2 Current image J(x)
Backward Correlation • Formulations for backward correlation Set of template candidates Correlation coefficient (likelihood)
Untextured Backgrounds • Limitation of backward correlation: • Fails if background has little texture. • Why? --- Examine the assumption. • Backward correlation has no reason to prefer the foreground to the background which is untextured. low Also low if untextured
Untextured Backgrounds • Likelihood of backward correlation: textured vs. untextured Foreground Template containing background pixels Textured background Untextured background
Gradient Module • Motivation: • Seek a module focusing on the boundary of the target being tracked. • An edge-based segmentation problem. • Prior information: an ellipse model. • Gradient Module: Unit vector normal at pixel i Intensity gradient
Combining Modules • Gradient module and backward correlation module have orthogonal failure modes. • Textured or Untextured? • Use sum of the gradient magnitude of the neighborhood region. • Combination of forward correlation module and backward correlation module is straightforward. • Combination of forward correlation module and gradient module requires the normalization of the matching scores.
Combining Modules • Normalize the matching score (likelihood): • Finial state is decided by:
Adaptive Scale • Vary the scale by ± 10 percent during search process. • Filter the result to avoid oversensitive scale adaptation. • Comaniciu et al. Kernel-based object tracking,TPAMI 2003 Size of the best state given by the alg. Size of the object in the previous frame
Experimental Results:Cluttered Background • Traditional template-based tracker slides off target:
Experimental Results:Cluttered Background • Our algorithm remains locked onto target:
Experimental Results:Cluttered Background • Tracking error plot: Our algorithm (blue, solid) vs. traditional template-based tracker (red, dashed) Error in x direction Error in y direction
Experimental Results:Untextured Background • Tracking results of traditional template-based tracker:
Experimental Results:Untextured Background • Tracking results of our algorithm:
Conclusion • Presented an extension to template-based tracking. • Achieved robustness to out-of-plane rotation. • Effective tracking in both textured and untextured environment. • Remaining challenges: • Robustness when scale changes. • Use motion discontinuities to improve performance. • Analysis of parameter sensitivity for untextured backgrounds.