1 / 10

LUCAS KANADE FEATURE TRACKER a pyramidal implementation

LUCAS KANADE FEATURE TRACKER a pyramidal implementation. di Stefano Pacifico – A.A. 2005/2006. Feature tracking goal: Given I,J grayscaled 2D images, and a point u on I , find v = u+d on J such as u and v are similar.

evan
Download Presentation

LUCAS KANADE FEATURE TRACKER a pyramidal implementation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. LUCAS KANADE FEATURE TRACKERa pyramidal implementation di Stefano Pacifico – A.A. 2005/2006

  2. Feature tracking goal: Given I,J grayscaled 2D images, and a point u on I, find v = u+d on J such as u and v are similar. Optical flow evaluation to estimate motion field (Hyp: Lambertian surfaces, infinity ranged light sources, no photometric distortion) Feature based methods vs. Direct methods Introduction Stefano Pacifico - 2005/2006

  3. B.K.P. Horn and B.G. Schunck. Determining optical flow. AI Memo 572. Massachusetts Institue of Technology, 1980. This article is the first proposal of an iterative method to compute optical flow. Dense motion vector field, weak to noise. A first approach… Stefano Pacifico - 2005/2006

  4. Assumptions: Pixel luminosity is the same in both images for each pair of pixels. Motion perpendicular to local gradient direction is not discernible. Uniform motion for near pixlel clusters (to avoid aperture problem). Little displacemente of pixels between image I and J Using a window of 5x5 size -> 25 equations per pixel; only u and v asunknowns. Optical flow constraint Stefano Pacifico - 2005/2006

  5. A’A must be invertible. L1 and L2 eigenvalues of A’A, L1/L2 must be not too big. L1 and L2 not too little. Overdetermined equation system Equivalent to solving with least squares method Iterate with Newton – Raphson method Lucas Kanade Stefano Pacifico - 2005/2006

  6. build pyramid representations of I and JILm, JLm Initialization of pyramidal guess gLm for L = Lm to 0 uL =location of u on ILm = u/2^L Ix = derivative of IL wrt x Iy = derivative of IL wrt y G = spatial gradient matrix init iterative L-K (guess V0) for k = 1 to K or until optical flow < treshold Vk = iterative L-K loop estimation end for final optical flow at level L dL = Vk calculate guess for next level gLm-1 end for final optical flow vector V = g0 + d0 return v = u +d Relaxes iterative L-K assumption of little displacement Algorithm complexity O(n^2*N+n^3) n= #warp parmeters N = integration window size See Baker, Matthews papers. Pyramidal Lucas Kanade Stefano Pacifico - 2005/2006

  7. Problem and its actual solution are well studied; KLT widely used with almost optimal performances (see Barron et al. paper). Bare KLT less reliable than affine tracker, but quite faster. Possible integration with compositional algorithm (see Baker,Matthews papers) Feature trackers alone don’t satisfy the needs of today AI applications. KLT is integrated into more complex architectures (e.g. RANSAC, ML) Alternatives: Mean shift tracking and Histogram tracking (see Bradski paper), Kalman Filter, Block Matching, Warping Theory (see Brax paper). Pros, cons and alternatives Stefano Pacifico - 2005/2006

  8. Stanford KLT implementation Pyramidal implementation Affine model feature checking Can track features in non-consecutive frames http://www.ces.clemson.edu/~stb/klt/ OpenCV implementation Iterative implementation Pyramidal implementation Affine transformation estimation http://Opencv.sourceforge.net Both have means to detect features, and to declare them lost (see also Shi, Tomasi paper). Software and demo Stefano Pacifico - 2005/2006

  9. Bibliography • B.K.P. Horn and B.G.: Schunck. Determining optical flow.AI Memo 572. Massachusetts Institue of Technology, 1980. • Lucas, B. D.; Kanade, T.: An Iterative Image Registration Technique with an Application to Stereo Vision : Proc. of 7th International Joint Conference on Artificial Intelligence, (1981), pp. 674-679. • Shi, J.; Tomasi, C.: Good Features to Track. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, (1994), pp. 593-600. • Barron, J. L.; Fleet, D. J.: Beauchemin, S. S.; Burkitt, T. A.: Performance of optical flow techniques. In: International Journal of Computer Vision 12, Vol. 1 (1992-02), pp. 43-77. • Bouguet, J.-Y.: Pyramidal Implementation of the Lucas Kanade Feature Tracker. Part of OpenCV Documentation. Stefano Pacifico - 2005/2006

  10. Bibliography(2) • Bradski , G. R.: Computer vision face tracking for use in a perceptual userinterface. Intel Technology Journal, 2nd Quarter, 1998. • Baker, S. ; Matthews, I.: Lucas-Kanade 20 years on: A unifying framework: Part 1. Technical Report CMU-RI-TR-02-16, CMU Robotics Institute, (2002). • U. Knauer, T. Dammeier, and B. Meffert.:The Structure Of Road Traffic Scenes As Revealed By Unsupervised Analysis Of The Time Averaged Optical Flow. 17th International Conference on the Applications of Computer Science and Mathematics in Architecture and Civil Engineering • Brox T., Bruhn, A. Papenberg, N. Weickert J.:High Accuracy Optical Flow Estimation Based on a Theory for Warping In Proc. 8th European Conference on Computer Vision, 2004 • F. Pirri – Lecture notes A.A. 2005-2006 Stefano Pacifico - 2005/2006

More Related