1 / 15

CSSE463: Image Recognition Day 30

CSSE463: Image Recognition Day 30. This week Today: motion vectors and tracking Friday: Project workday. First status report due. Please bring short printed copy to class: what you did what you didn’t and the technical challenges what you will do next week. Questions?.

zayit
Download Presentation

CSSE463: Image Recognition Day 30

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. CSSE463: Image Recognition Day 30 • This week • Today: motion vectors and tracking • Friday: Project workday. First status report due. Please bring short printed copy to class: • what you did • what you didn’t and the technical challenges • what you will do next week. • Questions?

  2. What is image flow? • Notice that we can take partial derivatives with respect to x, y, and time.

  3. Image flow equations • Goal: to find where each pixel in frame t moves in frame t+Dt • E.g. for 2 adjacent frames, Dt = 1 • That is, Dx, Dy are unknown • Assume: • Illumination of object doesn’t change • Distances of object from camera or lighting don’t change • Each small intensity neighborhood can be observed in consecutive frames: f(x,y,t)f(x+Dx, y+Dy, t+Dt) for some Dx, Dy (the correct motion vector). • Compute a Taylor-series expansion around a point in (x,y,t) coordinates. • Gives edge gradient and temporal gradient • Solve for (Dx, Dy) • See answers to first quiz question now

  4. Limitations • Assumptions don’t always hold in real-world images. • Doesn’t give a unique solution for flow • Sometimes motion is ambiguous • Look at last question on last quiz. • “Live demo” • http://www.cs.mcgill.ca/~chundt/coherence/

  5. Aperture problem • When we only see a small part of the image, motion can be ambiguous • Lesson: By constraining ourselves to a small neighborhood, we lose the “big picture” • Solutions: • Assume pixels belong to same object and have same motion • Propagate constraints between pixels • Can still take many frames to converge • Doesn‘t work at occlusions (hidden parts of object) • Sensitive to outliers • Only track motion of “interesting points” Q1

  6. Interest point tracking • Idea: corners and other easily-identifiable points can be tracked, and their motion is unambiguous. • Technique for calculating a set of interest points: • Loop over image: • Calculate an interest score for a small neighborhood around each pixel • If the score is above threshold, then add it to a point set to be returned. Q2

  7. Stretch break • When is your group willing to present? Q3,4

  8. Interest scores • One solution: • Take minimum variance of those found in 4 directions: • (3x3 shown, but would use bigger window) • Another related texture-based operator is based on directions with significant edge gradient. • Could also detect corners using Kirsch operators (similar to other edge operators).

  9. Tracking interest points • Search for them directly using a template. • Choose one with highest correlation • Just need to search in small neighborhood if frame rate is high enough • Shapiro’s Alg. 9.3

  10. Review: Correlation • Just the dot product between the template and a neighborhood in the image. • Idea: high correlation when the template matches. • Problem: always high correlation when matching with bright region • Solution: Normalize each region by subtracting mean before taking dot product

  11. Matlab libraries exist to extract frames (still images) from video for processing.

  12. Computing trajectories • Problem: what if unlabeled moving points belong to multiple trajectories? • Idea: want to maximize smoothness, defined as change in direction and change of speed. • Given two points pt, pt+1, thought to be related, calculate vt = pt+1 – pt. Then: Average speed compared to geom. mean direction

  13. Computing trajectories • Then the “Greedy-Exchange” algorithm is used to choose which points belong to which trajectory. • Operates by creating trajectories and exchanging points if it would increase total smoothness. • Local optimum.

  14. Example: Tracking Pedestrians with Human Models • http://www.umiacs.umd.edu/users/hismail/Ghost_outline.htm • Thanks to Thomas Root for link

  15. Another approach • The Kalman filter is a probabilistic model that combines noisy measurements with the expected trajectory of the object. It works even with occlusion. • See http://www.cs.unc.edu/~welch/kalman/ for a start.

More Related