1 / 60

Computing Motion from Images

Computing Motion from Images. Chapter 9 of S&S plus otherwork. General topics. Low level change detection Region tracking or matching over time Interpretation of motion MPEG compression Interpretation of scene changes in video Understanding human activites.

lilian
Download Presentation

Computing Motion from Images

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Computing Motion from Images Chapter 9 of S&S plus otherwork. Stockman MSU Fall 2008

  2. General topics • Low level change detection • Region tracking or matching over time • Interpretation of motion • MPEG compression • Interpretation of scene changes in video • Understanding human activites Stockman MSU Fall 2008

  3. Motion important to human vision Stockman MSU Fall 2008

  4. What’s moving: different cases Stockman MSU Fall 2008

  5. Image subtraction Simple method to remove unchanging background from moving regions. Stockman MSU Fall 2008

  6. Change detection for surveillance Stockman MSU Fall 2008

  7. Change detection by image subtraction Stockman MSU Fall 2008

  8. What to do with regions of change? • Discard small regions • Discard regions of non interesting features • Keep track of regions with interesting features • Track in future frames from motion plus component features Stockman MSU Fall 2008

  9. Some effects of camera motion that can cause problems Stockman MSU Fall 2008

  10. Motion field Stockman MSU Fall 2008

  11. FOE and FOC Will return to use the FOE or FOC or detection of panning to determine what the camera is doing in video tapes. Stockman MSU Fall 2008

  12. Gaming using a camera to recognize the player’s motion Decathlete game Stockman MSU Fall 2008

  13. Decathlete game Cheap camera replaces usual mouse for input Running speed and jumping of the avatar is controlled by detected motion of the player’s hands. Stockman MSU Fall 2008

  14. Motion detection input device Jumping (hands) Running (hands) Stockman MSU Fall 2008

  15. Motion analysis controls hurdling event (console) • Top left shows video frame of player • Middle left shows motion vectors from multiple frames • Center shows jumping patterns Stockman MSU Fall 2008

  16. Related work • Motion sensed by crude cameras • Person dances/gestures in space • System maps movement into music • Creative environment? • Good exercise room? Stockman MSU Fall 2008

  17. Computing motion vectors from corresponding “points” High energy neighborhoods are used to define points for matching Stockman MSU Fall 2008

  18. Match points between frames Such large motions are unusual. Most systems track small motions. Stockman MSU Fall 2008

  19. Requirements for interest points Match small neighborhood to small neighborhood. The previous “scene” contains several highly textured neighborhoods. Stockman MSU Fall 2008

  20. Interest = minimum directional variance Used by Hans Moravec in his robot stereo vision system. Interest points were used for stereo matching. Stockman MSU Fall 2008

  21. Detecting interest points in I1 Stockman MSU Fall 2008

  22. Match points from I1 in I2 Stockman MSU Fall 2008

  23. Search for best match of point P1 in nearby window of I2 For both motion and stereo, we have some constraints on where to search for a matching interest point. Stockman MSU Fall 2008

  24. Motion vectors clustered to show 3 coherent regions All motion vectors are clustered into 3 groups of similar vectors showing motion of 3 independent objects. (Dina Eldin) Motion coherence: points of same object tend to move in the same way Stockman MSU Fall 2008

  25. Two frames of aerial imagery Video frame N and N+1 shows slight movement: most pixels are same, just in different locations. Stockman MSU Fall 2008

  26. Can code frame N+d with displacments relative to frame N • for each 16 x 16 block in the 2nd image • find a closely matching block in the 1st image • replace the 16x16 intensities by the location in the 1st image (dX, dY) • 256 bytes replaced by 2 bytes! • (If blocks differ too much, encode the differences to be added.) Stockman MSU Fall 2008

  27. Frame approximation Left is original video frame N+1. Right is set of best image blocks taken from frame N. (Work of Dina Eldin) Stockman MSU Fall 2008

  28. Best matching blocks between video frames N+1 to N (motion vectors) The bulk of the vectors show the true motion of the airplane taking the pictures. The long vectors are incorrect motion vectors, but they do work well for compression of image I2! Best matches from 2nd to first image shown as vectors overlaid on the 2nd image. (Work by Dina Eldin.) Stockman MSU Fall 2008

  29. Motion coherence provides redundancy for compression MPEG “motion compensation” represents motion of 16x16 pixels blocks, NOT objects Stockman MSU Fall 2008

  30. MPEG represents blocks that move by the motion vector Stockman MSU Fall 2008

  31. MPEG has ‘I’, ‘P’, and ‘B’ frames Stockman MSU Fall 2008

  32. Computing Image Flow Stockman MSU Fall 2008

  33. Stockman MSU Fall 2008

  34. Assumptions Stockman MSU Fall 2008

  35. IMAGE FLOW EQUATION 1 of 2 Stockman MSU Fall 2008

  36. Image flow equation 2 of 2 Stockman MSU Fall 2008

  37. Aperture problem Stockman MSU Fall 2008

  38. Solving flow by propagation of constraints Stockman MSU Fall 2008

  39. Info at corner constrains the flow along both edges Solve constraints using contraint propagation or differential equation with boundary conditions. Stockman MSU Fall 2008

  40. Tracking several objects Use assumptions of physics to compute multiple smooth paths. (work of Sethi and R. Jain) Stockman MSU Fall 2008

  41. Stockman MSU Fall 2008

  42. Tracking in images over time Stockman MSU Fall 2008

  43. General constraints from physics Stockman MSU Fall 2008

  44. Other possible constraints • Background statistics stable • Object color/texture/shape might change slowly over frames • Might have knowledge of objects under survielance • Objects appear/disappear at boundary of the frame Stockman MSU Fall 2008

  45. Stockman MSU Fall 2008

  46. Sethi-Jain algorithm Stockman MSU Fall 2008

  47. Stockman MSU Fall 2008

  48. Total smoothness of m paths Stockman MSU Fall 2008

  49. Greedy exchange algorithm Stockman MSU Fall 2008

  50. Example data structure Total smoothness for trajectories of Figure 9.14 Stockman MSU Fall 2008

More Related