Optical Flow Computation in Computer Graphics and Image Processing

1. Optical flow (motion vector) computation Course: Computer Graphics and Image Processing Semester: Fall 2002 Presenter: Nilesh Ghubade (nileshg@temple.edu) Advisor: Dr Longin Jan Latecki Dept: Computer and Information Science, Temple University, Philadelphia, PA-19122

2. Motion Analysis Three groups of motion-related problems: • Motion detection • Registers any detected motion. • Single static camera. • Used for security purposes. • Moving object detection and location • Determination of object trajectory. • Static camera, moving objects OR Moving camera, static objects OR Both camera and objects moving. • Deriving 3D properties • Use of set of 2D projections acquired at different time instants of object motion.

3. Object motion assumptions Cmax * dt • Maximum velocity. • Small acceleration. • Common motion of object points. • Mutual correspondence. t2 t1 t0

4. Differential motion analysis • Simple subtraction of images acquired at different instants in time makes motion detection possible, assuming stationary camera position and constant illumination. • Difference image is a binary image  subtract two consecutive images. • Cumulative difference image: • Reveals motion direction. • Time related motion properties. • Slow motion and small object motion. Constructed from sequence of ‘n’ images taking first image as the reference image.

5. Example Motion in front of a security camera. Sobel filter edge detection.

6. Motion Detection- Sobel filter 10 frames/second 15 frames/second 25 frames/second 15 frames/second

7. Optical Flow • Optical Flow reflects the image changes due to motion during a time interval dt. • Optical flow field is the velocity field that represents the 3D motion of object points across a 2D image. • It should not be sensitive to illumination changes and motion of unimportant objects (e.g. shadows) • Exceptions: • Non-zero optical flow fixed sphere illuminated by a moving source. • Zero optical flow  smooth sphere under constant illumination, although there is rotational motion and true non-zero motion field.

8. Optical Flow (continued…) • Aim is to determine optical flow that corresponds with true motion field. • Necessary pre-condition of subsequent higher level motion processing  stationary or moving camera. • Provides tools to determine motion parameters, relative distances of objects in the image etc.. • Example: t2 t1

9. Assumptions Optical flow computation is based on two assumptions: • The observed brightness of any object point is constant over time. • Nearby points in the image plane move in a similar manner (the velocity smoothness constraint).

10. Optical flow computation The optical flow field represented in the form of Velocity vector: • Length of the vector determines the magnitude of velocity. • Direction of the vector determines the direction of motion. Global optical flow estimation— • Local constraints are propagated globally. • But errors also propagate across the solution. Local optical flow estimation— • Divide image into smaller regions. • But inefficient in the areas where spatial gradients change slowly  here use global method, neighboring image parts contribute.

11. Forms of motion

12. Representation Locate the position of a pixel (row,col) in the current image by computing shortest Euclidean distance with respect to 5-by-5 neighborhood in the next consecutive frame.

13. Experiments 3-by-3 neighborhood

14. Experiments (contd…) 5-by-5 neighborhood

15. Experiments (contd…)

16. Experiments (contd…)

17. Applications of optical flow • Object motion detection. • Action recognition. • Active vision or structure of motion – • Reconstruction of 3D object by computing depth information. • If you have distance (depth) maps, you can reconstruct surface of the object. • Facial expression recognition: reference http://athos.rutgers.edu/~decarlo/pubs/ijcv-face.pdf

18. Thank you