Independent Motion Estimation

1. Independent Motion Estimation Luv Kohli COMP290-089 Multiple View Geometry May 7, 2003

2. Outline • The motion segmentation problem • Motivation • Background • Recursive RANSAC • More sophisticated algorithms • Results

3. Motion segmentation • The problem according to Phil Torr: how to detect a set of independently moving objects in the 2D projection of an otherwise rigid scene, given that the camera is moving in an arbitrary and unpredetermined manner

4. Motivation • Many practical applications for motion segmentation • Navigation • Image compression and representation • Video indexing • Recovery of 3D structure • Difficult to generalize for all types of scenes

5. Background • The methods thus far proposed for motion segmentation can be split into several categories • Methods for a stationary camera: do not distinguish several independently moving objects in the scene – can determine that there is motion but now how many objects

6. Background (2) • Methods based on image motion constraints • For example, compute velocities in the image using a local correspondence scheme and group similar velocities

7. Background (3) • Methods that require knowledge of the camera motion • Methods based on world constraints and epipolar geometry • An object undergoing a rigid transformation is equivalent to a camera moving in the opposite direction – effective motion can be described by epipolar geometry

8. Recursive RANSAC • RANSAC can be used to robustly estimate the fundamental matrix • Determines a highly probable solution to the problem and separates matches into a set of inliers and a set of outliers • Outliers may correspond to a second rigid motion in the scene

9. Recursive RANSAC (2) • Run RANSAC on set of putative matches to get inliers and outliers • Remove inliers from putative match set, and run RANSAC on outliers • This can be repeated multiple times, but generally it is difficult to fit data for more than 2 or 3 objects • Each matrix can then be improved through nonlinear minimization

10. Degeneracy • Data is degenerate if insufficient to determine a unique solution • This can cause many problems especially when there is a significant level of noise in the data • Phil Torr created the PLUNDER (Pick Least UNDEgenerate Randomly) algorithm for detecting degeneracy

11. Degeneracy (2) • The PLUNDER algorithm essentially determines which model (affinity, projectivity, etc.) a data set is consistent with • Fundamental matrices for different subsets of data can be estimated using different models • Phil Torr’s thesis goes into much more detail

12. Results (Rec. RANSAC)

13. Results (putative)

14. Results (segmentation)

15. Results (segmentation)

16. Results (outliers)

17. Results (epipolar)

18. Results (epipolar)

19. Results

20. Results (putative)

21. Results (segmentation)

22. Results (segmentation)

23. Results (outliers)

24. Results (epipolar)

25. Results (epipolar)

26. Results

27. Results (putative)

28. Results (segmentation)

29. Results (segmentation)

30. Results (outliers)

31. Results (epipolar)

32. Results (epipolar)

33. Results

34. Results (putative)

35. Results (segmentation)

36. Results (segmentation)

37. Results (outliers)

38. Results (epipolar)

39. Results (epipolar)

40. References • P.H.S. Torr and D.W. Murray. Outlier detection and motion segmentation. In P.S. Schenker, editor, Sensor Fusion VI, pages 432-443. SPIE volume 2059, 1993. Boston. • P.H.S. Torr. Motion Segmentation and Outlier Detection. Ph.D Thesis, Department of Engineering Science, University of Oxford, 1995.