1 / 42

Putting Objects in Perspective

This paper explores the importance of considering object relationships, surface estimation, and viewpoint in image understanding. The authors propose a model that improves object detection and discusses potential applications and future work.

swider
Download Presentation

Putting Objects in Perspective

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Putting Objects in Perspective Derek Hoiem Alexei A. Efros Martial Hebert Carnegie Mellon University Robotics Institute

  2. Understanding an Image

  3. Today: Local and Independent

  4. What the Detector Sees

  5. Local Object Detection True Detection False Detections Missed Missed True Detections Local Detector: [Dalal-Triggs 2005]

  6. Work in Context • Image understanding in the 70’s Guzman (SEE) 1968 Hansen & Riseman (VISIONS) 1978 Barrow & Tenenbaum 1978 Yakimovsky & Feldman 1973 Brooks (ACRONYM) 1979 Marr 1982 Ohta & Kanade 1973 • Recent work in 2D context He, Zemel, Cerreira-Perpiñán 2004 Carbonetto, Freitas, Banard 2004 Winn & Shotton 2006 Kumar & Hebert 2005 Torralba, Murphy, Freeman 2004 Fink & Perona 2003

  7. Real Relationships are 3D Close Not Close

  8. Recent Work in 3D [Han & Zu 2003] [Oliva & Torralba 2001] [Torralba, Murphy & Freeman 2003] [Han & Zu 2005]

  9. Objects and Scenes • Biederman’s Relations among Objects in a Well-Formed Scene (1981): Hock, Romanski, Galie, & Williams 1978 • Support • Size • Position • Interposition • Likelihood of Appearance

  10. Contribution of this Paper • Biederman’s Relations among Objects in a Well-Formed Scene (1981): Hock, Romanski, Galie, & Williams 1978 • Support • Size • Position • Interposition • Likelihood of Appearance

  11. Object Support

  12. Object Surface? Support? Surface Estimation Image Support Vertical Sky V-Center V-Right V-Porous V-Solid V-Left [Hoiem, Efros, Hebert ICCV 2005] Software available online

  13. Object Size in the Image Image World

  14. Object Size ↔ Camera Viewpoint Input Image Loose Viewpoint Prior

  15. Object Size ↔ Camera Viewpoint Input Image Loose Viewpoint Prior

  16. Object Size ↔ Camera Viewpoint Viewpoint Object Position/Sizes

  17. Object Size ↔ Camera Viewpoint Viewpoint Object Position/Sizes

  18. Object Size ↔ Camera Viewpoint Viewpoint Object Position/Sizes

  19. Object Size ↔ Camera Viewpoint Viewpoint Object Position/Sizes

  20. P(surfaces) P(viewpoint) P(object | surfaces) P(object | viewpoint) What does surface and viewpoint say about objects? Image P(object)

  21. What does surface and viewpoint say about objects? Image P(surfaces) P(viewpoint) P(object | surfaces, viewpoint) P(object)

  22. Scene Parts Are All Interconnected Objects 3D Surfaces Camera Viewpoint

  23. Input to Our Algorithm Object Detection Surface Estimates Viewpoint Prior Local Car Detector Local Ped Detector Local Detector: [Dalal-Triggs 2005] Surfaces: [Hoiem-Efros-Hebert 2005]

  24. Scene Parts Are All Interconnected Objects 3D Surfaces Viewpoint

  25. Our Approximate Model Objects 3D Surfaces Viewpoint

  26. Inference over Tree Easy with BP Viewpoint θ Local Object Evidence Local Object Evidence Objects ... o1 on Local Surface Evidence Local Surface Evidence Local Surfaces … s1 sn

  27. Viewpoint estimation Viewpoint Prior Viewpoint Final Likelihood Likelihood Horizon Height Horizon Height

  28. Object detection Car: TP / FP Ped: TP / FP Initial (Local) Final (Global) Car Detection 4 TP / 1 FP 4 TP / 2 FP Ped Detection 4 TP / 0 FP 3 TP / 2 FP Local Detector: [Dalal-Triggs 2005]

  29. Experiments on LabelMe Dataset • Testing with LabelMe dataset: 422 images • 923 Cars at least 14 pixels tall • 720 Peds at least 36 pixels tall

  30. Each piece of evidence improves performance Pedestrian Detection Car Detection Local Detector from [Murphy-Torralba-Freeman 2003]

  31. Can be used with any detector that outputs confidences Car Detection Pedestrian Detection Local Detector: [Dalal-Triggs 2005] (SVM-based)

  32. Accurate Horizon Estimation [Dalal- Triggs 2005] [Murphy-Torralba-Freeman 2003] Horizon Prior Median Error: 8.5% 4.5% 3.0% 90% Bound:

  33. Qualitative Results Car: TP / FP Ped: TP / FP Initial: 2 TP / 3 FP Final: 7 TP / 4 FP Local Detector from [Murphy-Torralba-Freeman 2003]

  34. Qualitative Results Car: TP / FP Ped: TP / FP Initial: 1 TP / 14 FP Final: 3 TP / 5 FP Local Detector from [Murphy-Torralba-Freeman 2003]

  35. Qualitative Results Car: TP / FP Ped: TP / FP Initial: 1 TP / 23 FP Final: 0 TP / 10 FP Local Detector from [Murphy-Torralba-Freeman 2003]

  36. Qualitative Results Car: TP / FP Ped: TP / FP Initial: 0 TP / 6 FP Final: 4 TP / 3 FP Local Detector from [Murphy-Torralba-Freeman 2003]

  37. meters meters Summary & Future Work Ped Ped Car • Reasoning in 3D: • Object to object • Scene label • Object segmentation

  38. Conclusion • Image understanding is a 3D problem • Must be solved jointly • This paper is a small step • Much remains to be done

  39. Thank you

  40. Guzman (SEE), 1968 Hansen & Riseman (VISIONS), 1978 Barrow & Tenenbaum 1978 Brooks (ACRONYM), 1979 Marr, 1982 Ohta & Kanade, 1978 Yakimovsky & Feldman, 1973 A Return to Scene Understanding [Ohta & Kanade 1978]

  41. Images

More Related