1 / 28

Dynamic 3D Scene Analysis from a Moving Vehicle

Dynamic 3D Scene Analysis from a Moving Vehicle. Young Ki Baik (CV Lab.) 2007. 7. 11 (Wed). Dynamic Scene Analysis from a Moving Vehicle. References. Dynamic 3D Scene Analysis from a Moving Vehicle Bastian Leibe, Nico Cornelis, Kurt cornelis, Luc Van Gool

gavivi
Download Presentation

Dynamic 3D Scene Analysis from a Moving Vehicle

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Dynamic 3D Scene Analysis from a Moving Vehicle Young Ki Baik (CV Lab.) 2007. 7. 11 (Wed)

  2. Dynamic Scene Analysis from a Moving Vehicle • References • Dynamic 3D Scene Analysis from a Moving Vehicle • Bastian Leibe, Nico Cornelis, Kurt cornelis, Luc Van Gool • Awarded the best paper prize (CVPR 2007) • Fast Compact City Modeling for Navigation Pre-Visualization • Nico Cornelis, Kurt cornelis, Luc Van Gool (CVPR 2006) • Pedestrian detection in crowded scene • Bastian Leibe et. al. (CVPR 2005) • Putting Objects in Perspective • Derek Hoiem et. al. (CVPR 2006)

  3. Dynamic Scene Analysis from a Moving Vehicle • Why? • … were they received the best paper prize? • They completed the impressive real application with only toy computer vision algorithm. • They showed that the field of vision will be a key of future technique to the public.

  4. Dynamic Scene Analysis from a Moving Vehicle • Demo (Final result)

  5. Dynamic Scene Analysis from a Moving Vehicle • What? • …is the purpose of this paper? • Detect object in real environment (city road) • Localize them in 3D • Predict their future motion • … is the challenges of this paper? • We are moving • Objects can be moving • Ground may not be planar

  6. Dynamic Scene Analysis from a Moving Vehicle • What methods? • … are used to accomplish their purpose? • Structure from motion • 2D object detection • 3D trajectory estimation

  7. Stereo camera Aligned stereo image 3D structure info. Ground plane 2D and 3D Object 3D trajectory Orientation Dynamic Scene Analysis from a Moving Vehicle • Overall flow 1. SfM 3. Tracking 2. Object detection

  8. Dynamic Scene Analysis from a Moving Vehicle • 3D structure and ground plane • 3D Structure from Motion • Visual odometry (David Nister) • Use pre-calibrated stereo camera • Use rectified stereo images • Parallel processing → Extrinsic camera parameters → 3D camera trajectory (in real time) Nico Cornelis et. al. (CVPR 2006)

  9. Dynamic Scene Analysis from a Moving Vehicle • 3D structure and ground plane • Ground plane estimation • Known ground positions of wheel base points Nico Cornelis et. al. (CVPR 2006)

  10. Dynamic Scene Analysis from a Moving Vehicle • 3D structure and ground plane • Ground plane estimation • Compute normal locally • Average over spatial window Nico Cornelis et. al. (CVPR 2006)

  11. Dynamic Scene Analysis from a Moving Vehicle • SfM Demo Nico Cornelis et. al. (CVPR 2006)

  12. Dynamic Scene Analysis from a Moving Vehicle • Object detection • 2D/3D Interaction method • Likelihood of 3D object hypothesis H → Given image I and a set of 2D detections h:

  13. Dynamic Scene Analysis from a Moving Vehicle • Object detection • 2D object detection 2D recognition • ISM detectors Leibe et. al. (CVPR 2005)

  14. Dynamic Scene Analysis from a Moving Vehicle • Object detection • ISM detectors (Leibe et al., CVPR’05, BMVC’06) • Battery of 5(car)+1(human) single view detectors • Each detectors based on 3 local cues • Harris-Laplace, Hessian-Laplace, DoG interest regions • Local Shape Context descriptors • Result: detections + segmentations Leibe et. al. (CVPR 2005)

  15. Dynamic Scene Analysis from a Moving Vehicle • Object detection • 2D/3D transfer 2D/3D transfer • Two image-plane detections are consistent if they correspond to the same 3D object. → Cluster 3D detections → Multi-viewpoint integration

  16. Dynamic Scene Analysis from a Moving Vehicle • Object detection • 3D prior 3D prior • By Using 3D structure and ground plane constraint… → Distance prior (Distance from the ground plane) → Size prior (Gaussian) Significantly reduced search space and outlier Hoiem et. al. (CVPR 2006)

  17. Dynamic Scene Analysis from a Moving Vehicle • Quantitative results of detection • Detection performance on 2 test sequences • Stereo and Ground plane constraints significantly improves precision

  18. Dynamic Scene Analysis from a Moving Vehicle • Detection Demo

  19. Dynamic Scene Analysis from a Moving Vehicle • Object tracking • Localization and Trajectory estimation • By using detection results • Obtain orientation of objects • Space-time trajectory analysis • By using the concept of a bidirectional Extended Kalman Filter

  20. Dynamic Scene Analysis from a Moving Vehicle • Object tracking • 3D Localization for static objects (car) • Location • Mean-shift search to find set of 3D detection hypotheses • Orientation • Cluster shape and detector output

  21. Dynamic Scene Analysis from a Moving Vehicle • Object tracking • Dynamic model • Holonomic motion (Pedestrian) • Without external constraints linking its speed and turn rate • Nonholonomic motion (Car) • Only move along its main axis • Only turn while moving

  22. Dynamic Scene Analysis from a Moving Vehicle • Object tracking • Trajectory growing • Collect detection in time space

  23. Dynamic Scene Analysis from a Moving Vehicle • Object tracking • Trajectory growing • Collect detection in time space • Evaluate under trajectory • Bi-directionally • Static assumption

  24. Dynamic Scene Analysis from a Moving Vehicle • Object tracking • Trajectory growing • Collect detection in time space • Evaluate under trajectory • Bi-directionally • Static assumption • Adjust trajectory • Weighted mean • Predicted position • Supporting observations

  25. Dynamic Scene Analysis from a Moving Vehicle • Object tracking • Trajectory growing • Collect detection in time space • Evaluate under trajectory • Bi-directionally • Static assumption • Adjust trajectory • Weighted mean • Predicted position • Supporting observations • Iteration

  26. Dynamic Scene Analysis from a Moving Vehicle • Object tracking • Trajectory growing • Collect detection in time space • Evaluate under trajectory • Bi-directionally • Static assumption • Adjust trajectory • Weighted mean • Predicted position • Supporting observations • Iteration • Location and orientation

  27. Dynamic Scene Analysis from a Moving Vehicle • Demo (Final result)

  28. Dynamic Scene Analysis from a Moving Vehicle • Conclusion • Summary • Exact value of 3D information • help to propose the new concept of detection algorithm • raise the performance of detection algorithm. • Better detection results • Give more reliable tracking results • Good orientation estimation • Contribution • New detection algorithm using 3d information • Good integration and visualization of application system

More Related