1 / 30

Vision-based 3D Bicycle Tracking using Deformable Part Model

Vision-based 3D Bicycle Tracking using Deformable Part Model and Interacting Multiple Model Filter. May 11, 2011 Hyunggi Cho 1 , Paul Rybski 1,2 , and Wende Zhang 3. 1 Electrical and Computer Engineering School of Engineering Carnegie Mellon University. 3 The Electrical and Controls

nthompson
Download Presentation

Vision-based 3D Bicycle Tracking using Deformable Part Model

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Vision-based 3D Bicycle Tracking using Deformable Part Model and Interacting Multiple Model Filter May 11, 2011 Hyunggi Cho1, Paul Rybski1,2, and Wende Zhang3 1Electrical and Computer Engineering School of Engineering Carnegie Mellon University 3The Electrical and Controls Integration Lab. General Motors R&D 2Robotics Institute School of Computer Science Carnegie Mellon University

  2. Outline • Motivation and Overview • Bicycle Detection • Bicycle Tracking • Experimental Results • Conclusion and Future work

  3. Motivation • Motivation • In 2009, 630 bicyclists were killed and 51,000 were injured in traffic accidents in the United States*. • Bicyclists and pedestrians are the most vulnerable traffic participants. • There is less research on bicyclist detection and tracking compared to thatof pedestrians. Movie clip: Bicycle messengers in New York City (Youtube) *http://www.nhtsa.gov

  4. Sensors on Cars Source : http://www.tartanracing.org

  5. System Overview • Track bicycles using a single video camera mounted on a vehicle output : position & velocity Input : video System • System block diagram Bicycle Tracker Bicycle Detector Bicycle’s position & velocity

  6. Bicycle Detection – Deformable Part Model HOG Detector • Eight view-based bicycle detection root filters coarse resolution part filters finer resolution deformation models P. Felzenszwalb, R. Girshick, D. McAllester, D. Ramanan, Object Detection with Discriminatively Trained Part Based Models, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010

  7. DPM HOG Detector – Object hypothesis • Bicycle detection process P. Felzenszwalb, R. Girshick, D. McAllester, D. Ramanan, Object Detection with Discriminatively Trained Part Based Models, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010

  8. DPM HOG Detector – Performance Analysis • Examples of bicycle detection Test images from Google image

  9. DPM HOG Detector – Performance Analysis • Terminology • Eight view • Precision-Recall Curve (VOC2009 + Ours) • Training Set : 350 positive / 3300 negative • Test Set : : VOC2009 ‘val’ dataset • Recall (True Positive Rate) • Precision

  10. Overview of our single bicycle tracking system • Kalman filter-based tracking 2D image space state space : x y Prediction stage Update stage

  11. Overview of our single bicycle tracking system • Dynamic system model : Dynamic equation : Measurement equation : Initial state : Process noise : Measurement noise x y • Motion model : constant velocity

  12. Overview of our single bicycle tracking system • Measurement model : perspective projection Image Image plane Horizon height

  13. GM GM IMM - Choosing a model set • Model Set I • Model Set II • Model Set III • Constant Velocity • Constant Acceleration • Constant Velocity • Coordinated Turn • Constant Velocity • Simplified Bicycle with CV and CY angle CA CV CV CV CT SB

  14. IMM - Choosing a model set • Constant Velocity model : • Simplified Bicycle model :

  15. IMM - Performance analysis • IMM Tracking performance • We tested the IMM method on the GM bicycle dataset • Test Set : 6 sequences with a stationary GM test vehicle • Data statistics : Size : 320x240 , FPS : 10~12 , No. of bicycles : 1 • Details of six bicycle sequences ( SM vs. IMM )

  16. IMM - Performance analysis • Sequence 3 case

  17. Multiple bicycle tracking using a Rao-Blackwellized particle filter • Multiple bicycle tracking problem • Single bicycle tracking : We solved this problem. • Data association : Given a measurement, which target produced it, if any ? • Unknown number of targets : How many bicycles are there ? • In our multiple bicycle tracking case Particle filter Kalman filter Joint state vector : Data association indicator :Target visibility indicator Simo Särkkä, Aki Vehtari, and Jouko Lampinen (2007). Rao-Blackwellized Particle Filter for Multiple Target Tracking. Information Fusion Journal, Volume 8, Issue 1, Pages 2-15

  18. Multiple bicycle tracking using a Rao-Blackwellized particle filter • Particle filter for data association problem • Only one target can die • is associated with : • Clutter • One of the existing targets • A newborn target y2 • Example y3 t • All possible events between two measurements and t : Target : Measurement : Trajectory t-1 t-1 y1 t-2 t-2

  19. Experimental Results • Tracking performance • We tested our detection/tracking system on our bicycle dataset • Test Set : A challenging sequence from a moving Boss (so called ‘Free for all’) • Data statistics : Size : 320 x 240 , Frame rate : 13~15 frame per second • Sensor coverage area 4 m 15 m 4 m 5 m 0 m Minimum pixel size HOG Detector : 32x64

  20. Experimental Results - data collection • US Bicyclists Crash Types – Top 10covering 61% database samples W.H. Hunter, W.E. Pein, and J.C. Stutts, Bicycle Crash Types: A 1990's Informational Guide, Publication No. FHWA-RD-96-104, Federal Highway Administration, Washington, DC, April, 1997

  21. Experimental Results - data collection W.H. Hunter, W.E. Pein, and J.C. Stutts, Bicycle Crash Types: A 1990's Informational Guide, Publication No. FHWA-RD-96-104, Federal Highway Administration, Washington, DC, April, 1997

  22. Experimental Results - data collection We categorized the upper scenarios into 4 different classes in terms of bicycle motion patterns !!! W.H. Hunter, W.E. Pein, and J.C. Stutts, Bicycle Crash Types: A 1990's Informational Guide, Publication No. FHWA-RD-96-104, Federal Highway Administration, Washington, DC, April, 1997

  23. Experimental Results - data collection

  24. Experimental Results – Performance analysis • Scenario – Random moving case (‘Free for all’) 2D Bounding box {view (x coordinate, y coordinate)} Uncertainty level

  25. Experimental Results – Performance analysis • Scenario – Random moving case with 3D visualization 2D Bounding box {view (x coordinate, y coordinate)} Uncertainty level

  26. Summary and Future Work • Summary • Data collection • - Based on bicycle accident statistics • Detection part • - Applied DPM HOG detector into a multiple bicycle tracking system • Tracking part • - Incorporate Interacting Multiple Model (IMM) algorithm into our multiple bicycle tracking system to exploit several types of motion models • - RBPF data association algorithm • Future work • Real-time C++ implementation ( > 10fps) • Integration the system into the perception system of our autonomous vehicles at CMU

  27. Q&A

  28. Single bicycle tracking using an IMM • Main idea of Interacting Multiple Model (IMM) • True motion of a bicycle cannot be exactly modeled by just one model, only be sufficiently approximated by using several motion models for representing dynamic driving behaviors • of a target (i.e., maneuverings of a bicycle). • The IMM filter runs several motion models in parallel and estimates a state by computing • a weighted sum of several filter results which are based on different motion models.

  29. Integral HOG Detector - Performance Analysis II

  30. Vision-based Bicycle Detector • Related works

More Related