320 likes | 682 Views
Vision-based 3D Bicycle Tracking using Deformable Part Model and Interacting Multiple Model Filter. May 11, 2011 Hyunggi Cho 1 , Paul Rybski 1,2 , and Wende Zhang 3. 1 Electrical and Computer Engineering School of Engineering Carnegie Mellon University. 3 The Electrical and Controls
E N D
Vision-based 3D Bicycle Tracking using Deformable Part Model and Interacting Multiple Model Filter May 11, 2011 Hyunggi Cho1, Paul Rybski1,2, and Wende Zhang3 1Electrical and Computer Engineering School of Engineering Carnegie Mellon University 3The Electrical and Controls Integration Lab. General Motors R&D 2Robotics Institute School of Computer Science Carnegie Mellon University
Outline • Motivation and Overview • Bicycle Detection • Bicycle Tracking • Experimental Results • Conclusion and Future work
Motivation • Motivation • In 2009, 630 bicyclists were killed and 51,000 were injured in traffic accidents in the United States*. • Bicyclists and pedestrians are the most vulnerable traffic participants. • There is less research on bicyclist detection and tracking compared to thatof pedestrians. Movie clip: Bicycle messengers in New York City (Youtube) *http://www.nhtsa.gov
Sensors on Cars Source : http://www.tartanracing.org
System Overview • Track bicycles using a single video camera mounted on a vehicle output : position & velocity Input : video System • System block diagram Bicycle Tracker Bicycle Detector Bicycle’s position & velocity
Bicycle Detection – Deformable Part Model HOG Detector • Eight view-based bicycle detection root filters coarse resolution part filters finer resolution deformation models P. Felzenszwalb, R. Girshick, D. McAllester, D. Ramanan, Object Detection with Discriminatively Trained Part Based Models, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010
DPM HOG Detector – Object hypothesis • Bicycle detection process P. Felzenszwalb, R. Girshick, D. McAllester, D. Ramanan, Object Detection with Discriminatively Trained Part Based Models, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010
DPM HOG Detector – Performance Analysis • Examples of bicycle detection Test images from Google image
DPM HOG Detector – Performance Analysis • Terminology • Eight view • Precision-Recall Curve (VOC2009 + Ours) • Training Set : 350 positive / 3300 negative • Test Set : : VOC2009 ‘val’ dataset • Recall (True Positive Rate) • Precision
Overview of our single bicycle tracking system • Kalman filter-based tracking 2D image space state space : x y Prediction stage Update stage
Overview of our single bicycle tracking system • Dynamic system model : Dynamic equation : Measurement equation : Initial state : Process noise : Measurement noise x y • Motion model : constant velocity
Overview of our single bicycle tracking system • Measurement model : perspective projection Image Image plane Horizon height
GM GM IMM - Choosing a model set • Model Set I • Model Set II • Model Set III • Constant Velocity • Constant Acceleration • Constant Velocity • Coordinated Turn • Constant Velocity • Simplified Bicycle with CV and CY angle CA CV CV CV CT SB
IMM - Choosing a model set • Constant Velocity model : • Simplified Bicycle model :
IMM - Performance analysis • IMM Tracking performance • We tested the IMM method on the GM bicycle dataset • Test Set : 6 sequences with a stationary GM test vehicle • Data statistics : Size : 320x240 , FPS : 10~12 , No. of bicycles : 1 • Details of six bicycle sequences ( SM vs. IMM )
IMM - Performance analysis • Sequence 3 case
Multiple bicycle tracking using a Rao-Blackwellized particle filter • Multiple bicycle tracking problem • Single bicycle tracking : We solved this problem. • Data association : Given a measurement, which target produced it, if any ? • Unknown number of targets : How many bicycles are there ? • In our multiple bicycle tracking case Particle filter Kalman filter Joint state vector : Data association indicator :Target visibility indicator Simo Särkkä, Aki Vehtari, and Jouko Lampinen (2007). Rao-Blackwellized Particle Filter for Multiple Target Tracking. Information Fusion Journal, Volume 8, Issue 1, Pages 2-15
Multiple bicycle tracking using a Rao-Blackwellized particle filter • Particle filter for data association problem • Only one target can die • is associated with : • Clutter • One of the existing targets • A newborn target y2 • Example y3 t • All possible events between two measurements and t : Target : Measurement : Trajectory t-1 t-1 y1 t-2 t-2
Experimental Results • Tracking performance • We tested our detection/tracking system on our bicycle dataset • Test Set : A challenging sequence from a moving Boss (so called ‘Free for all’) • Data statistics : Size : 320 x 240 , Frame rate : 13~15 frame per second • Sensor coverage area 4 m 15 m 4 m 5 m 0 m Minimum pixel size HOG Detector : 32x64
Experimental Results - data collection • US Bicyclists Crash Types – Top 10covering 61% database samples W.H. Hunter, W.E. Pein, and J.C. Stutts, Bicycle Crash Types: A 1990's Informational Guide, Publication No. FHWA-RD-96-104, Federal Highway Administration, Washington, DC, April, 1997
Experimental Results - data collection W.H. Hunter, W.E. Pein, and J.C. Stutts, Bicycle Crash Types: A 1990's Informational Guide, Publication No. FHWA-RD-96-104, Federal Highway Administration, Washington, DC, April, 1997
Experimental Results - data collection We categorized the upper scenarios into 4 different classes in terms of bicycle motion patterns !!! W.H. Hunter, W.E. Pein, and J.C. Stutts, Bicycle Crash Types: A 1990's Informational Guide, Publication No. FHWA-RD-96-104, Federal Highway Administration, Washington, DC, April, 1997
Experimental Results – Performance analysis • Scenario – Random moving case (‘Free for all’) 2D Bounding box {view (x coordinate, y coordinate)} Uncertainty level
Experimental Results – Performance analysis • Scenario – Random moving case with 3D visualization 2D Bounding box {view (x coordinate, y coordinate)} Uncertainty level
Summary and Future Work • Summary • Data collection • - Based on bicycle accident statistics • Detection part • - Applied DPM HOG detector into a multiple bicycle tracking system • Tracking part • - Incorporate Interacting Multiple Model (IMM) algorithm into our multiple bicycle tracking system to exploit several types of motion models • - RBPF data association algorithm • Future work • Real-time C++ implementation ( > 10fps) • Integration the system into the perception system of our autonomous vehicles at CMU
Single bicycle tracking using an IMM • Main idea of Interacting Multiple Model (IMM) • True motion of a bicycle cannot be exactly modeled by just one model, only be sufficiently approximated by using several motion models for representing dynamic driving behaviors • of a target (i.e., maneuverings of a bicycle). • The IMM filter runs several motion models in parallel and estimates a state by computing • a weighted sum of several filter results which are based on different motion models.
Vision-based Bicycle Detector • Related works