1 / 20

Robust Multi-Pedestrian Tracking in Thermal-Visible Surveillance Videos

Robust Multi-Pedestrian Tracking in Thermal-Visible Surveillance Videos. Alex Leykin and Riad Hammoud. Goal. Create a pedestrian tracker that operates in: Varying illumination conditions Crowded environment To achieve it we create a fusion pedestrian tracker that uses input from: IR camera

atira
Download Presentation

Robust Multi-Pedestrian Tracking in Thermal-Visible Surveillance Videos

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Robust Multi-Pedestrian Tracking in Thermal-Visible Surveillance Videos Alex Leykin and Riad Hammoud

  2. Goal Create a pedestrian tracker that operates in: • Varying illumination conditions • Crowded environment To achieve it we create a fusion pedestrian tracker that uses input from: • IR camera • RGB camera Our approach consists of two stages: adaptive fusion background model blob array Bayesian tracker

  3. Related Work Fusion background model: • Y.Owechko, S.Medasani, and N.Srinivasa “Classifier swarms for human detection in infrared imagery”, OTCBVS 2004 • M.Yasuno, N.Yasuda, andM.Aoki “Pedestrian detection and tracking in far infrared images” OTCBVS 2004 • C. Dai, Y. Zheng, X. Li “Layered Representation for Pedestrian Detection and Tracking in Infrared Imagery” OTCBVS 2005 • J.Davis, V.Sharma “Fusion-based Background Subtraction Using Contour Saliency”, OTCBVS 2005 Bayesian formulation: • J. Deutscher, B. North, B. Bascle and A. Blake “Tracking through singularities and discontinuities by random sampling”, ICCV 1999 • A. Elgammal and L. S. Davis, “Probabilistic Framework for Segmenting People Under Occlusion”, ICCV 2001. • M. Isard, J. MacCormick, “BraMBLe: a Bayesian multiple-blob tracker”, ICCV 2001 • T. Zhao, R. Nevatia “Tracking Multiple Humans in Crowded Environment”, CVPR 2004

  4. codebook codeword Background Model Two stacks of codeword values (codebooks) • Color • μRGB • Ilow • Ihi • Thermal • thigh • tlow

  5. Adaptive Background Update • Match pixel p to the codebook b • If there is no match create new codeword • Else update the codeword with new pixel information • If >1 matches then merge matching codewords • Remove the codeword if it had not appeared for a prolonged period of time • Discard infrequent codewords • Exclude p from update if it corresponds to a currently tracked body I(p) > Ilow I(p) < Ihigh (RGB(p)∙ μRGB) < TRGB t(p)/thigh > Tt1 t(p)/tlow > Tt2

  6. Subtraction Results Color model only Combined color and thermal model

  7. state prior probability observation likelihood Tracking Location of each pedestrian is estimated probabilistically based on: • Current image • Model of pedestrians • Model of obstacles The goal of our tracking system is to find the candidate state x` (a set of bodiesalong with their parameters) which, given the last known state x, will best fitthe current observation z P(x’| z, x) = P(z|x’) · P(x’|x)

  8. Tracking – Accepting the State x’ and x candidate and current states P(x) stationary distribution of Markov chain mt proposal distribution Candidate proposal state x’is drawn with probability mt(x’|x) and then accept it with the probability α(x, x’)

  9. body coordinatesare weighted uniformlywithin the rectangular region R of the floor map. U(x)R and U(y)R  variation from Kalman predicted position d(xt, x’t−1) and d(y, y’t−1) Tracking: Priors Constraintson the body parameters: N(hμ, hσ2) and N(wμ,wσ2)body width andheight Temporal continuity: d(wt, wt−1) and d(ht, ht−1) variation from the previous size N(μdoor, σdoor) distance to the closest door (for new bodies)

  10. Tracking Likelihoods: Distance weight plane Problem: blob trackers ignore blob position in 3D (see Zhao and Nevatia CVPR 2004) Solution: employ “distance weight plane” Dxy = |Pxyz, Cxyz| where P and C are world coordinates of the camera and reference point correspondingly and

  11. Tracking Likelihoods: Z-buffer 0 = background, 1=furthermost body, 2 = next closest body, etc

  12. Tracking: Likelihoods Color observation likelihood is based on the Bhattacharya distance between candidate and observed color histograms Implementation of z-buffer (Z) and distance weight plane (D) allows to compute multiple-body configuration with one computationally efficient step. Let I - set of all blob pixels O - set of body pixels Then

  13. Tracking: Jump-Diffuse Transitions • Add a new body • Delete a body • Recover a recently deleted body • Change body dimensions • Change body position

  14. H t t t-1 t-1 Tracking: Anisotropic Weighted Mean Shift Classic Mean-Shift Our Mean-Shift

  15. Results

  16. Results

  17. Results

  18. Conclusions A method to fuse visible and thermal inputs for background model creation: • robust to illumination changes • adaptive • computationally efficient (30fps+) A novel formulation of priors in MCMC particle filter:

  19. Future Work • Extend binary background mask with foreground probability values • Incorporate these probabilities into appearance-based fitness equation for particle filter-based tracker • Utilize tracklet stitching to decrease the number of broken paths

  20. Thank you!

More Related