Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System

1. Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System QingMing qing@islab.ulsan.ac.kr May. 11. 2012

2. Vision Based Driver Assistant System

3. Problem setting Main Goal: Multiple vehicle detection and tracking Challenge work: -Different environment(illumination, distance, background…) -Tracking windows scale dynamic adjustment -Vehicle partial occlusion -Vehicle temporary missing

4. System architecture Frame 1 Frame 2 Frame n … Vehicle detection unit Vehicle detection unit Vehicle detection unit Detected new vehicle? Detected new vehicle? Detected new vehicle? yes yes yes … Particle filter 1 Particle filter 2 Particle filter m Internal storage Return to Vehicle detection unit Vehicle Tracking Unit

5. Vehicle Detection Algorithm Offline process Image Sequence Test Image Gabor Feature Extraction Vehicle Candidate Detection BP Neural Network Training Vehicle Candidate Verification BP Neural Network Classifier Detected Vehicle

6. Vehicle candidate detection + Original image Color segmentation Morphological operation Threshold obtained by tail light image statistical value

7. Vehicle candidate generation wc1c2 hc1 hc2 wc1c2: width between vehicle light pair. hc1, hc2: height ofC1andC2 d : a constant which depends on image size

8. Gabor feature 5 scale 8 orientations Tai Sing LEE, “Image Representation using 2D Gabor Wavelets,”IEEE Transactions on Parrern Analysis and Machine Intelligence,Vol 18,No.10, pp959-971,October 1996

9. Training -1 … Back propagation Neural network Non-vehicle database 1 … Stuart Russell and Peter Norvig, “Artificial Intelligence A Modern Approach”. p. 578. 1969 vehicle database

10. Vehicle candidate verification Gabor feature set … (Yes) training BPNN classifier (Yes) (No)

11. Vehicle Tracking Frame t+1 Frame t Particle generation … Detected vehicle Color histogram Updated histogram Particle selection Histogram generation Similarity computation Tracking Window estimation Histogram updating

12. Target Vehicle Representation Frame t Split into uniform Histogram bins Detected vehicle Color space representation Histogram representation

13. Particle Generation • Randomly generate particles at the position of tracking window in previous frame Color PDF Frame t+1 Particle selection Each Particle is consider as one pixel

14. Similarity Computing …… Each selected Particle is consider as one region • Bhattacharyya coefficient • Mean state of the particle set Dorin Comaniciu, Visvanathan Ramesh, P eterMeer, "Real-Time Tracking of Non-Rigid Objects using Mean Shift, " IEEE Conference on Computer Vision and Pattern Recognition, June 13 -15, Hilton Head, SC, USA, 2000 • Mean state of the particle set is computed for tracking window state estimation.

15. Partial occlusion and temporary missing Target vehicle Partial occlusion Temporary missing Frame 1 Frame 45 Frame 25 Partial occlusion Target vehicle re-tracking Temporary missing Frame 80 Frame 120 Frame 148

16. Temporary missing Particles are generated nearby the covering vehicle bounding box effective particles are searched Frame 125 Frame 80 When enough effective particles are searched, the missing vehicle start Tracking again

17. Color Histogram Updating • Target vehicle color histogram changing under different condition (ex: different distance, different illumination…)

18. Color Histogram Updating Tracking without color histogram updating Frame 1 Frame 55 Frame 62 Frame 115 Tracking with color histogram updating Frame 1 Frame 55 Frame 62 Frame 115

19. Detection result High way Urban Road Campus

20. Multiple vehicle detection result

21. Tracking result

22. Tracking result Horizontal trajectory Vertical trajectory

23. Tracking result Trajectory in image plane Tracking error

24. Partial occlusion and temporary missing result

25. Multiple vehicle tracking result

26. Conclusion Advantage • Detected multiple vehicles in different environment (different light • condition, different size vehicle, different speed) • Tracked partial occluded vehicle • Re-tracked temporary missing vehicle • Tracking windows dynamically adapt according to target vehicle scale • changing • Color histogram self-updating Disadvantage • Only color model based multiple vehicle tracking is not suitable for same • color Vehicle occlusion problem Future works • Camera stabilization for smooth trajectory generation • Combine with odometry information to predict dangerous situations

27. Publications • Qing Ming, Kang-Hyun Jo, “Vehicle Detection Using Tail Light Segmentation,” International Forum on Strategic Technology, August 22-24, Harbin, 2011. • Ming Qing and Kang-Hyun Jo, “Vehicle Detection and Scale-adaptive Tracking Using Tail Light Segmentation,” proc. of image and vision computing New Zealand, pp. 115-119, 2011. • Ming Qing and Kang-Hyun Jo, “A novel particle filter implementation for Multiple-Vehicle Detection and Tracking System using Tail Light Segmentation”,International Journal of Control, Automation, and Systems (Reviewing) • Ming Qing and Kang-Hyun Jo, “Vision Based Multiple Vehicle Detection and Tracking Using Tail Light Segmentation”, IECON Montreal (reviewing), October 25-28, 2012 • Mecatronics 2012, Paris, Date line: 30 May,2012

28. Thank you for your attention

29. BPNN Back propagation nerual network: Xi : input Zi :output t1: expect output wij : weights between input layer and hidden layer wjk : weights between hidden layer and output layer. The input goes through the neural network in order to obtain the forward propagation’s output. Compare with expect output, difference value backward propagation through network to update weights

30. HSV color model H: Hue S: Saturation V: Value

31. Particlefilter Particle filter: Zt+1 evaluation Particle generation Propagate Propagate … p(xt+2|zt+1) p(xt+1|zt) p(xt|zt) p(xt+1|zt+1) Zt: observation P(xt|Zt): particle state under current observation P(xt+1|Zt): particle state prediction

32. Bhattacharyya coefficient Bhattacharyya coefficient is an approximate measurement of the amount of overlap between two statistical samples. This coefficient can be used to describe the similarity of two discrete and normalized distributions.