1 / 32

Jungong Han, Eric J. Pauwels , Paul M. de Zeeuw , and Peter H.N. de With, Fellow, IEEE

Employing a RGB-D Sensor for Real-Time Tracking of Humans Across Multiple Re-Entries in a Smart Environment. Jungong Han, Eric J. Pauwels , Paul M. de Zeeuw , and Peter H.N. de With, Fellow, IEEE IEEE Transactions on Consumer Electronics , 2012. Outline. Introduction Related Works

ghita
Download Presentation

Jungong Han, Eric J. Pauwels , Paul M. de Zeeuw , and Peter H.N. de With, Fellow, IEEE

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Employing a RGB-D Sensor for Real-Time Tracking of Humans Across Multiple Re-Entries in a Smart Environment Jungong Han, Eric J. Pauwels, Paul M. de Zeeuw, and Peter H.N. de With, Fellow, IEEE IEEE Transactions on Consumer Electronics, 2012

  2. Outline • Introduction • Related Works • Proposed Method • Experimental Results • Conclusion

  3. Introduction

  4. Introduction especially for elderly or disabledpeople. • Smart environment • Location • Identity

  5. Introduction • Smart environment • Location • Identity • Goal: Detect and track humans in a home-used system • Using a low-cost consumer-level RGB-D camera. • Combine the advantages of color and depth characteristics . especially for elderly or disabledpeople.

  6. Introduction • Requirements of home-used human tracking system • Track multiple persons • Be robust against changes in the environment • Could re-identifying persons • Real-time performance • Low-cost camera sensors

  7. Related Works

  8. Related Works • Human segmentation • RGB camera: backgroundmodeling • Median filter[3]and Gaussian Mixture Model[4] • Depth camera[10] • Motion and depth[11] • Graph cut[12]

  9. Related Works • Human tracking • RGB camera: appearance modeling • Mean shift tracker[5]: real-time non-parametric technique • Particle-filter[6]: a random search • Depth camera • Expectation Maximization algorithm[12]

  10. Related Works • RGB camera • Intuitive and easy • Depend on color/intensity =>unreliable • Depth camera • Illumination suitable • Can’t handle occlusion and identification • Both camera [13,14,15] • Proposed method

  11. Reference [3] R. Culter, and L. Davis, “View-based detection,” Proc. ICPR, 1998. [4] Z. Zivkovic, “Improved adaptive Gaussian Mixture Model for background subtraction,” Proc. ICPR, pp. 28-31, Aug. 2004. [5] D. Comaniciu, V. Ramesh, and P. Meer, “Kernel-based object tracking,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 5, pp. 564-577, 2003. [6] K. Nummiaro, E. Koller-Meier and L. Van Gool “An adaptive color-based particle filter” Image Vis. Comp, 2003 [10] A. Bevilacqua, L. Di and S. Azzari,“People tracking using a Time-of-Flight depth sensor”Proc. IEEE Int. Conf. Video and Signal based Surveillance, 2006 [11]D. Hansen, M. Hansen, M. Kirschmeyer, R. Larsen and D. Silvestre “Cluster tracking with Time-of-Flight cameras” Proc. CVPR workshop on TOF-CV, June, 2008 [12] O. Arif, W. Daley, P. Vela, J. Teizer and J. Stewart “Visual tracking and segmentation using Time-of-Flight sensor” Proc. ICIP, pp. 2241-2244, Sept. 2010 [13] R. Crabb, C. Tracey, A. Puranik and J. Davis “Real-time foreground segmentation via range and color imaging” Proc. CVPR Workshop on TOF-CV, June 2008 [14] S. Gould, P. Baumstarck, P. Quigley, M. Ng and A. Koller “Integrating visual and range data for robotic object detection” Proc. ECCV Workshop on Multi-camera and Multimodal Sensor Fusion, June 2008 [15] L. Sabeti, E. Parvizi and Q. Wu “Visual tracking using color cameras and Time-of-Flight range imaging sensors” Journal of multimedia, vol. 3, no. 2, pp. 28-36, June, 2008

  12. Proposed Method

  13. Proposed Method 1. 2. 3. 4.

  14. 1. Object label • Motion detection using background subtraction • B: depth of background • Depth clustering using depth information • Seeds: Moving pixels detected in the previous step • Check the depth continuity of neighboring pixels of the seeds • Returns with several separated clusters as the objects

  15. 2. Detecting Human • Problem: depend on the posture • Solution: defer computation until the person is standing • Characteristic: a moving object can be promoted to be a human only when it is stable with sufficient height. • Stable: size changes are less than 10% in 5 successive frames • Height: related with depth [16] d: distance between the camera and the object a1 and a2 : parameters in off-line calibration [16] P. Remagnino, A. Shihab, and G. Jones, “Distributed intelligence for multi-camera visual surveillance,” Pattern Recognition, vol. 37, no. 4, pp. 675-689, April 2004.

  16. 3. Human Re-Entry Identification • How? • Track persons across successive appearances in the scene • Tagpersons with a persistent ID label • Technique: appearance-based matching • Extend color histogram including color and texture • Length ratio between different body parts is fixed • Head / torso / leg

  17. 3. Human Re-Entry Identification • Head • Top 1/3 part of the entire body • Detect the length in horizontal direction • The neck width is less than others. (local minimum) • Torso and leg: by ratio

  18. 3. Human Re-Entry Identification • Use color histogram and texture to describe the human appearance • Probability of feature u: • : the pixel locations in the defined region • : associates to the pixel at location Formula the index • C: normalization constant • the Kroneckerdelta function • Texture intensity using canny detection

  19. 3. Human Re-Entry Identification • Compare two histogram as similarity

  20. 4.Human ID Tracking • Depth continuity • Gaussian distribution: • Appearance similarity • Bhattacharyya distance: • The Probability Ti matching with Dj

  21. Proposed Method 1. 2. 3. 4.

  22. Experimental Results

  23. Experimental Results • Device: Dual core 2.53 GHz, 4 GB RAM with a 64-bits operation system • Implemented by C++ with OpenNI and OpenCV library • Evaluation • A. Object labeling • B. Human detection and ID tracking • C. System efficiency

  24. Experimental Results A. • Object labeling in different situation Stable light Similar color between F & B Stable light Different color between foreground & background

  25. Using depth data  Using RGB data 

  26. Experimental Results B. • Human region detection • 96.1% accuracy in 2000 frames • Only 78 frames failed

  27. Experimental Results B. • Identification • Set the RGB-D sensor in a living room for 30 minutes, and asked persons to leave and come back for 35 times.  • The persons used 5 different coats.  • Results: only 8 occasions fail • Coats with similar colors • Posture difference

  28. Experimental Results B. • Accuracy in human tracking module(based on 5 videos ,in total 2600 frames) • Proposed: 96.27% • Particle filter[6]: 83.54% =>illumination • Mean shift filter[5]: 71.23% =>occlusion [5] D. Comaniciu, V. Ramesh, and P. Meer, “Kernel-based object tracking,”IEEETrans. Pattern Anal. Mach. Intell., 2003. [6] K. Nummiaro, E. Koller-Meier and L. Van Gool “An adaptive color-based particle filter” Image Vis. Comp, 2003

  29. Experimental Results B. 1 1 0 0 0 0 1 1 Anonym Anonym

  30. Experimental Results C. • System efficiency in 100 frames • 1-person: 41.3 ms/frame • 2-person: 73.8 ms/frame • 3-person: 97.1 ms/frame • Overall about more than10 fps

  31. Conclusion

  32. Conclusion • Proposed a two-camera system based on a RGB-D sensor, which enables person detection, tracking and re-entryidentification. • Proposed system can achieve real-time performance with sufficient accuracy • 95% detection; 80% re-identification; 96% occlusion and illumination • Future Work • Improve human detector to execute with a more general descriptor 

More Related