1 / 36

Robust Hand Tracking with Refined CAMShift Based on Combination of Depth and Image Features

Robust Hand Tracking with Refined CAMShift Based on Combination of Depth and Image Features. Wenhuan Cui, Wenmin Wang, and Hong Liu. International Conference on Robotics and Biomimetics , IEEE , 2012. Outline. Introduction Related Work Proposed Method Experimental Results Conclusion.

moshe
Download Presentation

Robust Hand Tracking with Refined CAMShift Based on Combination of Depth and Image Features

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Robust Hand Tracking with Refined CAMShift Based on Combination of Depth and Image Features Wenhuan Cui, Wenmin Wang, and Hong Liu International Conference on Robotics and Biomimetics, IEEE, 2012

  2. Outline • Introduction • Related Work • Proposed Method • Experimental Results • Conclusion

  3. Introduction

  4. Introduction • Hand Tracking: • Essential for HCI • Most researchers simplify the issue by restrictions: • On users’ clothing • On the scene complexity • On hand motion Zhou Ren, Junsong Yuan, , JingjingMeng, M, and Zhengyou Zhang, "Robust Part-Based Hand Gesture Recognition Using Kinect Sensor", IEEE TRANSACTIONS ON MULTIMEDIA, AUGUST 2013

  5. Introduction • In this paper: • Propose a robust hand tracking method • Focus on reducing restrictions • Combining: • Depth cues • Color cues • (Motion cues) Refined CAMShift tracking

  6. Related Work

  7. Related work • Tracking: [a] fingertip [b] hand Geodesic distance ‧Seed Point ‧Predicted hand position GSP points Neighbor depth

  8. Related work [c] • Difficulties: [d] --(Red) : Side-mode ㄧ(Blue) : Frontal-mode

  9. Related work • [a] Hui Liang, Junsong Yuan, and Daniel Thalmann, "3D Fingertip and Palm Tracking in Depth Image Sequences", Proceedings of the 20th ACM international conference on Multimedia, 2012 • [b]Chia-Ping Chen, Yu-Ting Chen, Ping-Han Lee, Yu-Pao Tsai, and Shawmin Lei, "Real-time Hand Tracking on Depth Images", IEEE Visual Communications and Image Processing (VCIP), 2011 • [c] Ziyong Feng, Shaojie Xu, Xin Zhang, Lianwen Jin, Zhichao Ye, and Weixin Yang, “Real-time Fingertip Tracking and Detection using Kinect Depth Sensor for a New Writing-in-the Air System”, Proceedings of the 4th International Conference on Internet Multimedia Computing and Service, 2012 • [d] Zhichao Ye, Xin Zhang, Lianwen Jin, Ziyong Feng, Shaojie Xu, "FINGER-WRITING-IN-THE-AIR SYSTEM USING KINECT SENSOR", IEEE International Conference on Multimedia and Expo Workshops (ICMEW), 2013

  10. ProposedMethod

  11. Flow Chart Hand Detection Hand Tracking

  12. Foreground Segmentation Down-Sampled • Codebook model • Codeword: • Motion detection(Foreground): K. Kim, T. H. Chalidabhongse, D. Harwood, and L. Davis. Real time foreground-background segmentation using code book model. Real-Time Imaging, 11:172–185, 2005.

  13. Foreground Segmentation

  14. Histogram-based Segmentation • Stretch ahead • Depth histogram • Stretch laterally • X-projection histogram Mask Mask

  15. Histogram-based Segmentation • Stretch ahead • Depth histogram depth

  16. Histogram-based Segmentation Upper boundary • Stretch laterally • X-projection histogram Lower boundary j-th bin x

  17. Histogram-based Segmentation • Histogram Analysis • Depth histogram & X-projection histogram • Foothill algorithm:

  18. Histogram-based Segmentation max • Depth histogram  1 01 10 01 10 0 max  • X-projection • histogram  1 01 0 000000111111011100000

  19. Histogram-based Segmentation • X-projection • histogram Scaled x-mask

  20. Skin Color Feature ‧ ‧ ‧ Mask

  21. Integration of Features • Hand Detection: skin depth (stretch ahead) X-projection (stretch laterally)

  22. CAMShift • Like mean-shift • 1. Back projection • Choose an object → probability map → back projection • 2.Mean-shift(frame-frame)

  23. Refined CAMShift Tracking • Probability map: • Weights: • s1 : depth mask • s2 : x-mask skin depth (stretch ahead) X-projection (stretch laterally) blob size < threshold otherwise

  24. Refined CAMShiftTracking • Ecliptic shape representation Aspect ratio: Search window for the next frame:

  25. Refined CAMShiftTracking • Blob refinement 1. Choose proper reference line 2. 3. Reduce the lof the ellipse, untilla proper aspect ratio l/wis obtained.

  26. Refined CAMShiftTracking • Aspect ratio based blob refinement

  27. Detection + Tracking • Tracking fast movement

  28. Detection + Tracking • Face & Hand

  29. ExperimentalResults

  30. Experimental Results • Comparison of overall performance ‧Training: 4.8s / 10FPS [10] C. Shan, Y. Wei, T. Tan, F. Ojardias, ”Real Time Hand Tracking by Combining Particle Filtering and Mean Shift”, In: International Conference on Automatic Face and Gesture Recognition, 2004, pp. 669-674

  31. A) Refined CAMShift with color cue

  32. B) Multi-cue CAMShift without refinement

  33. C) The proposed approach

  34. Experimental Results • Video description experimental results

  35. Conclusion

  36. Conclusion • Focus on reducing restrictions • Hand Segmentation: • Depth + Skin + (Motion) • Histogram analysis • Hand tracking • CAMShift • Blob refinement

More Related