1 / 25

Yihua Lou, Wenjun Wu Beihang University

A REAL-TIME PERSONALIZED GESTURE INTERACTION SYSTEM USING WII REMOTE AND KINECT FOR TILED-DISPLAY ENVIRONMENT. Yihua Lou, Wenjun Wu Beihang University. OUTLINE. Background & Problem Related works System & Algorithm Design Experiments Conclusion. BACKGROUND.

sabin
Download Presentation

Yihua Lou, Wenjun Wu Beihang University

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A REAL-TIME PERSONALIZED GESTURE INTERACTION SYSTEM USING WII REMOTE AND KINECT FOR TILED-DISPLAY ENVIRONMENT Yihua Lou, Wenjun Wu Beihang University

  2. OUTLINE • Background & Problem • Related works • System & Algorithm Design • Experiments • Conclusion

  3. BACKGROUND • Large Tiled-Display Environment • Large virtual desktop (tens of millions of pixels) • View and manipulate juxtaposed applications • Suitable for multi-user interaction & collaboration

  4. BACKGROUND • Somatosensory devices • Wii Remote: 3-axis accelerometer • Kinect: 30fps RGB & depth image, Skeleton tracking

  5. PROBLEM • Interaction method • Device-based: Not suitable in a large-space environment • Gesture-based: Suitable, but some technical challenges • Gesture interaction challenges • Gestures are personal • Gestures may vary from time to time • Difficult to define standard gesture vocabulary

  6. OUTLINE • Background & Problem • Related works • System & Algorithm Design • Experiments • Conclusion

  7. RELATED WORKS • Gesture recognition systems • uWave: acceleration-based, personalized, DTW based • Wiigee: acceleration-based, user-independent, HMM based • Some other Kinect based recognition systems • Gesture recognition algorithms • HMM: Most popular for user-independent recognition, requires large training dataset • DTW: No need of training, suitable for personalized

  8. OUTLINE • Background & Problem • Related works • System & Algorithm Design • Experiments • Conclusion

  9. SYSTEM & ALGORITHM DESIGNOVERVIEW • Design Aspects • Easy-to-use • Personalized • Two-handed • Ongoing Gesture • System Architecture • Gesture Input Clinet

  10. SYSTEM & ALGORITHM DESIGNFEATURE SELECTION

  11. SYSTEM & ALGORITHM DESIGNFILTER AND QUANTIZATION • Moving-average Filter: Reduce noise • Window: five samples for acceleration data, three samples for skeleton position data • Step: one sample • Quantization: Improve efficiency • Acceleration data: Non-linear, [-3g, 3g] → [-31, 31] • Skeleton position data: Linear, [-1m, 1m] → [-30, 30]; ±31 for other values

  12. SYSTEM & ALGORITHM DESIGNDYNAMIC TIME WARPING Input time series Template time series

  13. SYSTEM & ALGORITHM DESIGNTEMPLATE ADAPTION • Template accuracy is important • Directly affect the recognition accuracy • Data may vary in each gesture • Templates are adapted when rejection occurred • Two continuous or three accumulative rejections • Use the gesture input with minimal accumulated DTW distance among all the previous successfully recognized gesture inputs as the new template

  14. SYSTEM & ALGORITHM DESIGNPROCESSING FLOW • Processing flow of Gesture Input Client

  15. OUTLINE • Background & Problem • Related works • System & Algorithm Design • Experiments • Conclusion

  16. EXPERIMENTSENVIRONMENT • Hardware • ThinkPad T420s: Core-i5 2520M / 4GB RAM • Wii Remote controller with the Nunchuk extension • Kinect for Windows sensor • Software • Windows 7 SP1 • Visual C++ 2012 • Kinect SDK 1.6 for Windows

  17. EXPERIMENTSDATASET • Eight two-handed gestures • Six individuals (two females, four males) • Five days • 2400 samples Horizontal Horizontal Vertical Vertical Zoom-In Zoom-Out Zoom-In Zoom-Out Rotate Left Rotate Right Push Forward Pull Back

  18. EXPERIMENTSRECOGNITION ACCURACY • Data from different days • Without template adaption: 6.7% rejection and 1.8% error in average • With template adaption: 4.4% rejection and 1.1% error in average

  19. EXPERIMENTSRECOGNITION ACCURACY • Data from the same day • Without template adaption: 1.7% rejection and 0.2% error in average • With template adaption: 1.7% rejection and 0.2% error in average

  20. EXPERIMENTSRECOGNITION ACCURACY • Comparison of error rate of using different input • Use only acceleration or skeleton: 5.6% in average • Use combination: 1.8% in average

  21. EXPERIMENTSONGOING ACCURACY • Recognition accuracy • 70% of input data with at least 85% accuracy rate • Recognition error • 70% of input data with at most 5% error rate

  22. EXPERIMENTSPRACTICAL EVALUATION • Tested in our SAGE-based tiled-display environment

  23. OUTLINE • Background & Problem • Related works • System & Algorithm Design • Experiments • Conclusion

  24. CONCLUSION • A personalized gesture interaction system • Use both acceleration data from Wii Remote and skeleton data from Kinect • A DTW based real-time gesture recognition algorithm • Ongoing gesture recognition support • Future work • Adding the user-identification feature

  25. THANK YOU!

More Related