1 / 36

One-Shot Learning Gesture Recognition

One-Shot Learning Gesture Recognition. Students: Itay Hubara Amit Nishry Supervisor: Maayan Harel Gal-On. Outline. Background. Gesture recognition is a strong upcoming field in computer vision

cruz
Download Presentation

One-Shot Learning Gesture Recognition

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. One-Shot LearningGesture Recognition Students: ItayHubara AmitNishry Supervisor: MaayanHarel Gal-On

  2. Outline

  3. Background Gesture recognition is a strong upcoming field in computer vision Gesture recognition can be seen as a way for computers to begin to understand human body language

  4. Goals Learn and understand existing Gesture recognition algorithms. Compare different approaches Design Gesture recognition algorithm which reduces training time

  5. Data • The Data is compose from several set each contains: • Gesture vocabulary (learning set) which contain only one sample per gesture. • Test set which contain one or more gestures. • Each of the sets has different vocabulary features such as large/small gesture hand/legs movement etc.

  6. Data – Train Base gesture

  7. Data – Test Multiple base gestures Large movements

  8. Data - Test Multiple base gestures Small movements

  9. Challenges • One shot learning - only one learning sample (unlike the common approach of multi class classification) • Tests videos segmentation • Same gesture can have different number of frames • Each set has different features (small/big gestures)

  10. Outline

  11. Reduced Problem • Assume that each of the test movies has only one gesture • Goal:finding features space and distance function which have good separation of the features space

  12. Features • Motion Energy • subtracting consecutive frames • Space Quantization

  13. Features • Harris Corner Detector • Find interest point in the difference image based on corner detection • Space Time Interest Points • Extend Harris to the time domain

  14. Features Harris STIP

  15. Features • Head Relative Interest Points

  16. Features Interest points Head Histogram

  17. Distance Functions • Good features space is defined not only by the features but also by the distance (similarity) function • Different features need different distance functions

  18. Principal Motion Using PCA • Using principal component analysis (PCA), to find the main motion vectors. • For test set - project feature onto each of train principals and evaluate similarity

  19. Earth Moving Distance • Given two sets of distribution, EMD will measure the minimum cost to shift “dirt” from one distribution to the other.

  20. Perturbed Variations • Given two sets of distribution and predefined value of permitted variations optimally perturbs the distribution to best fit each other. Transportation problem under permitted variations constrain

  21. LevenshteinDistance • Measure the difference between two sequences. • Consider lengths and classification.

  22. Results Top 20 Top 10

  23. Results

  24. Outline

  25. Complete Problem Separate problems • Basic Segmentation (equal/movement) Whole problem solving approach • Moving Window • Dynamic Time Warping (DTW)

  26. Moving Window • Move a window along the test video. • Assume each window frames has only one gesture • Preform basic analysis as did before to and build the distance matrix

  27. Dynamic Time Warping • Create a state machine from train data: • Module standing position • Form standing position can move to start of base gestures • Assume we can move forward, or stay in the same sate. • For a given gesture – find the best path along the sate machine

  28. Dynamic Time Warping

  29. Results

  30. Results

  31. Results Top 20 Top 10

  32. Outline

  33. Conclusions Each approach receive better results in different feature and similarity function Different algorithms has different strengths (segmentation\recognition) Segmentation require standing position model.

  34. Conclusions • Pre-processing unsupervised algorithms help better representing the data. • There is still allot left to do on the field

  35. Future Work Try different models for the standing position to improve segmentation results Try combing DTW for segmentation and PCA for recognition. Use different unsupervised algorithms to better represent the data.

  36. References Ivan Laptev, "On Space-Time Interest Points”, 2005 Hugo JairEscalantea and Isabelle Guyonb, "Principal motion: PCA-based reconstruction of motion histograms” M.Harel, S.Manor, "The Perturbed Variation”, NIPS 2012 ElizavetaLevina, Peter Bickel Department of Statistics, “The EarthMover’s Distance is the Mallows Distance: Some Insights from Statistics”. OfirPele,MichaelWerman, “Fast and Robust Earth Mover’s Distances”.2008

More Related