A Discriminative Key Pose Sequence Model for Recognizing Human Interactions

1. A Discriminative Key Pose Sequence Model for Recognizing Human Interactions Arash Vahdat, Bo Gao, Mani Ranjbar, and Greg Mori ICCV2011

2. Goal

3. Outline • Introduction • Related methods • Modeling Human Interactions • Single Subject Key Pose Sequence Model • Interaction Key Pose Sequence Model • Learning the parameters • Experiments

4. Introduction • This paper focuses on recognizing interactions between individuals. • The sequences of key poses between peoples will be combined into activity level. • The activities to be recognized include hugging ,shaking hands,pointing, punching, kicking,pushing each others.

5. Introduction • Not every movement or pose by the target is relevant to the activity to be recognized. • We use an examplar-based model by giving the pose a score to match the key poses. • When people do some activities, they will do the key poses in a chronological order.

6. Related methods • Ryoo and Aggarwal [13]develop a matching kernel that considers spatial and temporal relations between space-time interest points. • Yao et al. [23] use a Hough transform voting scheme from an interest point representation. [13] M. Ryoo and J. Aggarwal. Spatio-temporal relationship match: Video structure comparison for recognition of complex human activities. In ICCV, 2009. [23] A. Yao, J. Gall, and L. Van Gool. A hough transform-based voting framework for action recognition. In CVPR, 2010. 2, 6

7. Recognition result in [13]

8. Modeling Human Interactions • There are four things to know: • 1. Who is involved in the interaction? • Subject or object • 2. When do the key poses occur? • The interval of the key poses • 3. How are the key poses executed? • With hand or leg , powerful or weak • 4. Where are the people when the key poses occur?

9. Modeling Human Interactions • we will assume F maximizes a model G that includes the latent variables H: • The variables H are the answer of the four questions above.

10. Single Subject Key Pose Sequence Model

11. Single Subject Key Pose Sequence Model • We represent each key pose by h: • Denote K key poses of a sequense by H

12. Single Subject Key Pose Sequence Model • Exemplar Matching Link: • Compute the pose connection strength to the exemplar poses.

13. Single Subject Key Pose Sequence Model • Activity-Key Pose Link: • Compute the sequence similarity in the activity to give a score. • Direct Root Model:

14. Interaction Key Pose Sequence Model • Update the model from individual to two people interaction. • We should recognize who is subject or object. • The scoring function will be:

15. Learning the parameters • Define the scoring function E(x,y): • Use multiclass linear SVM classifier to find the best parameters.

16. Experiments • The UT-Interaction dataset contains videos of 6 classes of human-human interactions. • Set 1 is with a stationary background,and Set 2 is with slight background movement and camera jitter.

17. Experiments

18. Experiments

19. Experiments

20. Experiments

21. Conclusion • This paper focuses on the key poses method to recognize the interaction. • The precision is over 90% and outperform other methods in UT-interaction dataset.