Recognizing Human Body Motion

1. Recognizing Human Body Motion 2002. 10. 31 MAI LAB Kong Jae Hyun MAI-LAB Seminar

2. Toward the automatic analysis of complex human body motion J. Rittscher*, A.Blake**, S.J. Roberts*** *GE Global Research, I Research Circle, Niskayna, NY 12308, USA **Microsoft Research, 7 JJ Thomson Avenue, Cambridge CB3 0FB, UK **University of Oxford, Park Road, Oxford OXI PJ3, UK Image and Vision Computing 20 (2002) 905-916 MAI-LAB Seminar

3. Contents • Introduction • Methodology • Experiments and Results • Conclusion MAI-LAB Seminar

4. Introduction(1/3) • The interpretation of human action plays a vital role for intelligent environments and surveillance application • Surveillance application has to be instantaneous • classifying the type of motion without having to estimate the pose • Use a low dimensional representation MAI-LAB Seminar

5. Introduction(2/3) Simultaneously perceive and classify Classify directly from the set of spatio-temporal feature Use the statistical models Epipolar plane image analysis Autoregressive Process (ARP) Condensation filtering Treat the vector of the skew Factor(distribution of velocity) as a feature vector MAI-LAB Seminar

6. Introduction(3/3) HMMs Complex Movement Sequences Dynamic Programming Simple Dynamical Categories Linear Dynamical Models Q(t+1)=A1Q(t) Q(t+1)=A2Q(t) Kalman-Filter Q(1), Q(2), Q(3), Q(4),… Mixture of Gaussians Coherence Blob Hypotheses EM clustering XYT Gradients HSV, Texture Input Image Sequence ( Source : C. Bregler, Learning and Recognition Human Dynamics in Video Sequence, Proceeding of 11th IEEE computer vision and pattern recognition, 1997) MAI-LAB Seminar

7. Methodology(1/9) Simultaneous perception and classification 기본적인 아이디어 • 시간 t에서의 image의 spline contour로 부터 continuous state vector x를 정의 • Motion의 클래스를 discrete vector y로 정의 • Mixed state Xt=(xt,yt)T로 정의 • 새로운 상태 Zt={z0,...,zt} 의 입력이 주어지면 • p(Xt|Zt)의 확률 분포를 구해내고 classification MAI-LAB Seminar

8. Methodology(2/9) Simultaneous perception and classification Autoregressive Process • A linear-Gaussian Markov model of order K • Each class y has a set (Ay,By,dy) • P(yt=y'|yt-1=y)=M(y,y') • p(zt|xt) is taken to be Gaussian in experiment MAI-LAB Seminar

9. Methodology(3/9) Simultaneous perception and classification The mixed-state Condensation algorithm 1. Select a sample st‘(n) = (xt‘(n), i) 2. Predict by sampling form p(Xt | Xt-1 = st‘(n) ) to choose st(n) =(xt(n) , yt(n)) (a) Sample transition properties p(yt(n)=j | Xt-1 = st‘(n) )=Tij(xt‘(n)) (b) Sample sub-process density p(xt(n)|Xt-1 = st‘(n) , yt(n)=j )=pij(xt|xt‘(n)) 3. Measure and weight the new position in terms of the image data Zt wt(n) =p(Zt | Xt = st(n) ) ( Source : M. Isard, A. Blake, A mixed-state condensation tracker with automatic model-switching, Proceeding of Sixth ICCV, 1998) MAI-LAB Seminar

10. Methodology(4/9) Simultaneous perception and classification Motion models for classification • The motion model need to be finely tuned • Transition matrix M 은 motion을 인식하는 prior로 사용됨 • 따라서 Transition matrix M과 p(X)를 여러 가지 실험을 통해 잘 조절 해야 한다. MAI-LAB Seminar

11. Methodology(5/9) Simultaneous perception and classification Partial importance sampling • 하나의 동작에서 다음 동작으로 넘어가는 경우에 선행 동작의 각 particle을 모두 평가하는 것은 cost의 낭비 • e.g. 한 동작이 2초 동안 일어나고 video field rate가 50Hz인 경우 particle의 수는 100가지 이지만 실제로 동작이 바뀌는 부분의 하나의 particle만 있으면 된다. • partial importance sampling 필요 • Importance sampling functio g는 다음과 같이 나타남 • gt(Xt|Xt-1)=p(xt|xt-1,yt)P(yt|yy-1) • ( importance transition matrix G, P(yt|yy-1)=G(yt-1,yt) ) MAI-LAB Seminar

12. Methodology(6/9) Classifying motions directly from spatio-temporal features Epipolar plane image analysis • motion의 type을 분류하는데 epipolar slice를 이용 • epipolar slice의 pattern으로부터 특성을 찾아내고 model을 디자인 • running, walking, skipping과 같은 경우 그림과 같은 braided pattern 을 보임 MAI-LAB Seminar

13. green line (v=33) blue line (v=92) red line (v=189) Methodology(7/9) Classifying motions directly from spatio-temporal features Epipolar plane image analysis Source : http://graphics.lcs.mit.edu MAI-LAB Seminar

14. Methodology(8/9) Classifying motions directly from spatio-temporal features Epipolar plane image analysis • Epipolar slice에서 걷는 속도의 measure인 를 구한다. • 확률 분포 를 구한다. • 분포의 skewness 을 구한다. • 각 y에 따른 의 집합 • 을 바탕으로 classification 실시 MAI-LAB Seminar

15. Methodology(9/9) Classifying motions directly from spatio-temporal features Epipolar plane image analysis MAI-LAB Seminar

16. Experiments and Results(1/4) Simultaneous perception and classification MAI-LAB Seminar

17. Experiments and Results(2/4) Classifying motions directly from spatio-temporal features Running Skipping Walking Linear fisher discriminant 이용 MAI-LAB Seminar

18. Experiments and Results(3/4) Classifying motions directly from spatio-temporal features Jumping Half star Star-Jump MAI-LAB Seminar

19. Experiments and Results(4/4) Classifying motions directly from spatio-temporal features Running Walking turningthrowing Misclassification rate : 10 % MAI-LAB Seminar

20. Conclusion • 실시간으로 motion을 분류할 수 있는 도구가 필요 • 이전의 연구들은 계산 시간이 너무 오래 걸리 거나 단순한 동작만 인식하기 때문에 좋지 않음 • 이 연구는 적은 계산으로 좋은 분류 결과를 보임 • view point에 대해 독립적인 분류 방법이 필요 MAI-LAB Seminar