Robust Video Surveillance for Fall Detection Based on Human Shape Deformation

1. Caroline Rougier, Jean Meunier, Alain St-Arnaud, and Jacqueline Rousseau IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 21, NO. 5, MAY 2011 Robust Video Surveillance for Fall Detection Based on Human Shape Deformation

2. outline • Introduction • Our System and data set • Falls Characteristics • Shape deformation • mean matching cost • full Procrustesdistance • Fall Detection Using GMM • Experimental Results • Conclusion

3. Introduction (1/2) • Establish new healthcare systems to ensure the safety of elderly people at home. • Falls are one of the major risks for old people living alone. • Fall detection wearable sensor: • Accelerometers or help buttons Problem: -forget to wear -unconscious after the fall -recharged regularly

4. Introduction (2/2) • Computer vision systems has overcome these problems. • A camera provides a vast amount of information on his/her environment • Monocular Systems • Bounding box[8] • Only placed sideways • Occluding objects • Multi-Camera Systems • Auvinet et al.[17] reconstructed 3-D silhouette of an elderly person • Need to be calibrated • The video sequences need to be synchronize

5. Our System and data set (1/2) • Uncalibratedmulti-camera system • Low-cost IP cameras, 30 frames/s, 720 × 480 pixels • Wide angle to cover all the room

6. Our System and data set Total of 75 different events , more than 12 min

7. Falls Characteristics • Lack of significativemovement • A lying position • A person lying on the ground • Vertical speed • An impact shock • Body shape change

8. Silhouette Edge Point Extraction • The silhouette is extracted by a background subtraction • N = 250 landmarks * Canny edge detector[25]

9. Matching Using Shape Context (1/2) • Shape context[20] is a way of describing shapes. Matching cost for pair (pi, qj): , K=5*12 bins

10. Matching Using Shape Context (2/2) • Minimizing the total matching cost given a permutation π (i) • Use the Hungarian algorithm[27] for bipartite matching • Time complexity: O(n^3) • Bad landmarks due to segmentation errors or partial occlusions • Add dummy points (not easy to choose). • Match only the most reliable points in our implement (miniCij= minjCij) • mean matching cost: bipartite graph i j N∗: the total number of best matching points.

11. Procrustes analysis • Procrustes analysis [21] has been widely used to compare shapes. • Detect abnormal shape deformation for fall detection • Step1 : image registration(one translation, no rotation, no scaling) • Step2: Compute full Procrustes distancefor compare. two centered vectors : v = (v1, · · · , vk) w = (w1, · · ·,wk). Z centered landmarks Zc: full Procrustesdistance : Zc

12. Fall feature • mean matching cost • full Procrustesdistance • Consider 2 feature (F1, F2) 1) F1 representing the fall : F1 will high in case of fall 2) F2 representing the lack of significative movement after the fall : A period (t+1s to 5s) will low

13. Fall Detection Using GMM • Model normal activity data with a Gaussian Mixture Model(GMM). • GMM: weighted sum of Gaussian(normal) distributions • M :the number of components in the mixture • P (j) : the mixing coefficients • The jth Gaussian probability density function p (x | j) • d is the dimensionality of the input space expectation-maximization (EM) algorithm by maximizing the data likelihood GMM Classifier : only tell normal or abnormal!

14. Training and test the dataset • Leave-One-Out Cross-Validation • Divided the dataset into N video sequences • One sequence is removed • Training using the N − 1 remaining sequences (falls are deleted) • This sequence is classified with the resulting GMM. • Repeat N times • Count the number of errors, classification error rate

15. GMM Classifier Analysis • True Positives (TP): falls correctly detected; • False Negatives (FN): falls not detected; • False Positives (FP): normal activities detected as a fall; • True Negatives (TN): normal activities not detected as a fall; • Sensitivity: Se = TP/ (TP + FN); • Specificity: Sp= TN/ (TN + FP); • Accuracy: Ac = (TP+TN) / (TP+TN+FP+FN) ; • Classification error rate: Er= (FN+FP) / (TP+TN+FP+FN) .

16. Experimental Results • Shape matching : C++ using the OpenCV library [33] • Fall detection : MATLAB using the NETLAB toolbox [32] to perform the GMM classification. • The original video sequences frame : 30 frames/s • 5 frames/s was sufficient to detect a fall • Intel Core 2 Duo processor (2.4 GHz) • The computational time of the shape matching step is about 200 ms

17. Number of GMM Components • train a GMM with three components for our experiment.

18. Classification Results • Normalize training data. • Detection threshold depends on the sensitivity.

19. Receiver operating characteristic (ROC) analysis true positives false positives

20. Ensemble Classifier • Simply majority vote on all cameras (>= 3 vote) • In fig. 9 : error rate 10%2.7%

21. Comparative Study with Other 2-D Features (1/2)

22. Comparative Study with Other 2-D Features (2/2)

23. Occlusions and Other Difficulties

24. Conclusion • We presented a new GMM classification method to detect falls • By analyzing human shape deformation • Robust to large occlusions and other segmentation difficulties