A Layered Deformable Model for Gait Analysis

1. A Layered Deformable Model for Gait Analysis Haiping Lu, K.N. Plataniotis and A.N. Venetsanopoulos The Edward S. Rogers Sr. Department of Electrical and Computer Engineering University of Toronto International Conference on Automatic Face and Gesture Recognition, 2006

2. Outline • Motivation • Overview • The layered deformable model (LDM) • LDM body pose recovery • Experimental results • Conclusions FG2006, Southampton, UK

3. Motivation • Automated Human identification at a distance • Visual surveillance and monitoring applications • Banks, parking lots, airports, etc. • USF HumanID Gait Challenge problem • Articulated human body model for gait recognition • Manually labeled silhouettes • Layered, deformable FG2006, Southampton, UK

4. Overview Manual labeling LDM recovery Automatic extraction LDM recovery FG2006, Southampton, UK

5. The Layered Deformable Model (LDM) • Trade-off: • Complexity Vs. descriptiveness • Match manual labeling: • Close to human’s subjective perception • Assumptions: • Fronto-parallel, from right to left. FG2006, Southampton, UK

6. LDM – 22 Parameters • Ten segments • Static: • Lengths (6) • Widths (3) • Dynamic • Positions (4) • Angles (9) FG2006, Southampton, UK

7. LDM –Layers and deformation • Four layers • Deformation: FG2006, Southampton, UK

8. LDM – Summary • Summary: Realistic with moderate complexity • Compact: 13 dynamic parameters • Layered: model self-occlusion • Deformable: realistic limbs • Resemblance to manual labeling FG2006, Southampton, UK

9. Manual silhouettes pose estimation(ground truth & statistics) • Limb joint angles: • Reliable edge orientation • Spatial–Orientation mean-shift (mode-seeking): dominant modes  limb orientation • Others: • Joint positions, limb widths and lengths • Simple geometry • Torso: bounding box: • Head: “head top” and “front face” FG2006, Southampton, UK

10. Post-processing • Human body constraints: • Parameter variation limits • Limb angles inter-dependency • Temporal smoothing • Moving average filtering FG2006, Southampton, UK

11. Automatic pose estimation • Silhouette extraction (ICME06, Lu, et al.) • Static parameters • Coarse estimations: statistics from Gallery set • Silhouette information extraction based on ideal human proportion: • Height, head and waist center, joint spatial-orientation domain modes of limbs FG2006, Southampton, UK

12. Ideal proportion of the human eight-head-high figure in drawing FG2006, Southampton, UK

13. Automatic pose estimation • Dynamic parameters: • Geometry on static parameters and silhouette information, constraints. • Limb switching detection • Thighs & lower legs: variations of angles. • Arms: opposite of thighs • Frames between successive switch • Post-processing: smoothing FG2006, Southampton, UK

14. Experimental results • 285 sequences from five data sets, one gait cycle each sequence. • Imperfection due to silhouette extraction noise and estimation algorithm • Feedback LDM recovery to silhouette extraction process may help. FG2006, Southampton, UK

15. LDM recovery results • Raw • LDM manual • LDM auto FG2006, Southampton, UK

16. LDM recovery example (revisit) Manual labeling LDM recovery Silhouette extraction LDM recovery FG2006, Southampton, UK

17. Angle estimation – left & right thighs From manual silhouettes From automatically extracted silhouettes FG2006, Southampton, UK

18. Error rate (in percentage) for lower limb angles FG2006, Southampton, UK

19. Conclusions • A layered deformable model for gait analysis • 13 Dynamic and 9 static parameters • Body pose recovery from manual (ground truth) and automatically extracted silhouettes. • Average error rate for lower limb angles: 7% • Overall: close match to manual labeling, accurate & efficient model for gait analysis • Future work: model-based gait recognition FG2006, Southampton, UK

20. Acknowledgement • Thanks Prof. Sarkar from the University of South Florida (USF) for providing the manual silhouettes and Gait Challenge data sets. FG2006, Southampton, UK