150 likes | 163 Views
The Functional Space of an Activity Ashok Veeraraghavan , Rama Chellappa, Amit Roy-Chowdhury. Avinash Ravichandran. Motivation. Variability exists in activity across subjects and across instances. The Variability can be regarded as 2 types
E N D
The Functional Space of an ActivityAshok Veeraraghavan , Rama Chellappa, Amit Roy-Chowdhury Avinash Ravichandran
Motivation • Variability exists in activity across subjects and across instances. • The Variability can be regarded as 2 types • GlobalThese include changes in the frame rate and the overall duration of the activity • LocalThese include changes in the actual gait , i.e the leg moves a little faster or slower than a previous instance of walking for the same or different person • Ignoring this variability can lead to identical models having large distances in the modeling framework
Time warping for activities t ( ( ( ( ( ( ( ) ) ) ) ) ) ( ) [ [ ) ] ( ] [ ) [ ] ] ( ( ) ) ( ) ( ( ) ) f b f f f b l l b b l b l f h b l f f l d d f f b W P T T T T T i i i j i i i i i i 0 0 0 0 1 0 0 1 · · · · t t t t t t t t t t t t t t t t t t t t t a a w w - - : : s c p a a n - c o e p n r o o c a a a e r p e a e a a y r e m c v o m e o s c r s e y o w e r e a v r a p u e e u u n n c c o n o n e a w = ! ! = b b s a a a T ; ; ; ; ; ; . b Goal : Given multiple instances of a gait, to find a model that fits all of instances, and the time warping for each instance Model
Properties of the Function Space 1 ¡ ( ( ) ) [ ] ( ( ) ) [ ] ( ( [ ) ( ) ] ( ) ) ( ) f f f 8 f f f l f f ` l 8 f 8 d f 8 ` f f A A W W i i i 0 1 0 1 0 0 0 1 1 0 1 0 0 1 1 ¸ ¸ ¸ ¸ · t t t t t t t t t t t t 2 2 2 2 2 > 2 2 u u s c o n n o u : e s x a s n s c = ! ! = = ; , , , , ; ; ; ; ; ; ; A is convex Physical constraints reduce the space A to a subset of functions W Properties of W W is convex
Canonical Form 1 ¡ 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ( ( ( ( ) ) ( ( ( ( ) ) ) ( ) ) ( ) f ) ( ) f ( ( ) ) ( ( ( ( ) ) ( ) ) ) ) ( ) ( ( ( ( ( ) ( ( ( ( ) ) ( ( ) ) ( ( ) ( ) ) ( ) ) ) ( ) ( ) g g ( ) ( ) ) ) ( ( ) ) ) ) ( ) f f b f f f h f b f h l d b f h f l f f l l l l f f l l l l T L W W i 2 2 2 t t t t t t t t t t t t t t t t t t t t t t t t t t t t t t t t t t t t t t ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ + ¡ u u e e n s u a a u u e u u s u u a c n u a u u u a u r e e q u s v u a e n s = = = = = = = = = = = ( ) l l 1 1 1 1 i i 1 u ¡ u ; , ; ; 1 1 The model representation is not unique Let This is over come by picking a symmetric u,l , as the symmetric representation is unique Given a non symmetric representation, it can be converted into a symmetric form
Learning Model Parameters N N i i ^ ^ ^ 1 1 1 ¡ ¡ ¡ 1 1 = = f g P P ( ( ( ( ) ( ) ) ( ( ) ) ) ) ( ( ( ) ) ) ( ) ( ) ( ) ( ) ( ) ( ( ) ) b l ^ ^ b h l b b b f f f f l l h 8 8 h f f k l b k d f b h d h f b b f B G A W N i i i i i i i i i 0 0 1 1 t t t t t t t t t t t t t t t t t t t t t t t t t t t 2 2 a u u s v s u e m m n m n a n e p x r a r e a c a c z e a a o n s s a e w w e e m e p a n s o a s w u e m s w e e w n e e e n e w s u c a n a = = = = = = N N i i i i i i i i i i i 1 1 1 1 1 i i i i i s 1 1 N N = = ; , , , , ; ; , = = : : : : : : This is done using the dynamic time warping scheme (details on next slide) Symmetric Model
Dynamic Time warping • Used in speech recognition systems to warp instances to a template to learn a model • Works with the different spectra components, creating a vector valued signal from scalar speech signals • Based on dynamic programming, searching over a finite grid
Dynamic Time Warping ( ( ) ) ( ( ) ) Á Á k k Á Á k k 1 1 1 ¸ · + + ¡ x x x x • Constraints • Endpoint Constraint • Monotonicity Condition • Local Continuity Constraints • Global Path Constraint • Slope Weighting
Features for Activity Recognition C X 1 Z C I 1 ¡ = = k k k k k C X • Silhouette shape is used as a feature • Each silhouette has K landmark points, and the trajectory of these K points form a(t) • Invariant to scale and translation • Preshape lies on the unit sphere • Partial procrustes distance is used as a distance metric
Activity Recognition ^ ^ ^ ( ( ( ( ) ) ( ( ( ( ) ) ) ) f d d h h f f I i i i i t t t t t t a a r r g g m m n n s s a a = = f W N i i i i i 1 2 = ; ; s : : : i • Assumption : The test sequence is from one of the training classes • 100 sequences for 10 Activities. Training set was 90 sequences of 10 activities, Testing was done on the remaining 10. Best Warping Function Closest Fit to the model
Discussion • The concept of using DTW for gait is not new. • Previously DTW was used in a template matching framework where all training data is aligned to a common time frame • This works best when the testing data is among the training datasets, for new sequences they still suffer from the issue of time warping • This algorithm is feature independent, silhouettes can be replaced by intensities, joint location etc • Although the algorithm defines the space of all transformation, this is still dependent on the training dataset
Other aspects • ClusteringEM like framework to cluster the gaits, 92 % clustering accuracy with the n classes all representing different gaits • Database organization into a tree like structure to reduce the number of distance calculations