Video Based Gait Analysis in Biometric person authentication

1. Video Based Gait Analysis in Biometric person authentication • Jani Rönkkönen

2. Gait in general • Shortly gait means the walking style of a person • Gait signature of each person is unique and thus can be used as a biometric • To form a gait signature many different components like cadence or frequency of walking can be used • Most common way of receiving gait information is by video cameras (others for example radar, pressure mat or motion sensors)

3. Current status • Quite a lot of published articles in recent years • Research is still mostly basic research • No commercial solutions yet for authentication purposes (May be some medical applications) • Most promising areas are medical and surveillance applications • Georgia Institute of Technology is developing a method for recognizing people among crowd and estimate it could be commercialised in five years

4. Advantages • Can be used with low resolution video sequences • Target do not necessarily need to know about the surveillance • Sequences can be taken from long distance • Nonintrusive • Gait is not very easily conceivable biometric (although can be altered purposely)

5. Disadvantages • The uniqueness of a persons gait signature is not proven with large datasets • Not yet clear which components of gait signature are most useful • A lot of data usually means high computational cost • Gait may be changed purposely • Conditions may affect the gait signature more than differences between subjects

6. Conditions • Walking speed • Affects cadence, stride lenght, frequency, pose and hand swings • Walking surface • If surface is not smooth and obstacle free gait pattern will no longer be repeatable an periodic • Physical Conditions • For example pregnancy, drunkness, fatigue or physical injury

7. Conditions • Carrying a load • Carrying a load affects both the gait dynamics and physical borders of a person • Clothes • Clothes alter the borders of a person and may hide some movement (a dress for example) • footwear affect the gait dynamics (rubber boots vs high heels)

8. Camera conditions • Camera angle • The gait pattern is very different if looked from different angles • Lightning conditions • Shadows cause error in border or silhouette extraction • Contrast between clothing and backround • Too small contrast makes it harder to extract borders or sihouette

9. History • G. Johansson used small light bulbs attached to a person to generate 2D motion patterns in 1973 • He found out that people could recognize these to represent human movement • Later it was discovered that humans could also recognize the gender of the walker or even their friends identity from these patterns • A conclusion was drawn that gait could be used as a biometric

10. UMD database • University of Maryland (UMD) database • First dataset • 25 persons and 4 camera angles • Second dataset • 55 persons and 2 camera angles

11. CMU database • Carnegie Mellon University (CMU) database • 25 persons and 6 camera angles • Slow, fast and inclined walking style • A walk holding a ball

12. USF and USH databases • University of South Florida (USF) database • 71 persons and 2 camera angles • 2 different shoetypes and walking surfaces (concrete and grass) • Walk holding a briefcase • University of Southampton (USH) database • 28 persons and 1 camera angle • Uniform green backround to help extract clean silhouettes

13. Features • Gait signature includes numerous components and even more features can be derived from these • Not yet clear which features are most usefulfor authentication purposes • Why not just simply use all there is?

14. Why not all features? • Would be computationally costly, need much storage space and need complex algorithms • Conditions affect different features differently • Some are more robust to changes • Using more features may decrease performance • Bad ones only add unwanted noise • All features are not always available

15. Methods • Another question is how to use our feature(s) of choice for authentication? • Typically there is three main steps in a gait recognition algorithm: • Extracting the subject from the frames of video sequence (eg. silhouette) • Extracting and modifying the wanted features (eg. PCA to simplify the data) • Classification based on extracted features and somekind of decision (eg. KNN-classifier)

16. The width of outer contour • The basic biometric is the width of the outer contour of binarized silhouette of a walking person • Retains physical structure and swings of the limbs during walking • The pose information is lost • Smoothed and down sampled width vectors are used directly • Also a velocity profile is extrated by calculating the difference of subsequent vidth vectors

17. Overlay of width vectors

18. Results • UMD database • rank 1: 80% and rank 5: 91.2% • Velocity profile alone • rank 1: 56% and rank 5: 83% • CMU database • rank 1: over 95% and rank 5: 100% • Fast vs slow • rank 1: 75% and rank 5: 87.5%

19. Results with USF database G = Grass C = Concrete A, B = Footwear R, L = Camera angle

20. Notes • Both structural and dynamical information is important fo recognition • Leg region is the most important • Difference in walking surface causes a lot of problems to the method • Walking speed is also an issue

21. Moments from silhouette • Silhouette is divided in 7 parts • For each part an ellipse is fitted • Features: • Centroid, Major axis, minor axis and Major axis orientation • Height of the body • Another testset used gait spectral component features received via fourier transform

22. Divided silhouette

23. Results • Tested on dataset consisting of 24 persons • Sequences were taken in 4 different days • Sequences of one day were compared against other days • Rank 1: 30% - 47% and rank 10%: 53%-94% • With spectral component features • Rank 1: 31% - 82% and rank 10%: 70%-97%

24. Results • Also tested with CMU database • Results were almost perfect (only one mistake) • Third case was gender identification • Support vector machines • Best results using second degree polynomial kernel 94% correctness

25. Notes • Most errors caused by clothing changes • Spectral component features were more robust • If only sequences taken in a same day were compared, spectral component features were slightly worse • Notably good result of gender classification

26. Body shape and gait from silhouette • The periodic dimensional changes in silhouette width are used to locate the key frames • Key frames are compared to corresponding ones in training data • Four subsequent comparison scores are amalgated and used for classification

27. Extracting the key frames

28. Results • CMU database • Rank 1: at least 92% • Slow vs fast • rank 1: 76% and rank 10%: 92% • Second testset contained 25 persons with sequences taken in different days • rank 1: 45% and rank 10%: 77%

29. Notes • The main reason for failures (according to the author) were conditions that affected the quality of the silhouette • Lightning conditions • Clothes • Hairstyles

30. Stride length and cadence • The method makes following assumptions: • Walking velocity is constant • Persons walks a straight line for 10-15 seconds • Camera is calibrated with respect to the ground plane • Frame rate is greater than twice the walking frequency

31. Stride lenght and cadence • The key is the periodicity of human walk • The width of a sihouette is used to calculate the period • From period a number of steps can be calculated • Also the distance walked is measured • Stride = distance/steps • Cadence = steps/time

32. Results • A database of 131 sequences from 17 persons

33. Notes • The method (according to author) is robust to changes in: • Lightning conditions • Clothing • Tracking error • Also it is in principle wiev invariant, but the method used to calculate the frequency works best with fronto parallel sequences

34. Similarity plots • A sequence of images of a walking person is mapped to a similarity plot (SP)

35. Similarity plots • Each pixel in SP is a result of substracting two blobs • SP has dark main diagonal, because comparing a blob to itself results to zero • Is symmetric along main dianonal • Is periodic, because from key poses A and C and B and D are close to each other

36. Results • A dataset of 44 images of 6 persons from one camera angle • 90% accuracy with rank 1 • The second dataset consisted of 400 sequences of 7 peoples from 8 camera angles and 7 different days • 65% accuracy with rank 1

37. Notes • The method is view dependent and performs best with fronto-parallel sequences • Changes in clothing and lightning affect performance • The author used binary blobs and gray level blobs with and without backround • Best results with binary blobs, worst with gray level blobs with backround

38. Thigh and lower leg rotation • A sobel edge operator is used to obtain the leading edge of a walking person

39. The model • A model is then matched to the edge

40. Phase weighted magnitude • A phase weighted magnitude (PWM) is calculated from this with the help of fourier transform

41. Results • A dataset of 20 persons walkin and running • In addition to clean edged images, 25% grey scale random noise were added and also classification was done by decreasing the resolution (from 130*190 pixels to 65*95 pixels) • Best results were achieved with clean edged running sequences rank 1: 91.7% and worst with noisy walking sequences 60.8%

42. Notes • Reducing resolution did not reduce performance dramatically • Adding noise reduced to worse results • The reson for running sequences having slightly better identification accuracy was according to author the fact that there is more differeces between humans running styles than walking styles

43. Conclusions • Rather good results of identification can already be acquired with small datasets and fixed conditions • More robust methods are needed to achieve better accuracy in more general setup • A question still remains if gait is unique enough with larger datasets