Recognition and tracking of human body parts

1. Recognition and tracking of human body parts AlgirdasBeinaravičius Gediminas Mazrimas Salman Mosslem

2. Contents • Introduction • Background subtraction techniques • Image segmentation • Color spaces • Clustering • Blobs • Body part recognition • Problems and conclusion

3. Introduction. Project tasks • Background subtraction/Foreground extraction • Color spaces and K-Means clustering • Blob-level introduction • Body part recognition

4. Background subtraction • What is background subtraction? • Background subtraction models: • Gaussian model • “Codebook” model

5. Background subtractionGaussian model • Learning the model • Gaussian parameters estimation • Thresholds - Foreground/Background determination

6. Background subtraction“Codebook” model

7. Background subtractionModel comparison Original image Background subtraction using Gaussian model Background subtraction using Codebook model

8. Image segmentation • Color spaces • RGB • HSI • I3 (Ohta) • YCC (LumaChroma) • Clustering • K-Means • Markov Random Field

9. Image segmentationColor space: RGB • RGB (Red Green Blue) • Classical color space • 3 color channels (0-255) • In this project: • Used in background subtraction

10. Image segmentationColor space: HSI • HSI (Hue Saturation Intensity/Lightness) • Similar to HSV (Hue Saturation Value) • 3 color channels: • Hue – color itself • Saturation – color pureness • Intensity – color brightness • Converted from normalized RGB values • Intensity significance minimized • In this project: • Used in clustering • Blob formation • Body part recognition

11. Image segmentationClustering • Image data (pixels) classification to distinct partitions (labeling problem) • Color space importance in clustering

12. Image segmentationClustering: K-Means • Clustering without any prior knowledge • Working only with foreground image • Totally Kclusters • Classification based on cluster centroid and pixel value comparison • Euclidean distance: • Mahalanobis distance:

13. Image segmentationClustering: K-Means example

14. Image segmentationClustering: K-Means Euclidean/Mahalanobis distance comparison Euclidean distance Mahalanobis distance

15. Image segmentationClustering: K-Means color space comparison RGB HSI

16. Image segmentationClustering: MRF • Probabilistic graphical model using prior knowledge • Usage: • Pixel-level • Blob level • Concepts from MRF: • Neighborhood system • Cliques

17. Image segmentationClustering: MRF Neighborhood system Cliques

18. Blobs • Blob parameters • Blob formation • Blob fusion conditions • Blob fusion

19. Blobs • Higher level of abstraction • Ability to identify body parts • Faster processing

20. BlobsParameters • Label. • Set of area pixels. • Centroid. • Mean color value. • Set of pixels, forming convex hull. • Set of neighboring blobs. • Skin flag.

21. BlobsInitial creation • Input: K-means image/matrix. • Output: Set of blobs

22. BlobsSkin blobs • Particularly important in human body part recognition. • Can not be fused. • Technique to identify skin blobs: • Euclidean distance

23. BlobsFusion • Conditions: • Blobs have to be neighbors • Blobs have to share a large border ratio • Blobs have to be of similar color • Small blobs are fused to their largest neighbor • Neither of these conditions apply to skin blobs

24. Body part recognition (I) • Associate blobs to body parts

25. Body part recognition (II) • Skin blobs play the key role: • Head and Upper body: • Torso identification • Face and hands identification • Lower body: • Legs and feet identification

26. Body part recognition (III)

27. Problems (I) • Computational time • Background subtraction quality • Subject clothing • Subject position • Number of clusters in K-Means algorithm • Skin blobs

28. Problems (II)

29. Problems (III)

30. Conclusion and future work • Main tasks completed • Improvements are required for better results • Possible future work: • Multiple people tracking • Detailed body part recognition

31. Questions, comments ?