Pedestrian Detection in Crowded Scenes

1. Pedestrian Detection in Crowded Scenes Dhruv Batra ECE CMU

2. Pedestrian Detection in Crowded Scenes Pedestrian Detection in Crowded Scenes. Bastian Leibe, Edgar Seemann, and Bernt Schiele. In IEEE International Conference on Computer Vision and Pattern Recognition (CVPR'05), San Diego, CA, June 2005. An Evaluation of Local Shape-Based Features for Pedestrian Detection. Edgar Seemann, Bastian Leibe, Krystian Mikolajczyk, and Bernt Schiele. In British Machine Vision Conference (BMVC'05) Oxford, UK, September 2005. Combined Object Categorization and Segmentation with an Implicit Shape Model. Bastian Leibe, Ales Leonardis, and Bernt Schiele. In ECCV'04 Workshop on Statistical Learning in Computer Vision, Prague, May 2004.

3. Theme of the Paper • Probabilistic top-down/bottom-up formulation of segmentation/recognition • Basic Premise: “[Such a] problem is too difficult for any type of feature or model alone”

4. Theme of the Paper • Open Question: How would you do pedestrian detection/segmentation? • Solution: integrate as many cues as possible from many sources Original image Support of Segmentation from local features Support of segmentation from global features (Chamfer Matching) Segmentation from local features Segmentation from global features (Chamfer Matching)

5. Theme of the Paper • Goal: Localize AND count pedestrians in a given image • Datasets Training Set: 35 people walking parallel to the image plane Testing Set (Much harder!): 209 images of 595 annotated pedestrians

6. Theme of the Paper

7. Evaluation Criteria • Criteria 1: Relative Distance Threshold d < 0.5 Fixed aspect ratio- 11:15

8. Evaluation Criteria • Criteria 2 & 3: Cover and Overlap Threshold cover >50% overlap >50%

9. Initial Recognition Approach • First Step: Generate hypotheses from local features (Intrinsic Shape Models) • Training: • Code book Approach (with spatial information)

10. Initial Recognition Approach • First Step: Generate hypotheses from local features (Intrinsic Shape Models) • Training: Lowe’s DoG Detector 3s x 3s patches Resize to 25 x 25

11. Initial Recognition Approach • First Step: Generate hypotheses from local features (Intrinsic Shape Models) • Training: • Agglomerative Clustering

12. Initial Recognition Approach • First Step: Generate hypotheses from local features (Intrinsic Shape Models) • Training: • Agglomerative Clustering

13. Initial Recognition Approach • First Step: Generate hypotheses from local features (Intrinsic Shape Models) • Training: • Agglomerative Clustering

14. Initial Recognition Approach • First Step: Generate hypotheses from local features (Intrinsic Shape Models) • Training: • Agglomerative Clustering store figure-ground masks for these entries Codebook entries

15. Initial Recognition Approach • First Step: Generate hypotheses from local features (Intrinsic Shape Models) • Training: • But wait! We just lost spatial information … Run again Lowe’s DoG Detector 3s x 3s patches Resize to 25 x 25 Learn Spatial Distribution Find codebook patches

16. Initial Recognition Approach • First Step: Generate hypotheses from local features (Intrinsic Shape Models) • Testing: • Initial Hypothesis: Overall

17. Initial Recognition Approach • First Step: Generate hypotheses from local features (Intrinsic Shape Models) • Testing: • Initial Hypothesis (Probabilistic Hough Voting Procedure) learnt from spatial distributions of codebook entries measuring similarity between patch and codebook entry Search for maximum in probability space Using a fixed size search window

18. Initial Recognition Approach • First Step: Generate hypotheses from local features (Intrinsic Shape Models) • Testing: • Initial Hypothesis: found as maxima in 3D voting space maxima computed using Mean Shift Mode Estimation over this balloon density estimator Uniform Cubicle Kernel

19. Initial Recognition Approach • First Step: Generate hypotheses from local features (Intrinsic Shape Models) • Testing: • Initial Hypothesis: Overall

20. Initial Recognition Approach • First Step: Generate hypotheses from local features (Intrinsic Shape Models) • Testing: • Initial Hypothesis: Overall

21. Initial Recognition Approach • First Step: Generate hypotheses from local features (Intrinsic Shape Models) • Testing: • Probabilistic top down segmentation Estimate from training data From similarity measure Assumption: Uniform Priors start here Intermediate Goal: Find this

22. Initial Recognition Approach • First Step: Generate hypotheses from local features (Intrinsic Shape Models) • Testing: • Probabilistic top down segmentation Substitute this here to get this Marginalized over all patches in image

23. Initial Recognition Approach • First Step: Generate hypotheses from local features (Intrinsic Shape Models) • Testing: • Probabilistic top down segmentation

24. Initial Recognition Approach • First Step: Generate hypotheses from local features (Intrinsic Shape Models) • Testing: • Probabilistic top down segmentation

25. Initial Recognition Approach • Second Step: Handling overlapping detections

26. Initial Recognition Approach • Second Step: Segmentation based Verification (Minimum Description Length) • Saving that can be achieved by explaining part of image by a particular hypothesis Number of pixels N explained by h Model complexity Cost of describing the error made by hypothesis h Probability of being a background Sum over all pixels hypothesized as figure

27. Initial Recognition Approach • Second Step: Segmentation based Verification (Minimum Description Length) • With this framework we can resolve conflicts between overlapping hypothesis Bias term Relative importance assigned to support of hypothesis

28. Initial Recognition Approach • Second Step: Segmentation based Verification (Minimum Description Length) • Voila! It works

29. Initial Recognition Approach • Second Step: Segmentation based Verification (Minimum Description Length) • Caveat: it leads to another set of problems Or four legs and three arms ISM doesn’t know a person doesn’t have three legs! Global Cues are needed

30. Assimilation of Global Cues • Distance Transform, Chamfer Matching get Feature Image by an edge detector get DT image by computing distance to nearest feature point Chamfer Distance between template and DT image

31. Assimilation of Global Cues (Attempt 1) • Distance Transform, Chamfer Matching Use scale estimate to cut out surrounding region Initial hypothesis generated by local features Apply Canny detector and compute DT Yellow is highest Chamfer score Chamfer distance based matching

32. Assimilation of Global Cues (Attempt 2) • Maximize Chamfer Score AND overlap with overlap with hypothesized segmentation instead of pure Chamfer Score Overlap expressed as Bhattacharya coeff. Joint score is linear combination of the two

33. Assimilation of Global Cues (Attempt 3) • Apply hypothesis saving MDL method again • Boolean quadratic formulation

34. Results