Pedestrian Detection: introduction

1. Pedestrian Detection: introduction • Approaches • Holistic detection: use local search window that meets criterias • Part-based detection: pedestrian as a collection of parts (to be found!) • Patch-based detection: local features matched against a (learned) codebook, then voting for final detection For a survey on the approaches see: N.Dalal “Finding people in images and videos”, PhD thesis, July 2006.

2. Holisticapproaches Some remarkablepedestrian detector • Haarwavelets + SVM P. Papageorgiou and T. Poggio, “A trainable system for object detection,” Intl. J. of Computer Vision, vol. 38, no. 1, pp. 15–33, 2000. • the popular face detector from Viola Jones (haar+adaboost face-detector) + motioncues P. Viola, M. Jones, and D. Snow, “Detecting pedestrians using patterns of motion and appearance,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition, New York, NY, volume 1, 2003, pp. 734–741. • Histogramoforientedgradients (HOG) N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition, San Diego, CA, volume 1, 2005, pp. 886–893. Q. Zhu, S. Avidan, M. C. Yeh, and K. T. Cheng, “Fast human detection using a cascade of histograms of oriented gradients,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition, New York, NY, volume 2, 2006, pp. 1491 – 1498. • CovarianceDescriptors + Boosting O. Tuzel, F. Porikli, and P. Meer, “Pedestrian detection via classification on Riemannian manifolds,” To appear in IEEE Trans. Pattern Anal. Machine Intell., 2008.

3. Pedestrian detection usingcovariancedescriptors and boosting rejectioncascadesapproach Casc 2 Casc 1 Casc N Detection on negatives increases  Detection on positives decreases 

4. Pedestrian detection usingcovariancedescriptors and boosting rejectioncascadesapproach Detection on negatives increases  Detection on positives decreases 

5. Performance Cov-Desc + Boosting HOG + Ker SVM HOG + Lin SVM HOG + Boosting each black dot is an additional cascade

6. Inside a cascade: the weak learners • Add a weak learner to the cascade, that is • Assign to it a (randomly extracted) sub-region • Compute covariance-descriptor on this region for each • Positive sample • Negative sample • Are positives and negatives “easily” separable? • No? Go to 1 • Yes? Done with this cascade

7. Covariancedescriptors Defineanimage Region R Extract Pixel-wise Feature 1 Mean, var Covariance CR (NxNmatrix, symposdef) Extract Pixel-wise Feature 2 Mean, var Extract Pixel-wise Feature N Mean, var (e.g.: color components, luminance, gradients, …)

8. Covariancedescriptors • PROs • Versatile and flexible (you can use the pixel-wise features most suitable for your goal) • Computed very quickly using integral images • Compact (N*(N+1)/2 independent values) • CONs • Euclidean distance is NOT appropriate over symmetric positive matrices: they lie over a Riemannian manifolds

9. Riemannian Manifolds • In order to use traditional machine learning techiniques: • Move back and forth from Riemannian manifold of sym pos def matrices to euclidean space of symmetric matrices using respectively • Exponential of Matrix • Logarithm of Matrix •  Expansive computations! • J. Jost, “Riemannian Geometry and Geometric Analysis”. Springer, fourth edition, 2005. • X. Pennec, P. Fillard, and N. Ayache, “A Riemannian framework for tensor computing,” Intl. J. of Computer Vision, vol. 66, no. 1, pp. 41–66, 2006.

10. Using this detector in real-time surveillance applications • Reduce the pedestrian search region where there is (or was) motion • Build a motion history and focus the detection search over the most recent motion but keeping an eye on oldest motion regions • Exploit the background when camera is static with implicit relevance feedback: • In many surveillance scenarios it is possible to assume that the background image does not contain humans • therefore, enrich the generic pedestrian classifier • training some additional ad-hoc and view-dependent cascades • that tackle the false-positives detected in the background • The enriched classifier is to be used in the ped-detection over the given view • Use the pedestrian detection to infer the scene perspective • False detections are very limited, and most of them are out of perspective • Therefore, having defined a perspective model, it is possible to estimate it, rejecting the outliers • Once estimated, it can reject out-of-perspective detections @ICDSC09: CovarianceDescriptors on MovingRegionsforHuman Detection in VeryComplex Outdoor ScenesGiovanni Gualdi, Andrea Prati and Rita Cucchiara.Univ. of Modena and Reggio Emilia

11. Using this detector in real-time surveillance applications

12. Examples Detections using the generic classifier Detections using the generic classifier + Perspective model Detections using the generic classifier + Perspective model + Additional relevance feedback cascades

13. Videos