Lior Wolf and Noga Levy The SVM-minus Similarity Score for Video Face Recognition

1. Lior Wolf and Noga LevyThe SVM-minus Similarity Score for Video Face Recognition 24.06.2013 Makarand Tapaswi CVPR Reading Group @ VGG

2. Same / Not Same ?

3. One liner “How similar is the face in one video sequence to the other, where the similarity is uncorrelated with pose-induced similarity” • illumination, expression, image quality, pose • classifier should • discriminate positive/negative AND • uncorrelate w.r.to additional feature set

4. Basic Notation • : the background (negative) set

5. Matched Background Similaritysame person B1 X1 B X2 B2

6. MBGSdifferent persons B1 B2 X1 X2 B

7. MBGS

8. SVM-minus Classifier • Inputs • Training set • Privileged info. • Labels • + + + + + - - - - -

9. SVM-minus classifier • = train • = test signed dist. to hyperplane • Un-correlate with

10. SVM-minus Classifier (2) • Split • into and • into and • necessary since classifiers are correlated • Normalization • and each feature-dimension to 0 mean • and to mean 0, and

11. SVM– loss function • Pearson correlation coefficient • Convexity: ignore denom. and square num.

12. Reduce to standard SVM Can be reduced to standard SVM in the dual form. In the dual and signed by

13. Projection Matrix • thus is positive definite • is p.d., Cholesky decomposition

14. SVM-minus Similaritysame person • Cancel influence from pose +ve scoring poses need not be same person –ve scoring poses need not be different person Pan angle

15. One-Side SVM-minus Similarity

16. SVM-minus Similarity • Use one-side SVM-minus for online tasks

17. YouTube Faces DB • DB from [36] • Video LFW • 3,425 videos; 1,595 people • 2.15 videos / person • min-duration: 48 frames • Average-clip length: 181.3 frames • Evaluation • 5000 pairs • 10 fold cross-validation • 250+, 250– • Person exclusive splits (person appears only in one split)

18. Experimental info • Detect face, expand bbox, align, resize 100x100 • Extract features • LBP • Center-Symm. LBP • Four-Patch LBP • 3D head orientation () from face.com API*

19. MBGS Results from [36]Lior Wolf, Tal Hassner and ItayMaoz. Face Recognition in Unconstrained Videos with Matched Background Similarity. CVPR 2011.

20. This paper results Results where SVM– did most better than MBGS

21. Results • MBGS > SVM– at Accuracy • but, MBGS + SVM– wins • Combination done by stacking • learning yet another SVM for the 2D scores

22. Is it really useful? • Combined score “statistically significant” for [FP]LBP • Use entire background set, AUC: 83.6% to 79.9% • Online applications (one-side), AUC: 83.6% to 81.9% • Correlations: • Within method higher, different scores • Across methods, highest for same feature (as expected)

23. Conclusion • SVM– : unlearn using additional features • MBGS : be choosy about the negative set • 3D Pose : a good “privileged” information source • They don’t talk about pose estimation accuracy • Different types of privileged info that might work • Metric learning (and relatives) not compared Thank You!

24. Some more results from other sourcesYouTube Faces DB References:[1]  Lior Wolf, Tal Hassner and ItayMaoz. Face Recognition in Unconstrained Videos with Matched Background   Similarity. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2011.[2]  Lior Wolf and Noga Levy. The SVM-minus Similarity Score for Video Face Recognition. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2013.[3]  Haoxiang Li, Gang Hua, Zhe Lin, Jonathan Brandt, Jianchao Yang. Probabilistic Elastic Matching for Pose Variant Face Verification. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2013.[4]  Zhen Cui, Wen Li, Dong Xu, Shiguang Shan and Xilin Chen. Fusing Robust Face Region Descriptors via Multiple Metric Learning for Face Recognition in the Wild. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2013.