Large-Scale Object Recognition with Weak Supervision

1. Large-Scale Object Recognition with Weak Supervision WeiqiangRen, Chong Wang, Yanhua Cheng, KaiqiHuang, TieniuTan {wqren,cwang,yhcheng,kqhuang,tnt}@nlpr.ia.ac.cn

2. Task2 : Classification + Localization Task 2b: Classification + localization with additional training data — Ordered by classification error Only classification labels are used Full image as object location

3. Outline • Motivation • Method • Results

4. Motivation

5. Why Weakly Supervised Localization (WSL)? Knowing where to look, recognizing objects will be easier ! However, in the classification-only task, no annotations of object location are available. Weakly Supervised Localization

6. Current WSL Results on VOC07

7. 13.9: Weakly supervised object detector learning with model drift detection, ICCV 2011 15.0: Object-centric spatial pooling for image classification, ECCV 2012 22.4: Multi-fold mil training for weakly supervised object localization, CVPR 2014 22.7: On learning to localize objects with minimal supervision, ICML 2014 26.2: Discovering Visual Objects in Large-scale Image Datasets with Weak Supervision, submitted to TPAMI 26.4: Weakly supervised object detection with posterior regularization, BMVC 2014 31.6: Weakly supervised object localization with latent category learning, ECCV 2014 Sep 11, Poster Session 4A, #34

8. Our Work Weakly Supervised Object Localization with Latent Category Learning Discovering Visual Objects in Large-scale Image Datasets with Weak Supervision ECCV 2014 Submitted to TPAMI For the consideration of high efficiency in large-scale tasks, we use the second one.

9. Method

10. Framework 2 Det Prediction 3 Rescoring 4 Cls Prediction … Conv Layers 1 Input Images FC Layers

11. 1st: CNN Architecture Chatfield et al. Return of the Devil in the Details: Delving Deep into Convolutional Nets

12. 2nd: MILinear SVM

13. MILinear : Region Proposal • Good region proposal algorithms • High recall • High overlap • Small number • Low computation cost • MCG pretrained on VOC 2012 • Additional Data • Training: 128 windows/ image • Testing: 256 windows/image • Compared to Selective Search (~2000)

14. MILinear: Feature Representations • Low Level Features • SIFT, LBP, HOG • Shape context, Gabor, … • Mid-Level Features • Bag of Visual Words (BoVW) • Deep Hierarchical Features • Convolutional Networks • Deep Auto-Encoders • Deep Belief Nets

15. MILinear: Positive Window Mining • Clustering • KMeans • Topic Model • pLSA, LDA, gLDA • CRF • Multiple Instance Learning • DD, EMDD, APR • MI-NN, • MI-SVM, mi-SVM • MILBoost

16. MILinear: Objective Function and Optimization • Multiple instance Linear SVM • Optimization: trust region Newton • A kind of Quasi Newton method • Working in the primal • Faster convergence

17. MILinear: Optimization Efficiency

18. 3rd: Detection Rescoring • Rescoring with softmax train softmax … … max 128 boxes …… …… 1000 dim 1000 dim 1000 classes Softmax: consider all the categories simultaneously  at each minibatch of the optimization – Suppress the response of other appearance similar object categories

19. 4th: Classification Rescoring • Linear Combination … … … 1000 dim 1000 dim 1000 dim One funny thing: We have tried some other strategies of score combination, but it seems not working !

20. Results

21. 1st: Classification without WSL

22. 2nd: MILinear on ImageNet 2014

23. 2nd: MILinear on VOC 2007

24. 2nd: MILinear on ILSVRC 2013 detection mAP: 9.63%! vs 8.99% (DPM5.0)

25. 2nd: MILinear for Classification

26. 3rd: WSL Rescoring (Softmax) The Softmax based rescoring successfully suppresses the predictions of other appearance similar object categories !

27. 4th: Cls and WSL Combinataion WSL and Cls can be complementary to each other!

28. Russakovskyet al. ImageNet Large Scale Visual Object Challenge.

29. Conclusion • WSL always helps classification • WSL has large potential: WSL data is cheap

30. Thank You!