1 / 48

Attributes for Classifier Feedback

Attributes for Classifier Feedback. Amar Parkash and Devi Parikh. You can teach a child by examples. And on, and on, and on…. Is this a giraffe?. No. Is this a giraffe?. Yes. Is this a giraffe?. No. And on, and on, and on…. Proposed Active Learning Scenario. Focused feedback

ingrid
Download Presentation

Attributes for Classifier Feedback

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Attributes for Classifier Feedback AmarParkash and Devi Parikh

  2. You can teach a child by examples...

  3. And on, and on, and on…

  4. Is this a giraffe? No. Is this a giraffe? Yes. Is this a giraffe? No.

  5. And on, and on, and on…

  6. Proposed Active Learning Scenario

  7. Focused feedback Knowledge of the world Current belief I think this is a giraffe. What do you think? No, its neck is too short for it to be a giraffe. • Learner learns better from its mistakes • Accelerated discriminative learning with few examples [Animals with even shorter necks] …… Ah! These must not be giraffes either then. Feedback on one, transferred to many

  8. Communication Need a language that is • Machine understandable • Human understandable Attributes! • Mid-level shareable • Visual • Semantic

  9. Proposed Active Learning [Label-feedback] [Attributes-based feedback] [Predicted Label] Attribute predictors: a1 a2 … aM Unlabeled pool of images Classifiers: Concepts to teach: h1 h2 … hK C1 C2 … CK Any feature space Any discriminative learning algorithm

  10. Relative Attributes[Parikh and Grauman, ICCV 2011] Image features Attribute predictors: Parameters a1 a2 … aM Unlabeled pool of images Openness

  11. Attributes-based Feedback No, It is too open to be a forest Forest Attribute predictors: a1 a2 … aM Unlabeled pool of images Unlabeled pool of images Not Forest Openness

  12. Attributes-based Feedback No, It is too open to be a forest Forest Attribute predictors: a1 a2 … aM Unlabeled pool of images Unlabeled pool of images Classifiers: h1 h2 … hK Not Forest

  13. Proposed Active Learning [Label-feedback] [Attributes-based feedback] [Predicted Label] Attribute predictors: a1 a2 … aM Unlabeled pool of images Classifiers: Concepts to teach: h1 h2 … hK C1 C2 … CK

  14. Label-based Feedback • Not our contribution • Experiment with different scenarios • Benefits of attributes-based feedback • Small when label-based is very informative • Large when label-based feedback is weak

  15. Label-based Feedback Forest • Accept: Yes, this is a forest. • Strong: It is not anything else Example: Classification • Weak: It can be other things Example: Annotation Unlabeled pool of images

  16. Label-based Feedback Forest • Reject: • Strong: No, it is a coast. Example: Classification with few classes • Weak: No, this is not a forest. Example: Large-scale classification Example: Biased binary classification • 4 different scenarios in experiments Unlabeled pool of images

  17. Datasets • Datasets and relative attribute predictors from [Parikh and Grauman, ICCV 2011]

  18. Datasets • Faces: • 8 celebrity categories • 11 attributes (chubby, white, etc.) [Kumar et al., ICCV 2009]

  19. Datasets • Scenes: • 8 categories • 6 attributes (open, natural, etc.) [Oliva and Torralba, IJCV 2001]

  20. Settings • Feedback from MTurk • Features: • Raw image features (gist, color) • Attribute scores • Category classifiers: SVM with RBF kernel • Results on • 2 datasets x 2 features x 4 label-feedback scenarios • Show 2 here, rest in paper. This image doesn’t have enough perspective to be a street scene.

  21. Results Faces, Attribute features, Strong label-feedback Accuracy # iterations

  22. Results Scenes, Image features, Weak label-feedback 1/4th ! Accuracy # iterations More results in the paper.

  23. Conclusion • Attributes for providing classifier feedback • Novel learning paradigm with enhanced human-machine communication • Discriminative learning + domain knowledge • Learning with few examples • Connections to semi-supervised learning • Shrivastava, Singh and Gupta: Up Next!

  24. Thank you!

  25. Backup Slides

  26. Negative Feedback • Many reasons come together to make a concept • Hard to describe why an image is a concept • One reason can break a concept • Easier to describe why an image is not a concept • (Arguably) waste to give feedback when right

  27. Related Work • Zero-shot learning • Convey domain knowledge • Zero training images • Generative model • Classification in attribute space • Performance compromised • Proposed paradigm • Convey domain knowledge to transfer • Few training images • Discriminative model • Classification in any feature space • Performance maintained • Discriminative • No domain knowledge is conveyed • Many training images • Discriminative model • Classification in any feature space • State-of-art performance – – S – C Smiling + J + H + Cis younger than H C smiles more than H M Age Mis younger than J Msmiles more than J [Lampert et al., CVPR 2009] [Parikh and Grauman, ICCV 2011]

  28. Related Work Focused discrimination • Mining of hard negatives [Felzenszwalb 2010] • To understand classifier [Golland 2001] • Here, supervisor provides the discriminative direction by verbalizing semantic knowledge [Felzenszwalb et al., CVPR 2008] [Golland, NIPS 2001]

  29. Related Work What we are not doing: • Collecting deeper annotations of images • Segmentation masks [Russell 2008], Parts [Farhadi 2010], Pose [Bourdev 2009], Attributes [Kumar 2009] • We use attributes for broad propagation of category labels to unlabeled images

  30. Related Work What we are not doing: • Actively interleaving attribute annotations • Object & attributes [Kovashka 2011], Image, boxes & segments [Vijaynarsimhan 2008], Parts & attributes [Wah 2011], etc. • Human-in-the-loop at test time [Branson 2010] • Our supervisor provides additional information at training time which is leveraged for better category models

  31. Related Work • Rationales • Human feature selection in NLP [Raghavan 2005] • Spatial and attribute rationales [Donahue 2011] • Restricted to classification in attribute space • We can operate in any feature space [Donahue and Grauman, ICCV 2011]

  32. Imperfect Attribute Predictors • In the end, all images labeled with ground truth • Discriminative training can deal with outliers • Attributes are pre-trained, and so unlikely to be severely flawed • Experiments: used predictors directly from Parikh and Grauman, 2011

  33. Large-scale Classification • Categories may require expert knowledge, attributes need not • Can show a few exemplars to verify category

  34. Results Faces, Classification, Attribute features (overestimate) Accuracy # iterations

  35. Label-based Feedback • Simulate the different scenarios in responses Benefits of attributes-based feedback • Small when label-based is very informative • Large when label-based feedback is weak Classification Large-scale Classification Street Outdoor City Grayscale …. Annotation Biased Binary Classification

  36. Results Scenes, Annotation, Image features Accuracy # iterations

  37. Results Faces, Biased binary classification, Attribute features Accuracy # iterations More results in the paper.

  38. Traditional Active Learning Highest Entropy [Label] Unlabeled pool of images Classifiers: Concepts to teach: h1 h2 … hK C1 C2 … CK

More Related