1 / 48

Presented By: Yonatan Dishon Nov 2013

HOGgles : Visualizing Object Detection Features (to be appeared in ICCV 2013) Carl Vondrick Aditya Khosla Tomasz Malisiewicz and Antonio Torralba ,MIT . Presented By: Yonatan Dishon Nov 2013. Today talk. Motivation Related Work HOGgles under the hood. 3 Baselines + main algorithm

swann
Download Presentation

Presented By: Yonatan Dishon Nov 2013

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. HOGgles: Visualizing Object Detection Features (to be appeared in ICCV 2013)Carl VondrickAdityaKhosla Tomasz Malisiewicz and Antonio Torralba ,MIT Presented By: Yonatan Dishon Nov 2013

  2. Today talk • Motivation • Related Work • HOGgles under the hood. • 3 Baselines + main algorithm • Limitations • Quantative + Qualitative evaluation • Human+HOG detector • Paper Conclusion • Future Development

  3. So why HOGgles? Image from: C. Vondrick, A. Khosla, T. Malisiewicz, A. Torralba. "HOGgles: Visualizing Object Detection Features" 2013

  4. So why HOGgles? !? HOGgles Visualization Maybe I should put my HOGgles!!

  5. So why HOGgles? • This is a visualization of the descriptor space – this is what a classification/detection algorithm sees! • So which of the following would you classify as a car? • Remember Humans are the “perfect” classifier! • Object categories (PASCAL VOC): Airplane, Bicycle, Bird, Boat , Bottle, Bus, Car, Cat, Chair, Cow, Table, Dog, Horse, Motorbike, Person, Potted Plant, Sheep, Sofa, Train, TV/Monitor 1 2 3 4 5 6 7

  6. Motivation • So why did my detector failed? • Training set Maybe nota good one? • Learning Algorithm Maybe should have been different? • Features Maybe there are better features for this kind of problem? • Visualizing the Feature space can bring us to an intuitive understanding of our detection system limitations and failures

  7. HOGgles Contributions • A tool to explain some of the failure of object detection systems. • Algorithm to present the feature space of object detectors (general features – not only HOG!). • 4 different algorithms to do so are presented. • Public feature visualization toolbox - http://web.mit.edu/vondrick/ihog/#code

  8. Today talk • Motivation • Related Work • HOGgles under the hood. • 3 Baselines + main algorithm • Limitations • Quantative + Qualitative evaluation • Human+HOG detector • Paper Conclusion • Future Development

  9. Related Work (1) • Reconstruct an image given keypoint of SIFT based on a huge database P. Weinzaepfel, H. J´egou, and P. P´erez. Reconstructing an image from its local descriptors. In CVPR, 2011 • Image from: P. Weinzaepfel, H. J´egou, and P. P´erez. Reconstructing an image from its local descriptors. In CVPR, 2011.

  10. Related Work (1)

  11. Related Work (1) elliptic region of interest calculate SIFT affine normalization to square patch Reconstructing an image from its local descriptors, Philippe Weinzaepfel, HervéJégou and Patrick Pérez, Proc. IEEE CVPR’11. original image copied patches blending interpolation Slide credit: EzgiMercan

  12. Related Work (1) • Image from: P. Weinzaepfel, H. J´egou, and P. P´erez. Reconstructing an image from its local descriptors. In CVPR, 2011. • Website for more examples : http://www.irisa.fr/texmex/people/jegou/projects/reconstructing/index.html

  13. Related Work (2) • Reconstruct an image given only LBP features E. d’Angelo, A. Alahi, and P. Vandergheynst. Beyond bits: Reconstructing images from local binary descriptors. ICPR, 2012. 2 • Using LBD descriptors (Local Binary Descriptors) – BRIEF and FREAK • No external information! • Reconstruction as a regularized inverse problem • A. Alahi, R. Ortiz, and P. Vandergheynst. FREAK: Fast Retina Keypoint. In IEEE Conference on Computer Vision and Pattern Recognition (To Appear), 2012. • M. Calonder, V. Lepetit, C. Strecha, and P. Fua. BRIEF: Binary Robust Independent Elementary Features. Computer Vision–ECCV 2010, pages 778–792, 2010.

  14. Related Work (2) Image reconstruction results of FREAK (top row) and BRIEF (bottom row) descriptors. E. d’Angelo, A. Alahi, and P. Vandergheynst. Beyond bits: Reconstructing images from local binary descriptors. ICPR, 2012. 2

  15. Today talk • Motivation • Related Work • HOGgles under the hood. • 3 Baselines + main algorithm • Limitations • Quantative + Qualitative evaluation • Human+HOG detector • Paper Conclusion • Future Development

  16. HOGgles – under the hood • The problem of feature visualization as a feature inversion problem. Given a feature vector - what was the image/patch that created it? • Let be an image and be the corresponding HOG feature descriptor. • is a many to one function. The inversion problem cannot be solved analytically!

  17. HOGgles under the hood • The problem is formalized as an optimization problem – given a descriptor y we seek an image x that minimize: • This function isn’t convex • Trying to find a minima with ordinary optimization algorithms didn’t work (Steepest decent and Newton’s method).

  18. Today talk • Motivation • Related Work • HOGgles under the hood. • 3 Baselines + main algorithm • Limitations • Quantative + Qualitative evaluation • Human+HOG detector • Paper Conclusion • Future Development

  19. HOGgles under the hood • 4 algorithms are showed – 3 as a baselines and one is offered as the main algorithm. • Baseline 1: Exampler LDA • Baseline 2: Ridge Regression • Baseline 3: Direct Optimization • Main Algorithm : Paired Dictionary

  20. HOGgles under the hood (Baseline 1 -ELDA) • Baseline 1: ExamplerLDA (B.Hariharan, , J.Malik & D.Ramanan ECCV 2012) • HOG inverse is the average of the top K detections of the ELDA detector in RGB space .

  21. HOGgles under the hood (Baseline 1 -ELDA) Slide credit: EzgiMercan

  22. HOGgles under the hood (Baseline 1 -ELDA) • PROs: • Simple. • Surprisingly accurate results! Even when the database doesn’t contain the category of HOG template! • CONs: • Computationally expensive! – running an object detector over a large database • Yields blurred results.

  23. HOGgles under the hood (Baseline 2 –Ridge Regression) • Baseline 2: Ridge Regression • Statistically most likely image given a HOG descriptor. • Calculating the most probable grayscale image given its HOG feature. • Modeling as • The HOG inverse (the visualization) is given by • are estimated on a large database • Single matrix multiplication!

  24. HOGgles under the hood (Baseline 2 –Ridge Regression) • PROs: • Simple – this is a matrix multiplication! • Very fast! – under a second for inversion. • CONs: • Inversion yields blur images.

  25. HOGgles under the hood (Baseline 3 -Direct) • Baseline 3: Direct Optimization • Describing a natural image basis • Any image can be encoded by coefficients in this basis: • And we wish to find

  26. HOGgles under the hood (Baseline 3 -Direct) • PROs: • Recover high frequencies • CONs: • adding noise to the image

  27. HOGgles under the hood (the main algorithm –Pair Dictionary) • Let be an image be its HOG descriptor. Suppose we write x and y in terms of bases and respectively where are shared coefficients • y can be projected to V basis and then to image basis U

  28. HOGgles under the hood (Main Algorithm –PairDict)

  29. HOGgles under the hood (Baseline 4 –PairDict) • How the bases U and V are found? • Solving Pair dictionaries learning problem. • The objective is simplified to a standard sparse coding and dictionary learning problem, optimized with SPAMS J. Mairal, F. Bach, J. Ponce, and G. Sapiro. Online dictionary learning for sparse coding. In ICML, 2009. 4 SPAMS – SParse Modeling Software, Code available at http://spams-devel.gforge.inria.fr/

  30. HOGgles under the hood (Baseline 4 –PairDict) • Dictionaries optimization time is a few hours (offline). • U and V are estimated with dictionary size • Training samples from a large database • examples for U & V pairs – notice the correlation of the dictionaries

  31. HOGgles visual results

  32. Today talk • Motivation • Related Work • HOGgles under the hood. • 3 Baselines + main algorithm • Limitations • Quantative + Qualitative evaluation • Human+HOG detector • Paper Conclusion • Future Development

  33. HOGgles – Limitations • There exist a better visualization than paired dictionaries although it may not be traceable to construct it. • On Recursive iterative solution some high frequencies are lost.

  34. HOGgles – Limitations – cont. • Inversion is sensitive to the HOG dimensionality

  35. Today talk • Motivation • Related Work • HOGgles under the hood. • 3 Baselines + main algorithm • Limitations • Quantative + Qualitative evaluation • Human+HOG detector • Paper Conclusion • Future Development

  36. Evaluation of Inversions • PASCAL VOC 2011 dataset. • Inverting patches correspond to objects. • Quantitative evaluation: • How well each pixel in xis reconstructed from y by each Algorithm? • Qualitative evaluation: • How well the high level content is saved? (Human research using MTurk platform).

  37. Evaluation of InversionsQuantitative • Mean normalized cross correlation of inverse image to ground truth. (Higher is better , max. is 1)

  38. Evaluation of InversionsQualitative • MTurk workers where asked to classified inversions to 1 of 20 categories. • MIT PhD. In CV refers as experts. Trying the same task with HOG Glyphs. (*) Numbers are percentage classified correctly, chance is 0.05

  39. Evaluation of InversionsQualitative • Graphs credit : EzgiMercan

  40. Evaluation of InversionsQualitative • Gliphs vs. HOGgles

  41. Today talk • Motivation • Related Work • HOGgles under the hood. • 3 Baselines + main algorithm • Limitations • Quantative + Qualitative evaluation • Human+HOG detector • Paper Conclusion • Future Development

  42. Human+HOG detector • Get Insight on performance of HOG with the perfect learning algorithm (people). • Large Human Expirement consist of: • MTurk workers • Dataset – Top detections from DPM on PASCAL 2007 VOC. • 5000 windows per category. 20% are true positive. • 25 votes per window.

  43. Human+HOG detectorResults

  44. Paper Conclusions • DPM is operating very close to the performance limit of HOG. • HOG may be too lossy of a descriptor for high performance object detection. • The features we are using are the one to blame in current novel object detection algorithms. • To advance to the next level recognition – finer details and higher level information capturing features are needed to be built.

  45. Written Level of the Paper • Pros: • Well written • Well referenced • Novel solution • Large and detailed Human experiment • Website. • Cons: • Details/examples on baseline algorithms are lacking • The chosen algorithm settings are lacking • Some conclusions on the Human+HOG are questionable

  46. Future development • Applying visualization on other descriptors. • Color reconstruction of HOG features. • Developing new feature that can be more discriminate. • Developing Algorithms that better model the interaction of the simple atoms of the image.

  47. Thank You! Questions?

  48. Links • PASCAL 2 challenge • Deformable Part Model • HOGgles

More Related