1 / 26

Context-based vision system for place and object recognition

Context-based vision system for place and object recognition. Antonio Torralba Kevin Murphy Bill Freeman Mark Rubin Presented by David Lee. Some slides borrowed from Kevin Murphy. Object out of context. Object in context. Wearable test-bed. System diagram. Computing the features.

tyne
Download Presentation

Context-based vision system for place and object recognition

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Context-based vision system for place and object recognition Antonio Torralba Kevin Murphy Bill Freeman Mark Rubin Presented by David Lee Some slides borrowed from Kevin Murphy

  2. Object out of context

  3. Object in context

  4. Wearable test-bed

  5. System diagram

  6. Computing the features

  7. 4x4x24 =384 dim 80 dim Downsample to 4x4 24 filtered Images

  8. Visualizing the filter bank output Images 80-dimensional representation

  9. Place recognition system

  10. Hidden Markov Model • Hidden states = location (63 values) • Observations = vGt∈ R80 • Transition model encodes topology of environment • Observation model is a mixture of Gaussians (100 views per place)

  11. Mixture of Gaussians MLE (counting) Hidden Markov Model Observation Likelihood Prediction Prior Transition Matrix

  12. Scene Categorization • 17 Categories (Office, Corridor, Street, etc) • Train a separate HMM on category labels

  13. Place recognition demo

  14. Performance on known env. Ground truth System estimate Specific location Location category Indoor/outdoor

  15. Performance on new env.

  16. Comparison of features Categorization Recognition

  17. Effect of HMM on recognition Without With (But with temporal smoothing)

  18. From place to object recognition

  19. Object priming • Predict object properties based oncontext (top-down signals): • Visual gist, vtG • Specific Location, Qt • Kind of location, Ct

  20. MLE Mixture of Gaussians Object Priming Estimate of current place (Output of HMM) Probability of object i in image vi given entire video sequence Probability of object i Given current observation & place Prior probability of object i being in place q Observation Likelihood Probability of object i Again…

  21. Predicting object presence

  22. ROC curves for object detection

  23. Predicting object position and scale

  24. Predicting object position and scale Probability of an object i being present and location being q (Output of previous system) Estimate of mask Estimate of mask given current gist, place, and object delta Gaussian

  25. Predicted segmentation

  26. Conclusion • Real world problem (and it works!) • Uses only global feature (context) • How much did {HMM / place prior} affect{place recognition / object detection}?Can we really say “context” did the job?

More Related