1 / 28

Jonathan Warrell, 1 Simon Prince, 2 Philip Torr, 1 Lubor Ladicky, 1 Chris Russell 1

Models for Scene Understanding – Global Energy models and a Style-Parameterized boosting algorithm ( StyP -Boost) . Jonathan Warrell, 1 Simon Prince, 2 Philip Torr, 1 Lubor Ladicky, 1 Chris Russell 1 1 Oxford Brookes University, 2 University College London. Overview .

jess
Download Presentation

Jonathan Warrell, 1 Simon Prince, 2 Philip Torr, 1 Lubor Ladicky, 1 Chris Russell 1

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Models for Scene Understanding – Global Energy models and a Style-Parameterized boosting algorithm (StyP-Boost) Jonathan Warrell,1 Simon Prince,2 Philip Torr,1 Lubor Ladicky,1 Chris Russell1 1Oxford Brookes University, 2University College London

  2. Overview • CRF-based semantic segmentation • Recent models • Detectors • Stereo • Co-occurence • Hierarchical Energies • Style parameterized boosting (StyP-Boost) • Open questions / problems

  3. CRF-based semantic segmentation • Semantic segmentation = dense labeling using fixed object set

  4. CRF-based semantic segmentation • Conditional Random Field model (pairwise) Observed Variables Hidden Variables Unary Pairwise

  5. Example: -expansion Status: Tree Ground Initialize with Tree Expand Ground Expand House Expand Sky House Sky Courtesy: Pushmeet Kohli

  6. Move Making Algorithms Current Solution Search Neighbourhood Optimal Move Energy Solution Space

  7. Higher order CRF models • Higher order models Unary Pairwise Higher-order

  8. Segment-based Potentials No. of pixels not taking l in c

  9. Detector-based Potentials Strength of detector response LuborLadicky, Paul Sturgess, KarteekAlahari, Chris Russell, Philip H.S. Torr, What,Where & How Many? Combining Object Detectors and CRFs, ECCV 2010

  10. Co-occurrence Potentials Global image label set LuborLadicky, Chris Russell, PushmeetKohli, Philip H.S. Torr, Graph Cut based Inference with Co-occurrence Statistics, ECCV, 2010

  11. Joint Stereo + Segmentation Object only potentials Joint potentials LuborLadicky, Paul Sturgess, Chris Russell, SunandoSengupta, Philip H.S. Torr, Joint Optimisation for Object Class Segmentation and Dense Stereo Reconstruction, BMVC 2010

  12. Hierarchical Energies Energy between levels 1 and 0 LuborLadicky, Chris Russell, PushmeetKohli, Philip H.S. Torr, Associative Hierarchical CRFs for Object Class Image Segmentation, ICCV, 2009.

  13. Style-based Potentials Style 1: Style 2: Style-based unary potential Jonathan Warrell, Simon Prince, Philip H.S. Torr, StyP-Boost: A Bilinear Boosting Algorithm for Learning Style-Parameterized Classifiers, BMVC, 2010

  14. TextonBoost (Shotton et al ’09) • Image first convolved with 17-d filter bank • Vectors are clustered, and assigned to ~150 texton indices

  15. TextonBoost (Shotton et al ’09) • Texture-layout features derived from textons • Boosted classifier predicts semantic class

  16. DenseBoost (Ladicky et al ’09) • DenseBoost extends TextonBoost to include • HOG • ColourHOG • Structure / Motion features • State of the art performance on • MSRC (Ladicky et al ’09) • CamVid (Sturgess et al ’09) LuborLadicky, Chris Russell, PushmeetKohli, Philip H.S. Torr, Associative Hierarchical CRFs for Object Class Image Segmentation, ICCV, 2009. Paul Sturgess, KarteekAlahari, LuborLadicky, Philip H.S. Torr, Combining Appearance and Structure from Motion Features for Road Scene Understanding, BMVC, 2009

  17. StyP-Boost Framework (Training) • Training Set • Objective • Classifier form Local features Style Parameters Target vectors

  18. StyP-Boost Framework (Training) • Training Set • Objective • Classifier form Strong learner for class k Loss for class k

  19. StyP-Boost Framework (Training) • Training Set • Objective • Classifier form Weak learner m Style s

  20. Corel: Styles through clustering • Styles found in Corel through clustering 2-styles (98%) 3-styles(96%) 4-styles (89%)

  21. Corel: Styles through clustering • Cluster images based on label histograms during training (2-4 clusters) • Train classifier to predict cluster from image • Use smoothed classifier posteriors as style parameters (training and testing) label label label cluster

  22. Corel: Qualitative results • StyP-Boost reduces noise from classes which don’t co-occur

  23. Corel: Qualitative results • StyP-Boost provides better discrimination of • co-occuring classes

  24. Corel: Quantitative results Test set Training set

  25. Open questions / Problems • Learning from sparsely labeled data Lamp-post Sign

  26. Open questions / Problems • Incorporating 3D and Video Volumetric CRF Ground-plan CRF Image CRF

  27. Open questions / Problems • Using temporal information • Extend detector potentials to include tracking • Use global scene variables for times of day, seasons etc.

  28. Further Questions • Further Questions?

More Related