1 / 20

Segmentation with Scene and Sub-Scene Categories

Segmentation with Scene and Sub-Scene Categories. Joseph Djugash. Input Image. Scene/Sub-Scene Classification. Segmentation. Problem Statement. Goal: Accurate segmentation of salient objects/regions in any image. Problems/Issues: What is a salient object(s)?

kalyca
Download Presentation

Segmentation with Scene and Sub-Scene Categories

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Segmentation with Scene and Sub-Scene Categories Joseph Djugash Input Image Scene/Sub-Scene Classification Segmentation

  2. Problem Statement • Goal: • Accurate segmentation of salient objects/regions in any image. • Problems/Issues: • What is a salient object(s)? • How do we identify the presence of these objects? • What metric/feature/cue do we use? • Is this consistent over all images?

  3. Outline • Normalized Cut • Learning segmentation • Methods • The Naïve Approach • Affinity Matching • Minimizing Segmentation Error • Discussion of Results

  4. Bottom-Up Segmentation • Normalized Cuts, Mean Shift, etc. • Drawbacks • Parameters • Number of Clusters/Segments • Cluster Size (Sigma values) • Performance drops when dealing with a wide variety of images 1. Jianbo Shi; Malik, J. Normalized cuts and image segmentation. PAMI (2000)2. D. Comaniciu, P. Meer. Mean Shift: A Robust Approach toward Feature Space Analysis. PAMI (2002)

  5. Class Specific Segmentation • Image patches from training images are fit to the input image • Patch consistency checks correct inaccurate matching and segmentation • Only known object classes can be segmented correctly • Large database required to encompass all possible objects 3. Eran Borenstein, Shimon Ullman. Class-Specific, Top-Down Segmentation. ECCV (2002)

  6. Outline • Normalized Cut • Learning segmentation • Methods • The Naïve Approach • Affinity Matching • Minimizing Segmentation Error • Discussion of Results

  7. The Naïve Approach Scene Categories Training Phase Parameter Learning • Learnt Parameters: • Cluster Range {min & max # of clusters} • Sigma Values {controls size of cluster/segment} • The parameters are learnt using a supervised reinforcement learning algorithm (Feedback +/o/–) Testing Phase Parameter Look-Up using Bag-Of-Words Scene Categories Image Segmentation

  8. Results: The Naïve Approach

  9. Outline • Normalized Cut • Learning segmentation • Methods • The Naïve Approach • Affinity Matching • Minimizing Segmentation Error • Discussion of Results

  10. Affinity Matching Training Phase Parameters Learnt using reinforcement learning Parameter Learning Testing Phase Nearest Neighbor Matching performed on a set of stored Affinity matrices Query images are classified and learnt parameters are used to Segments Image Nearest Neighbor Classification& Parameter Look-Up Image Segmentation

  11. Results: Affinity Matching

  12. Outline • Normalized Cut • Learning segmentation • Methods • The Naïve Approach • Affinity Matching • Minimizing Segmentation Error • Discussion of Results

  13. Segmentation Error Dissimilarity within ObjectsJN = je() pxInW(pxIn, pxIn) LabelMe Data Minimizing Segmentation Error Training Phase Gist Features Kmeans Clustering J=i(JN + JS + JO) Object Pixels i єImages in Cluster j єLabeled Objects Parameter Learning Minimizing Segmentation Cost to Human Labeled Data Similarity to Non-ObjectsJN = j W(pxIn, pxOut) Cluster CostJS = je(γ*|#Seg – #Obj|) Testing Phase Image Segmentation 4. A. Torralba,  K. P. Murphy, W. T. Freeman and M. A. Rubin. Context-based vision system for place and object recognition. AIM (2003)

  14. Results: Minimizing Segmentation Error

  15. Results: Minimizing Segmentation Error

  16. Discussion • The Naïve Approach • Uses intuitive categories that humans can identify with • Human feedback can be unreliable and inconsistent • Broad category labels lead to segmentation inaccuracy • Affinity Matching • Affinity matrix provides a good feature space for segmentation • Closeness in the affinity matrix might not necessarily imply similarity in segmentation parameters • Minimizing Segmentation Error • A qualitative measure for evaluating the validity of a segmentation • Human segmentation should try to realize image cues • NCut segmentation prefers edge contours in the image

  17. Questions?

  18. Outline – Detailed • The Naïve Approach • Reinforcement Learning • Bag-of-words scene classification • Affinity Matching • NN classification on Affinity Matrix • Learning similar image structure • Human Segmentation & Minimizing Cost • LabelMe Data • Gist Features

  19. Heavy Clutter Scenery Outdoor Objects with Sky Outdoor Objects Buildings Close-Up Objects Partically Cluttered

  20. Learning Scene Categories – Bag-of-Words • Scene Categories learnt from clustering on image codewords • Feature Detection and Representation can be done using various Image Cues • Codewords stored in the database need to cover a wide spectrum 3. Fei-Fei and P. Perona. A Bayesian hierarchical model for learning natural scene categories. CVPR (2005)

More Related