1 / 38

Human-Computer Interaction Segmentation

Human-Computer Interaction Segmentation. Hanyang University Jong-Il Park. Why segment images?. Form large chunks of pixels that can be dealt with together for efficiency because these might represent objects Join up image tokens that together convey information. Grouping.

edana
Download Presentation

Human-Computer Interaction Segmentation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Human-Computer InteractionSegmentation Hanyang University Jong-Il Park

  2. Why segment images? • Form large chunks of pixels that can be dealt with together • for efficiency • because these might represent objects • Join up image tokens that together convey information

  3. Grouping • Humans interpret image information collectively • in “groups” • Eg. Muller-Lyer illusion

  4. Applications • Shot boundary detection • summarize video by • find shot boundaries • obtain “most representative” frame • Background subtraction • find “interesting bits” of image by subtracting known background • Eg. find person in an office • Eg. find cars on a road • Interactive segmentation • user marks some foreground/background pixels • system cuts object out of image • useful for image editing, etc.

  5. Technique: Shot Boundary Detection • Find the shots in a sequence of video • shot boundaries cause big differences between succeeding frames • Strategy: • compute interframe distances • declare a boundary where these are big • Possible distances • frame differences • histogram differences • block comparisons • edge differences

  6. Technique: Background Subtraction • If we know the background, easy to find “interesting bits” • Approach: • use a moving average to estimate background image • subtract from current frame • large absolute values are interesting pixels • trick: use morphological operations to clean up pixels

  7. Interactive segmentation • Goals • User cuts an object out of one image to paste into another • User forms a matte • weights between 0 and 1 to mix pixels with background • to cope with, say, hair • Interactions • mark some foreground, background pixels with strokes • put a box around foreground • Technical problem • allocate pixels to foreground/background class • consistent with interaction information • segments are internally coherent and different from one another

  8. Superpixels • Pixels are too small and too detailed a representation • for recognition • for some kinds of reconstruction • Replace with superpixels • small groups of pixels that are • clumpy • like one another • a reasonable representation of the underlying pixels

  9. Segmentation as clustering • Cluster together (pixels, tokens, etc.) that belong together • Agglomerative clustering • attach closest to cluster it is closest to • repeat • Divisive clustering • split cluster along best boundary • repeat

  10. The watershed algorithm • An agglomerative clusterer with a special metric

  11. Clustering pixels • Natural to use k-means • represent pixels with • intensity vector; color vector; vector of nearby filter responses • perhaps position

  12. The Mean Shift Algorithm • Originally intended to find modes in scattered data • Strategy • start at a promising estimate of mode • iterate until the estimate doesn’t change • fit a model of probability density to some points near estimate • find the peak of this model • Model • smoothing kernel • the update takes a special form • shift the mode to a weighted mean of the nearby points • hence the name.

  13. Clustering with Mean Shift • Model data points as samples from a probability model • clusters are associated with modes • but it might be hard to find one mode per cluster • if there’s more than one mode per cluster, they should be close together • Apply mean shift to find modes • modes should form small, widely separated clusters • Now cluster the modes with (say) agglomerative clusterer • easy, because there are small, widely separated clusters • Point belongs to cluster that its closest mode belongs to

  14. Mean Shift Segmentation • Cluster pixels using mean shift • each cluster is a segment • Represent with color, position • important • color distances are not the same as position distances • choose one scale for each

  15. Evaluating Segmenters • Collect “correct” segmentations • from human labellers • these may not be perfect, but ... • Now apply your segmenter • Count • % human boundary pixels close to your boundary pixels -- Recall • % of your boundary pixels close to human boundary pixels -- Precision

  16. Segmentation codes

More Related