1 / 27

Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes

Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes. IEEE TCSVT 2011 Wonjun Kim Chanho Jung Changick Kim. Outline. Introduction Proposed Method Experiment Result Application Conclusion. Introduction. Problem occurs when background is highly textured.

debra
Download Presentation

Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes IEEE TCSVT 2011 Wonjun Kim Chanho Jung Changick Kim

  2. Outline • Introduction • Proposed Method • Experiment Result • Application • Conclusion

  3. Introduction • Problem occurs when background is highly textured

  4. Proposed Method • Feature Representation • Edge orientation histogram (EOH) • Color orientation histogram (COH) • Temporal Feature • Self-ordinal Measure • Saliency Map • Scale-invariant Saliency Map

  5. Edge Orientation Histogram (EOH) • Compute the edge orientation of every pixel in the local region center at the pixel • Quantized into K angle in the range of [,] • Compute the histogram of edge orientation local region m(x,y,n):edge magnitude (x,y,n):quantized orientation

  6. Color Orientation Histogram (COH) • Quantize the angle in HSV color space in the range of [,] into H angles • Compute the histogram of color orientation s(x,y,n):saturation value (x,y,n):quantized hue value

  7. Temporal Feature • Compute the intensity differences between frames • Feature at the pixel of frame P :total number of pixels in local region j :index of those pixels in P :user-defined latency

  8. Self-ordinal Measure • Define a 1(K+1) rank matrix by ordering the elements of EOH(COH) ex:

  9. Self-ordinal Measure

  10. Saliency Map of Edge and Color • Compute the distance from the rank matrix of center region to surrounding regions Saliency Map of Edge Saliency Map of Color N :total number of local regions in a center-surround window ,:maximum distance between two rank matrices

  11. Spatial Saliency Map • Combine the edge and color saliency

  12. Combining with Temporal Saliency • Compute the SAD of temporal gradients between center and the surrounding regions • Combine the spatial and temporal saliency

  13. Scale-invariant Saliency Map • Combine 3 different scales of saliency Map(3232, 6464, 128128) 3232 128128 6464

  14. Algorithm

  15. Experiment Result • Static Images • Video Sequences

  16. Experiment Result • Static Image • Local region = 55 • center-surround window = 77 • K = 8, H= 6 • = 40, = 24 • Video Sequence • = 49 • Speed: 23ms per frame (43 fps)

  17. Static Images

  18. Static Images

  19. Video Sequences

  20. Video Sequences

  21. Application • Image Retargeting • Moving Object Extraction

  22. Image Retargeting

  23. Image Retargeting

  24. Moving Object Detection • G:the set of salient pixels in the ground truthimage • P:salient pixels in the binarized object map • Card(A):the size of the set A

  25. Moving Object Detection

  26. Conclusion • Ordinal signature can tolerate more local feature distribution than sample values. • The proposed scheme performs in real-time and can be extended in both static and dynamic scenes.

More Related