1 / 32

Temporal Video Boundaries

Temporal Video Boundaries. Computer Science Engineering Lee Sang Seon. Why Temporal Video Boundaries Technique is useful in the Video content analysis?. Index. Introduction Basic notions for temporal video boundaries Micro-Boundaries Macro-Boundaries Mega-Boundaries Conclusion

emile
Download Presentation

Temporal Video Boundaries

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Temporal Video Boundaries Computer Science Engineering Lee Sang Seon

  2. Why Temporal Video Boundaries Technique is useful in the Video content analysis?

  3. Index • Introduction • Basic notions for temporal video boundaries • Micro-Boundaries • Macro-Boundaries • Mega-Boundaries • Conclusion • Q & A

  4. Introduction • Brief definition of Temporal Video Boundary technique → Examine the temporal boundary problem at different levels of video content structure analysis • Why we need Temporal Video Boundary technique? Show example

  5. Example : Oscar awards opening Insufficient metadata ending

  6. Example : Oscar awards opening actor Detailed metadata winners awards ending ending

  7. Basic notions - modalities • Video contains three types of modalities (i) Visual (ii) Audio (iii) Textual • Each modality has three levels (i) low-level (ii) mid -level (iii) high-level → levels describe the amount of details described in each modality in terms of granularity and abstraction

  8. Basic notions - modalities • For each modality and for each level there if a set of attributes. These can be formalized as vectors:

  9. Basic notions - modalities • Adding to this, given a set of vectors → their average value denote the vector

  10. Basic notions - method • Local method → the difference is computed between consecutive frames • Global method → the difference if computed over a series of frames

  11. Micro-Boundaries • Definition • Boundaries associated to the smallest video units for which a given attribute is constant or slowly varying • The attribute can be any feature in the visual, audio, or text domain

  12. Example

  13. Make family histogram = Frame histogram Data structure that represents the color information of a family of frames. Set of frames that exhibits uniform features

  14. Histogram difference measures • Histogram difference using L1 metrics • Bin-wise histogram intersection Total number of color bins used Histogram of current frame Histogram of previous frame

  15. Merging of family histograms

  16. Multiple ways to compare and merge families - contiguity & memory 1. Contiguous with zero memory → A new frame histogram is compared with previous frame histogram 2. Contiguous with limited memory → A new frame histogram is compared with previous family histogram

  17. Multiple ways to compare and merge families - contiguity & memory 3. Non contiguous with unlimited memory → A new frame histogram is compared with all previousfamily histograms within the same video. 4. Hybrid → First a new frame histogram is compared using the contiguous frames and then generated family histograms are merged using non contiguous case.

  18. Compare different Histogram difference measures

  19. Macro-Boundaries • Definition • Boundaries between collections of video micro-segments that are clearly identifiable organic parts of an event defining a structural (action) or thematic (story) unit • Video : collection of stories that may or may not be interconnected → Macro-Boundaries detection = Segmenting stories textual cues visual cues audio cues

  20. Two types of uniform segment detection • Unimodal segment detection • A video segment exhibits same characteristic over a period of time • Multimodal segment detection • A video segment exhibits a certain characteristic taking into account attributes from different modalities

  21. Single Modality Segmentaion Audio segmentation & classification Text transcript Partition a continuous bitstream of audio data into non-overlapping segments Extracted from either the closed captions or speech-to-text conversion Classification Frequency-of-word-occurrence metric is used Using low-level audio features Segmented and categorized with respect to a predefined topic list Seven mid-level audio categories

  22. Multimodal Segmentaion Goal : Create macro-boundaries that are more accurate than the boundaries produced by individual modalities.

  23. Descent Methods Text segment Audio segment Video segment

  24. Single descent Method

  25. Mega-Boundaries • Definition • Boundaries between collections of macro-segments that exhibit different structural and feature consistency (e.g. different genres) • Example • Commercial detection method

  26. Trigger & Verifiers Model

  27. Black frames Letterbox change High cut rate(= low cut distance)

  28. Bayesian Belief Network Model start

  29. Genetic Algorithms

  30. Conclusion

  31. Whenever metadata is available or unavailable, we can segment video by using this technique that categorized three types

  32. Thank you! & Q & A

More Related