1 / 24

Video Trails: Representing and Visualizing Structure in Video Sequences

This publication discusses the efficient analysis and representation of the temporal structure in video sequences through Video Trails. It covers trail segmentation, classification, and gradual transition detection using features from MPEG frames and FastMap dimensionality reduction.

lkathie
Download Presentation

Video Trails: Representing and Visualizing Structure in Video Sequences

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Video Trails: Representing and Visualizing Structure in Video Sequences Vikrant Kobla David Doermann Christos Faloutsos

  2. Outline • Background and Motivation • Overview • Video Trails • Trail Segmentation • Trail Classification • Gradual Transition Detection • Experiments and Results • Conclusion

  3. Background and Motivation • Video is a valuable information resource • There are still few efficient ways to provide access to the information the video contains • Early work on indexing video treated video sequence as collections of still images, ignored the temporal structure • Efficient analysis and representation of the temporal structure of a video is necessary

  4. Overview • Generate a trail of points (Video Trails) in a low-dimensional space • Segment the video trails • Classify each of those segmented trails into two types: Stationary (low activity) VS Transitional (high activity) 4. Detect gradual transition

  5. Video Trails • Definition: A trail of points in a low-dimensional space where each point is derived from physical features of a single frame in the video clip • Features: DC coefficients of the luminance and chrominance components of an MPEG frame • Dimensionality Reduction (FastMap) initial feature vector a vector in that dimensional target dimension space FastMap

  6. Example • Consider a video clip with a 320x240 frame size • Each frame has 20x15 MBs( Macroblock) • Each MB contains 6 DC coefficients ( 4 luminance and 2 chrominance) • Totally, 20x15x6=1800 coefficients (initial vector) 1800-by-1 vector (X1,X2,X3) 3 (target dimension) FastMap

  7. Example

  8. Example

  9. Trail Segmentation • Segment the video in order to determine regions of high activity corresponding to transitions and low activity corresponding to individual shots • The problem of segmenting the video into sets of frames is transformed into the problem of splitting the video trails into smaller trails corresponding to segments of video

  10. Splitting Algorithm • Start by placing the first point in a new trail • Consider each successive point in the sequence in order • Perform a test for “inclusion” of this point in the current trail • if (the test pass) • Include the point in the current trail • Move to the next point • Goto 2 • else • Close the current trail with the previous point as the last one • Start a new trail with only the current point • Goto 2

  11. “Inclusion Test” • Marginal Cost:Total cost per point in the trail • Consider a clip with N frames • Assume there are m points in the current trail, denoted by set , and be the point being considered for inclusion • Define ,d is the dimensionality • So the new marginal cost is new marginal cost > previous one : not include new marginal cost < previous one : include

  12. Example

  13. Example (close-up)

  14. The sequence of frames that yield the sparse transition between the two dense clusters

  15. Trail Classification • Classify each of those segmented trails into: Stationary (low activity) or Transitional (high activity) • Classification Criteria • Monotonicity W1=0.4 • Sparsity W2=0.3 • Convex Hull Volume Ratio W3=0.2 • MBR Shape W4=0.1

  16. Monotonicity • If a trail is (close to) monotonic, in some direction,it’s likely transitional projection of distance along k • projected distance ratio • the length of MBR dimension k • Minimum projected distance ratio

  17. Monotonicity (Normalization) • Recall: • W1 is the weight of monotonicity • Tlow is the lower bound=1.1 • Tup is the upper bound=2.0

  18. Sparsity • Sparsity: total MBR volume per point • Average Sparsity • Sparsity Ratio • Normalize

  19. Convex Hull Volume Ratio • The ratio of volume of the convex hull of points in a trail to the volume of MBR • Normalize

  20. MBR Shape • Cuboidal • Planar • Elongated

  21. Classification

  22. Gradual Transition Detection • Dissolves, Fades, Wipes • Difficulty: activity arising from camera or large object motion also yields trails similar to trails resulting from gradual edits • Filter out any kind of global motion leading to a transitional trail, Analysis global motion

  23. Results

  24. Conclusion • Provide a compact representation of a video sequence structure • Reduce a sequence MPEG frames to a trail of points in a low dimensional space • Segment trails and classify each segment as either stationary or transitional • Detect gradual edits

More Related