240 likes | 407 Views
Video Trails: Representing and Visualizing Structure in Video Sequences. Vikrant Kobla David Doermann Christos Faloutsos. Outline. Background and Motivation Overview Video Trails Trail Segmentation Trail Classification Gradual Transition Detection Experiments and Results
E N D
Video Trails: Representing and Visualizing Structure in Video Sequences Vikrant Kobla David Doermann Christos Faloutsos
Outline • Background and Motivation • Overview • Video Trails • Trail Segmentation • Trail Classification • Gradual Transition Detection • Experiments and Results • Conclusion
Background and Motivation • Video is a valuable information resource • There are still few efficient ways to provide access to the information the video contains • Early work on indexing video treated video sequence as collections of still images, ignored the temporal structure • Efficient analysis and representation of the temporal structure of a video is necessary
Overview • Generate a trail of points (Video Trails) in a low-dimensional space • Segment the video trails • Classify each of those segmented trails into two types: Stationary (low activity) VS Transitional (high activity) 4. Detect gradual transition
Video Trails • Definition: A trail of points in a low-dimensional space where each point is derived from physical features of a single frame in the video clip • Features: DC coefficients of the luminance and chrominance components of an MPEG frame • Dimensionality Reduction (FastMap) initial feature vector a vector in that dimensional target dimension space FastMap
Example • Consider a video clip with a 320x240 frame size • Each frame has 20x15 MBs( Macroblock) • Each MB contains 6 DC coefficients ( 4 luminance and 2 chrominance) • Totally, 20x15x6=1800 coefficients (initial vector) 1800-by-1 vector (X1,X2,X3) 3 (target dimension) FastMap
Trail Segmentation • Segment the video in order to determine regions of high activity corresponding to transitions and low activity corresponding to individual shots • The problem of segmenting the video into sets of frames is transformed into the problem of splitting the video trails into smaller trails corresponding to segments of video
Splitting Algorithm • Start by placing the first point in a new trail • Consider each successive point in the sequence in order • Perform a test for “inclusion” of this point in the current trail • if (the test pass) • Include the point in the current trail • Move to the next point • Goto 2 • else • Close the current trail with the previous point as the last one • Start a new trail with only the current point • Goto 2
“Inclusion Test” • Marginal Cost:Total cost per point in the trail • Consider a clip with N frames • Assume there are m points in the current trail, denoted by set , and be the point being considered for inclusion • Define ,d is the dimensionality • So the new marginal cost is new marginal cost > previous one : not include new marginal cost < previous one : include
The sequence of frames that yield the sparse transition between the two dense clusters
Trail Classification • Classify each of those segmented trails into: Stationary (low activity) or Transitional (high activity) • Classification Criteria • Monotonicity W1=0.4 • Sparsity W2=0.3 • Convex Hull Volume Ratio W3=0.2 • MBR Shape W4=0.1
Monotonicity • If a trail is (close to) monotonic, in some direction,it’s likely transitional projection of distance along k • projected distance ratio • the length of MBR dimension k • Minimum projected distance ratio
Monotonicity (Normalization) • Recall: • W1 is the weight of monotonicity • Tlow is the lower bound=1.1 • Tup is the upper bound=2.0
Sparsity • Sparsity: total MBR volume per point • Average Sparsity • Sparsity Ratio • Normalize
Convex Hull Volume Ratio • The ratio of volume of the convex hull of points in a trail to the volume of MBR • Normalize
MBR Shape • Cuboidal • Planar • Elongated
Gradual Transition Detection • Dissolves, Fades, Wipes • Difficulty: activity arising from camera or large object motion also yields trails similar to trails resulting from gradual edits • Filter out any kind of global motion leading to a transitional trail, Analysis global motion
Conclusion • Provide a compact representation of a video sequence structure • Reduce a sequence MPEG frames to a trail of points in a low dimensional space • Segment trails and classify each segment as either stationary or transitional • Detect gradual edits