1 / 18

Performance Characterization of Video-Shot-Change Detection Methods

Performance Characterization of Video-Shot-Change Detection Methods. U. Gargi, R. Kasturi, S. Strayer Presented by: Isaac Gerg. What is a Shot?.

erasto
Download Presentation

Performance Characterization of Video-Shot-Change Detection Methods

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Performance Characterization of Video-Shot-Change Detection Methods U. Gargi, R. Kasturi, S. Strayer Presented by: Isaac Gerg

  2. What is a Shot? • “The process of identifying changes in the scene content of a video sequence so that alternate representations may be derived for the purposes of browsing and retrieval.” ~ Quoted directly • Shot – A sequence of frames shot from the same camera. • Shot-Change examples: cuts, transitions, wipes, etc.

  3. Why Do We Care? • Indexing – video retrieval • Compression (e.g. MPEG) – determining key frames. • Removing commercials! (TiVo)

  4. Preview • Create a method for measuring the performance of a shot-change algorithm. • Measure both false detections & missed detections. • Measure performance of both cut detection & gradual transition detection. • Apply shot-change algorithms to ground truth video sequence. • Perform measurements and throughput analysis. • Compare the results.

  5. Ground Truth Video Sequence • 640x480 @ 30 frames/s. • ~75 minutes in length • M-JPEG format • Human volunteers used to establish ground truth. • Custom software used to notate shot-change.

  6. Defining a Detection • Algorithm detection must occur within so many frames of ground truth detection. Mapping Range = RM • Cut changes: RM= 3Gradual Transition:RM=10

  7. Detection Performance Measurements Where: MD is Missed Detections; FA is False Alarms.

  8. Desirable Characteristics • 90%-95% recall with 70%-75% precision. • Robust. • Automatic thresholds. • High throughput. • Perform well on both cuts and gradual transitions.

  9. Algorithms Evaluated • Color Histograms • RGB, HSV, YIQ, XYZ, L*a*b, L*u*v, Munsell, Opponent • Frame Difference Measurements • Bin-to-bin Differences (B2B), Chi-square test, Histogram intersection, Average Color • Dimensionality – 1D, 2D, 3D • MPEG Algorithms – A, B, C, D, E, F • Block Matching Methods – A, B, C

  10. Best Methods - Cut • Histogram intersection: • 1D and 3D methods.

  11. Best Methods - Cut • MTM colorspace (many flops). • LAB appeared as good compromise when considering throughput. • Opponent (OPP) almost as good as LAB, but needs only integer computations. [image] Hall, E. L. . Computer Image Processing and Recognition. Academic Press, New York

  12. Best Methods - Cut • Best recall: MPEG-A, 97% with 6% precision. Uses only I frames. • Best precision: MPEG-D, 88% with 79% recall. Uses I, B, & P frames.

  13. Worst Methods - Cut • Chi-square test histogram difference: • Average color of a frame. • 2D methods.Indicates luminance is important. • YYY colorspace. Indicates color content is important • All the block-matching methods.

  14. Best Methods - Transition Only MPEG algorithms evaluated. • MPEG-D: Uses all frames (I, P, B). Uses multiframe differences to detect gradual transitions. Uses 11 parameters. • MPEG-F: Uses color information (Y, Cr, Cb). Uses order statistics to detect gradual transitions. Needs 7 parameters.

  15. Worst Methods - Transition • MPEG-A was the worst. Only contains I frames. • Most performed poorly as they expected a particular transition curve.

  16. MPEG - Source Effects • Desirable to have a good MPEG method independent of encoder. • Authors found dependence on algorithm performance and MPEG encoder used. • MPEG does not specify encoding method, only syntax of encoded bitstream. • Different error estimates or DCT matrices may be used during encoding. • MPEG-F appeared to be the most robust.

  17. Conclusions • Need accurate model of color. • Color & luminance information combined yield best results. • MPEG shot detection & gradual transition methods have a long way to go. Encoding too variable. • Gradual transitions not detected well by an of the MPEG methods.

  18. Questions?

More Related