1 / 16

Yu-Gang Jiang, Yanran Wang , Rui Feng Xiangyang Xue , Yingbin Zheng , Hanfang Yang

Understanding and Predicting Interestingness of Videos. Yu-Gang Jiang, Yanran Wang , Rui Feng Xiangyang Xue , Yingbin Zheng , Hanfang Yang. Fudan University, Shanghai, China. AAAI 2013, Bellevue, USA, July 2013. Motivation. Large amount of videos on the Internet

brasen
Download Presentation

Yu-Gang Jiang, Yanran Wang , Rui Feng Xiangyang Xue , Yingbin Zheng , Hanfang Yang

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Understanding and Predicting Interestingness of Videos Yu-Gang Jiang, Yanran Wang, RuiFeng XiangyangXue, YingbinZheng, HanfangYang Fudan University, Shanghai, China AAAI 2013, Bellevue, USA, July 2013

  2. Motivation • Large amount of videos on the Internet • Consumer Videos, advertisement… • Some videos are interesting, while many are not Moreinteresting Lessinteresting TwoAdvertisementsofdigitalproducts

  3. Applications • Web Video Search • Recommendation System • ...

  4. Related Work … … … • Predicting Aesthetics and Interestingness of Images • Datta et al. ECCV, 2006; Dhar et al. CVPR, 2011; N. Murray et al. CVPR, 2012… • We are the first to explore the interestingness of Videos Moreinteresting Less interesting

  5. Two New Datasets • Flickr • source: Flickr.com Consumer Video • videos: 1200 (20 hrs in total) • YouTube • source: Youtube.com Advertisement Video • videos: 420 (4.2 hrs in total)

  6. Flickr Dataset • Collected by 15 interestingness-enabled queries • Top 10% of 400 as interesting videos; Bottom 10% as uninteresting • 80 videos per category/query

  7. YouTube Dataset • Collected by 15 ads queries on YouTube • 10 human assessors (5 females, 5 males) • Compare video pairs General observation: videos with humorous stories, attractive background music, or better professional editing tend to be more interesting AnnotationInterface

  8. Our Computational Framework • Aim: compare two videos and tell which is more interesting Visual features Audio features Multi-modal fusion Ranking SVM vs. results High-level attribute features

  9. Feature Rule of Thirds Vanishing Point Soft Focus Motion Blur ShallowDOF … Flower, Tree, Cat, Face…

  10. Prediction & Evaluation • Prediction • Ranking SVM trained on our dataset • Chi square kernel for histogram-like features • RBF kernel for the others • 2/3 for training and 1/3 for testing • Evaluation • Prediction accuracy • The percentage of correctly ranked test video pairs

  11. Visual Feature Results 76.6 74.5 Flickr Prediction Accuracies(%) 68.0 67.0 67.1 YouTube

  12. Audio Feature Results 74.7 Flickr Prediction Accuracies(%) 65.7 64.8 YouTube

  13. Attribute Feature Results 74.8 64.5 Flickr Prediction Accuracies(%) Different from predicting Image Interestingness 64.3 YouTube

  14. Visual+Audio+Attribute Results 78.6 76.6 2.6% Flickr Prediction Accuracies(%) Visual+Audio Audio Attribute Visual Visual+Audio+Attribute 71.7 68.0 5.4% YouTube

  15. Summary • Conducted a pilot study on video interestingness • Built two datasets to support this study • Publicly available at: www.yugangjiang.info/research/interestingness • Evaluated a large number of features • Visual + audio features are very effective • A few features useful in image interestingness do not work in video domain (e.g., Style Attributes…)

  16. Thank you ! Datasets are available at: www.yugangjiang.info/research/interestingness

More Related