1 / 33

IBM QBIC: Query by Image and Video Content

IBM QBIC: Query by Image and Video Content. Jianping Fan. Department of Computer Science. University of North Carolina at Charlotte. Charlotte, NC 28223. http://www.cs.uncc.edu/~jfan. Networks. How can I access video in database over networks?. Challenge Problems:.

zulma
Download Presentation

IBM QBIC: Query by Image and Video Content

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. IBM QBIC: Query by Image and Video Content Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC 28223 http://www.cs.uncc.edu/~jfan

  2. Networks How can I access video in database over networks? Challenge Problems: 1. How to access video ? 2. How to represent video ? 3. How to index large-scale videos ? 4. How to access videos in database ? 5. How to transmit query results over IP ? 6. How to control user’s access ?

  3. 1. System Architecture How to access video in database? Images Videos R-frames Shot extraction 1. Representative Frame 2. Motion-based objects Object Identification 3. shots Feature extraction Motion objects scene objects shots scene object video sketch User-defined Positional color/texture Object motion Camera motion Location/color Color/texture

  4. 1. System Architecture How to access video in database? user Query interface Color Texture Shape Multi-object Sketch Location Text Positional color/texture object motion camera motion user defined existing image Match engine Database Indexing Returned via similarity order

  5. 2. How to access image or video in QBIC? a. Video shots b. Video objects or sketches & drawings c. Representative frames d. Camera & object motion e. Other graphical information Access approaches a. Example images b. user-constructed sketches or drawings c. Selected color and texture patterns d. Motion types e. Other information

  6. 2. How to obtain accessing units in QBIC? Shot Detection

  7. 2. How to obtain accessing units in QBIC? a. Difference Calculations b. Automatic Decision Making Via Pre-Defined Thresholds.

  8. How we do the shot detection?

  9. 2. How to obtain accessing units in QBIC? Object extraction

  10. 2. How to obtain accessing units in QBIC? Object extraction

  11. How we can do the object extraction?

  12. 3. How to represent these video units? Images Global color Global texture Positional color Positional texture Sketch, shape User-defined color/texture

  13. 3. How to represent these video units? Videos Global color Global texture Positional color Positional texture Sketch, shape User-defined color/texture Camera motion/object motion

  14. How we do the shot representation? Video Sequence Shot 1 Shot i Shot n Color HSV color histogram, dominant color, … Texture Edge histogram, wavelet coefficients, Tamura features, … Motion Directional motion histogram, Camera motion, … Other features

  15. 4. How to index images/videos in QBIC? .. feature space . . . . . . . . . . . . . . . . . Videos in . . . Database . . . . . .

  16. 4. How to index images/videos in QBIC? High-dimensional visual features K-L Transform to reduce dimensions Low-dimensional R*-tree indexing

  17. 4. How to index images/videos in QBIC? Overlap on two Dimensions!

  18. 4. How to index images/videos in QBIC? Karhunen-Loeve Transformation New Eigenvectors M is the matrix for videos! S is the KL transform matrix!

  19. 5. How to realize query in QBIC?

  20. 5. How to realize query in QBIC?

  21. 5. How to realize query in QBIC?

  22. How we can do the video query?

  23. 5. How to realize query in QBIC?

  24. How we can do the mosaic?

  25. Why we use mosaic for video representation?

  26. 6. What lost by QBIC? a. Mapping from visual features to semantic concepts It is hard, but we have to do this. Why? Visual Features Semantic Clusters Weighted mapping? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Video Contents in Database How to do this mapping?

  27. 6. What lost by QBIC? b. High-dimensional visual indexing It is a basic problem in database area, but only database people cannot solve this challenging problem for visual indexing! Video in Database Cluster 1 Cluster i Cluster n Subcluster n1 Subcluster 11 Subcluster 1j Subcluster nl Subregion 111 Subregion nl1 Subregion nlm Subregion 11k object nlm1 object1111 Disk for Cluster i Disk for Cluster n Disk for Cluster 1

  28. 6. What lost by QBIC? c. User input in the query procedure: QBIC can permit user to select something.

  29. 6. What lost by QBIC? d. How to integrate keywords with visual features?

  30. 6. What lost by QBIC? e. How to provide user-intensive browsing?

  31. 7. What happen now on QBIC? You can find the current version of QBIC system on: http://wwwqbic.almaden.ibm.com/ Homework: What kind of technique we have discussed used in QBIC?

  32. 8. Other Projects • Chabot at UC Berkeley • http://www.cs.berkeley.edu/~ginger/chabot.html • b. Viper at Europe • http://viper.unige.ch/ • c. Virage • http://www.virage.com/

  33. 8. Other Projects d. PicHunter at NEC http://www.neci.nj.nec.com/homepages/vision/index.html e. Ifind at Microsoft http://wwww.microsoft.com/china/research/group/ f. Photobook at MIT http://www-white.media.mit.edu/~tpminka/photobook/

More Related