Hierarchical Indexing Structure for Efficient Similarity Search in Video Retrieval

1. Hierarchical Indexing Structure for Efficient Similarity Search in Video Retrieval Hong Lu, Beng Chin Ooi, Heng Tao Shen, Xiangyang Xue IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGG, VOL. 18, NOVEMBER 2006 Presented By :- Bhaumik Shah

2. Outline • Motivation • Video Management • Video Data Management • Content-based video retrieval and Indexing • Hierarchical Indexing Structure for efficient video retrieval. (OVA - File) • Compare with previous methods (VA-File and iDistance) • Conclusion • References

3. Motivation • Large amount of distributed Web Video resources and rapid increase in centralized video archives, tremendous amount of video data being generated. • Now a user will ask for specific part of video for this content-based retrieval is gaining an importance. To support such applications efficiently, Content-based video indexing must be addressed

4. Motivation • Finding a desired video data from a large amount of distributed databases remains a very difficult and time-consuming task. • Lack of tools for classify and retrieve video content and complexity in video indexing motivated researchers to find some structure which is efficient as well as give better results.

5. Video Management Video Contents :: Low-level visual content features :: colors, shapes, textures  Semantic contents features :: high-level concepts such as objects and events

6. Video Data Management Video Parsing Manipulation of whole video for breakdown into key frames. Video Indexing Retrieving information about the frame for indexing in a database. Video Retrieval and browsing Users access the db through queries or through interactions.

7. Video Parsing • Scene: single dramatic event taken by a small number of related cameras. • Shot: A sequence taken by a single camera • Frame: A still image

8. Video Parsing Obvious Cuts Video Scenes Shot Boundary Analysis Shots Key Frame Analysis Frames

9. Video Parsing

10. How to retrieve results…

11. Content-based Video Retrieval and Indexing • A specific part of the video, contain some semantic information. • Query results for this specific part can be presented through many visual presentations. • The process of extracting the semantic content is more complex because, it requires domain knowledge or user interaction, while extraction of visual features is usually domain independent.

12. Example of Content-based Video Retrieval • Specialized Search Engine for Australian Open Tennis Tournament Website. URL :: http://tournament.ausopen.org/ A very good example of Content-based video retrieval and finding semantic content. “Show me video scenes of left-handed female players who have won the Australian Open in the past”

13. Content-based Video Indexing • Process of attaching content-based labels to video units, which we called clips. • video indexing is the process of extracting from the video data the temporal location of a feature and its value • High-Dimensional Indexing :: Index terms are organized based on high-level categories like action, time, event, etc.

14. Hierarchical Indexing Structure ( OVA –File ) OVA – Ordered Vector Approximation • Dynamic high-dimensional indexing structure is needed for fast similarity query in typical multimedia applications and also multiframe video representation increases the problem complexity. • This dynamic high-dimensional indexing structure is called OVA – File (Ordered VA File) which is based on the VA-file.

15. Content-based Video Indexing problem Solution using OVA- File • OVA-File is a hierarchical structure and it has tow novel features :: 1) partitioning the whole file into slices such that only a small number of slices are accessed and checked during k Nearest Neighbor (kNN) search 2) efficient handling of insertions of new vectors into OVA-File, such that an average distant between the new vectors and those approximations near that position is minimized.

16. OVA – File Structure • It has three layers 1) Original vector file 2) Ordered approximation file ( OVA-Slice file) 3) Slice summaries file

17. OVA- File Structure Diagram

18. OVA – File Structure ( How to create ?? ) • Original vector file contains high-dimensional vectors, which represents visual features of the frames. • To create OVA-File we first obtain an Ordered approximation file. The benefit of it is approximations close to each other in data space are placed in the closed positions. • Ordered approximation file which segmented into small OVA-Slices which are further summarized into a slice summaries file to facilitate the retrieval.

19. OVA – File Structure continue … • With OVA-File kNN query processing method called OVA-LOW is also used. • In this kNN search, the principle is to look into only a portion of OVA-Slices that are most likely to contain the desired query results, instead of sequentially scanning all of them. • Therefore, the query response time of OVA-File would be reduced greatly.

20. Comparison with previous methods(Advantages of OVA- File Structure) • Previous methods :: VA-file method iDistance method (non VA-file method) • Issues :: Better Performance More Efficient and Effective Compatible with all video types Less query response time Good quality result Less number of disk accesses respect to iDistance method

21. Conclusion • OVA-File is an efficient hierarchical indexing structure used for content based video retrieval and better query results. • Because the approximation file of OVA-File is virtually that of equal to VA-File, any query search algorithm based on VA-File will be applicable to OVA-File. • OVA-File also proposed efficient video-retrieval method.

22. References [1] I. Koprinska and S. Carrato, “Temporal Video Segmentation: A Survey,” Signal Processing: Image Comm., vol. 16, pp. 477-500, 2001. [2] A. Hanjalic, “Shot-Boundary Detection: Unraveled and Resolved?” IEEE Trans. Circuits and Systems for Video Technology, vol. 12, no. 2, pp. 90-105, 2002. [3] A. Girgensohn and J.S. Boreczky, “Time-Constrained Keyframe Selection Technique,” Multimedia Tools and Applications, vol. 11, no. 3, pp. 347-358, 2000. [4] T. Liu and J.R. Kender, “Optimization Algorithms for the Selection of Key Frame Sequences of Variable Length,” Proc. European Conf. Computer Vision, pp. 403-417, 2002. [5] N. Dimitrova, H.-J. Zhang, B. Shahraray, M. Sezan, T. Huang, and A. Zakoh, “Applications of Video-Content Analysis and Retrieval,” IEEE Trans. Multimedia, vol. 9, no. 3, pp. 42-55, 2002. [6] Y.A. Aslandagan and C.T. Yu, “Techniques and Systems for Image and Video Retrieval,” IEEE Trans. Knowledge and Data Eng., vol. 11, no. 1, pp. 56-63, Jan./Feb. 2002.

23. References [7] G. Lu, “Techniques and Data Structures for Efficient Multimedia Retrieval Based on Similarity,” IEEE Trans. Multimedia, vol. 4, no. 3, pp. 372-384, 2002. [8] J.S. Boreczky and L.A. Rowe, “Comparison of Video Shot Boundary Detection Techniques,” SPIE Proc. Storage and Retrieval for Still Image and Video Databases IV, vol. 2670, pp. 170-179, Mar. 1996. [9] U. Gargi, R. Kasturi, and S.H. Strayer, “Performance Characterization of Video-Shot-Change Detection Methods,” IEEE Trans. Circuits and Systems for Video Technology, vol. 10, no. 1, pp. 1-13, 2000. [10] C. Bo¨hm, S. Berchtold, and D. Keim, “Searching in High- Dimensional Space: Index Structures for Improving the Performance of Multimedia Databases,“ vol. 33, no. 3, pp. 322-373, 2001. [11] M. Flickner, H. Sawhney, W. Niblack, J. Ashley, Q. Huang, B. Dom, M. Gorkani, J. Hafner, D. Lee, D. Petkovic, D. Steele, and P. Yanker, “Query by Image and Video Content: The QBIC System,” Computer, vol. 28, no. 9, pp. 23-32, 1995.