Content-Based Image Retrieval - Approaches and Trends of the New Age

1. Content-Based Image Retrieval - Approaches and Trends of the New Age Ritendra Datta, Jia Li, and James Z. Wang The Pennsylvania State University MIR2005

2. INTRODUCTION • 為什麼image無法處理的像text一樣好 • Text is man’s creation, images are a mere replica of what man has seen • Interpretation of what we see is hard to characterize • visual similarity != semantic similarity • CBIR has grown tremendously after 2000, not just in terms of size, but also in the number of new directions explored

3. INTRODUCTION

4. INTRODUCTION • The theoretical foundation behind how we humans interpret images is still an open problem • A brief scanning of about 300 relevant papers published in the last five years revealed that less than 20% were concerned with applications or real-world systems

5. CBIR領域研究方向 • Feature Extraction • Approaches to Retrieval • Annotation and Concept Detection • Relevance Feedback and Learning • Hardware and Interface Support

6. Feature Extraction • 如何抽 Color Feature • “An Efficient Color Representation for Image Retrieval” (比傳統histograms好) • “Multiresolution Histograms and Their Use for Recognition” (用在textured image) • “Image retrieval using color histograms generated by Gauss mixture vector quantization” (利用GMVQ抽color histogram)

7. Feature Extraction • Color + Texture 抽取 • “Wavelet-Based Texture Retrieval Using Generalized Gaussian Density and Kullback-Leibler Distance” • Shape • “Shape Matching and Object Recognition Using Shape Contexts” (is fairly compact yet robust to a number of geometric transformations)

8. Feature Extraction • Segmentation • “Normalized Cuts and Image Segmentation” (最重要的方向之一) • “Blobworld: Image Segmentation Using Expectation-maximization and Its Application to Image Querying” (我之前用過的方法) • “Segmentation of Brain MR Images Through a Hidden Markov Random Field Model and the Expectation-Maximization Algorithm” (處理medical imaging)

9. Feature Extraction • 線條相似度 • “Image retrieval using wavelet-based salient points” • 如何選擇feature • Application-specific feature sets (最直觀的) • “SIMPLIcity:Semantics-Sensitive Integrated Matching for Picture Libraries” (semantics-sensitive feature selection) • “Feature Selection for SVMs” (用classifier)

10. Approaches to Retrieval • Region based image retrieval • “A Scalable Integrated Region-Based Image Retrieval System” • region-based querying (BlobWorld) • Vector quantization (VQ) on image blocks • “Keyblock: An Approach for Content-based Image Retrieval” (generate codebooks for representation and retrieval, taking inspiration from data compression and text-based strategies)

11. Approaches to Retrieval • Windowed search • “Object-Based Image Retrieval Using the Statistical Structure of Images” (more effective than methods based on inaccurate segmentation) • Anchoring-based image retrieval • “A Study of Image Retrieval by Anchoring” (Anchoring is based on the idea of finding a set of representative “anchor” images and deciding semantic proximity between an arbitrary image pair in terms of their similarity to these anchors)

12. Approaches to Retrieval • Probabilistic frameworks for image retrieval • “A Probabilistic Architecture for Content-based Image Retrieval”

13. Annotation and Concept Detection • Supervised classification • “Image Classification for Content-Based Indexing” (involving simple concepts such as city, landscape, sunset,and forest, have been achieved with high accuracy) • Translation approach • “Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary” (我們在clef 2004就是follow這方法)

14. Annotation and Concept Detection • 為何如此困難 • We humans segment objects better than machines, having learned to associate over a long period of time, through multiple viewpoints, and literally through a “streaming video” at all times • The association of words and blobs become truly meaningful only when blobs isolate objects well

15. Relevance Feedback and Learning • “Relevance Feedback in Image Retrieval: A Comprehensive Review” • Problems • One problem with RF is that after every round of user interaction, usually the top results with respect to the query have to be recomputed • Another issue is the user’s patience in supporting multi-round feedbacks

16. REAL-WORLD REQUIREMENTS • Performance • Semantic learning • Volume of Data • Concurrent Usage • Heterogeneity • Multi-modal features • User-interface • Operating Speed • System Evaluation

17. CURRENT RESEARCH TRENDS • Journals • IEEE T. Pattern Analysis and Machine Intelligence (PAMI) • IEEE T. Image Processing (TIP) • IEEE T. Circuits and Systems for Video Technology (CSVT) • IEEE T. Multimedia (TOM) • J. Machine Learning Research (JMLR) • International J. Computer Vision (IJCV)

18. CURRENT RESEARCH TRENDS • Pattern Recognition Letters (PRL) • ACM Computing Surveys (SURV) • Conferences • IEEE Computer Vision and Pattern Recognition (CVPR) • International Conference on Computer Vision (ICCV) • European Conference on Computer Vision (ECCV) • IEEE International Conference on Image Processing (ICIP)

19. CURRENT RESEARCH TRENDS • ACM Multimedia (MM) • ACM SIG Information Retrieval (IR) • ACM Human Factors in Computing Systems (CHI)

22. CONCLUSIONS • We have presented a brief survey on work related to the young and exciting fields of content-based image retrieval and automated image annotation, spanning 120 publications in the current decade • We have laid out some guidelines for building practical, real-world systems