Large-Scale Content-Based Image Retrieval

1. Large-Scale Content-Based Image Retrieval Project Presentation CMPT 880: Large Scale Multimedia Systems and Cloud Computing Under supervision of Dr. Mohamed Hefeeda By: Ahmed Abdelsadek (aabdelsa@sfu.ca)

2. Outlines • Introduction • Project Scope • Work Flow • Image Features • Indexing and Retrieval • Matching • Evaluation • Conclusion

3. Introduction • Current image search engines rely heavily on text to retrieve images • User provides keywords, and images having that keyword in the filename or in nearby html are candidates for retrieval. • In this project we are willing to try content-based retrieval techniques where the query is an image.

4. Project Scope • Similarity using local features. • Extracting features from the reference images. • Index these features in efficient data structure in a scalable large scale environment • Process query images. • Search and Match. • This project is NOT • Recognition, Classification, Categorization

5. Work Flow

6. Image Features • Using SIFT features (Scale-invariant feature transform). • A SIFT feature is a selected image region (also called keypoint) with an associated descriptor. • A SIFT descriptor is a histogram of the image gradients surrounding a keypoint. • Using PCA for Dimension Reduction

7. KD-Tree • Using KD-Trees • Each tree level represent a dimension of a feature • Searching the index for the K-nearest neighbours

8. Logical View

9. Physical View

10. Matching • For each query we extract the features and then search the index for the K-NN features. • For each query feature, each neighbouring feature of it votes to certain image with a score of its rank. • The maximum 10 images for the voting array are reported as the most similar images.

11. Evaluation • Core KNN • Experiments on local machine. • Our results vs brute force • Image retrieval • CalTech, and TRICVID datasets • On amazon AWS cloud. • We 8 machines. • Dual core • 4 GB ram

12. Precision of KNN

13. Scanned Bins Size

14. Affect of Data Size

15. Image Recall @ K

16. First Correct @ K

17. Implementation Details • The system is implemented in Java • We use Hadoop 1.0.3 • We run cloud experiments on AWS services • S3 • EMR • We use some open source libraries • For images preprocessing we use : FFMPEG • For extracting SIFT features we use : VLFeat

18. Conclusion • We implement a full pipeline for image retrieval problem. • The framework can easily support different types of features, different indexing methods. • We show how we can build a big cloud system from small components.

19. Conclusion • Intersection with my research • Contributions • Feature Selection and Extraction • Implement Dimension Reduction • Design and Implement Map/Reduce Index • Implement Image Matching and Ranking

20. Questions ?

21. Thank you !