Color-Attributes-Related Image Retrieval Week 4

1. Color-Attributes-Related Image RetrievalWeek 4 Student: Kylie Gorman Mentor: Yang Zhang

2. Debugging Code • Isolate object in binary image • Take corresponding boxes in HSV image • Speed up processing • Find bugs in original code and rewrite • Addition: break up individual channel • Set threshold to half the block size: 32

3. HSV Feature Matrix

4. “Object Only” Feature Matrix

5. Revised Google Steps • Calculate feature matrix • Concatenate feature matrices • Calculate PCA and GMM • Multiply individual feature matrices by coefficient matrix • Use GMM results to calculate feature vectors • Train 11 SVM’s

6. Revised Steps for EBay Data • Calculate feature matrix of each image, isolating the object first • Use PCA and GMM results from training data (Google data) to calculate fisher vectors • Apply Fisher Vector to each individual result to obtain vectors that are the same size • Classify eBay images using 11 SVM’s from training data • Calculate Precision

7. Train Data Test Data Features Features Fisher GMM PCA Fisher Classify 11 SVM Precision

8. Precision Results Determine if system made the correct assertion and calculate accuracy Program works for any 3 channel color space

9. Test Data: EBay Dress Data Set: Images of Dresses HSV Images 12 Images Per Color 132 Images Total

10. Test Data: EBay Dress Data Set: Images of Dresses RGB Images 12 Images Per Color 132 Images Total

11. Test Data: EBay Dress Data Set: Images of Dresses CIELAB Images 12 Images Per Color 132 Images Total

12. Interpretation • Average precision for HSV images: ~25% • Average precision for RGB images: ~20% • Average precision for CIELAB images: ~17% • Incorrect images were not removed from test data • The objects were not isolated in the training data • Switch training and testing data: ~16% • Most likely errors • Still requires debugging

13. Future Goals • Apply Code to new Data Sets • Google-512: 12 images per color, 5632 images total • FahadShahbaz Khan, Rao Muhammad Anwer, Joost van de Weijer, Andrew D. Bagdanov, Maria Vanrell, Antonio M. Lopez • Image retrieval test • Object Detection