1 / 17

Scalable Image Matching

Scalable Image Matching. David Strickland ENGN 256 Spring 2013. Reference Paper: Inverted Index Compression for Scalable Image Matching.

olive
Download Presentation

Scalable Image Matching

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Scalable Image Matching David Strickland ENGN 256 Spring 2013

  2. Reference Paper: Inverted Index Compression for Scalable Image Matching • Chen, D.M.; Tsai, S.S.; Chandrasekhar, V.; Takacs, G.; Vedantham, R.; Grzeszczuk, R.; Girod, B., "Inverted Index Compression for Scalable Image Matching," Data Compression Conference (DCC), 2010 , vol., no., pp.525,525, 24-26 March 2010

  3. Scalable Image Matching for the BlindFind Project • BlindFind aims to help the blind navigate unfamiliar indoor environments to locate places via a wearable navigation device powered by crowd-sourced maps • The device needs to know its map location • Image matching is required • Image matching must be fast and real time • Needs to be scalable

  4. Image Matching • Most solutionsare base on local image features: • SIFT (Scale-Invariant Feature Transform) • SURF (Speeded Up Robust Features” • CHoG (Compressed Histogram of Gradients) • Process: • Detect Features • Extract Feature Locations and Descriptors • Compare the Features of two images to determine similarity

  5. Example of Features http://en.wikipedia.org/wiki/File:Sift_keypoints_filtering.jpg

  6. Problem • Comparing an image with every image in a large database takes an extremely long amount of time • Doesn’t scale • Some databases contain millions of images

  7. Solution: Vocabulary Tree + Inverted Index • Detect Features • Extract Feature Locations and Descriptors • Quantize Descriptors into a Vocabulary Tree • Score Database Images using Inverted Index • Pairwise Match using Geometric Check

  8. Vocabulary Tree + Inverted Index • The Vocabulary Tree is a tree-structured vector quantizer constructed by hierarchical k-means clustering of feature descriptors. • Inverted Index: Each node has two lists • Image IDs • Array of counts 1Image from Chen et al.

  9. Similarity Scoring & Memory Usage • Each image ik1 in the database of N images is given a similarity score • For each node visited by query descriptors the node’s inverted list of images all have the scores incremented : Where:

  10. New Problem • Inverted index requires lots of memory • Memory usage for VT with K leaf nodes, where Nk database images have visited each node: • Need to find a way to reduce memory usage of the inverted index

  11. New Solutions for Dealing with Large Amounts of Data • Fast decoding methods • Carryover Code (32 bit word) • Recursive Bottom Up Complete (RBUC) • Inverted Index Compression • Encode Image IDs by consecutive differences • Reorder database to minimize differences • Soft-binned feature descriptor histograms • Improves accuracy of VT search

  12. Fast decoding methods • Carryover Code (32 bit word) • 2-bit selector + 30 bit data portion • Selector indicates the precision of the data portion • E.g. 30 1 bit data symbols, 15 2 bit data symbols, etc. • Recursive Bottom Up Complete (RBUC) • Similar to carryover code encoding • The precision for each pair is calculated as the max of the two precisions • Then P’ itself is encoded

  13. Inverted Index Compression • Encode each inverted index’s Image IDs by consecutive differences • Inverted index compression techniques can significantly reduce memory usage by up to 5X without any loss in recognition accuracy • Reorder database to minimize differences • Minimize:

  14. Soft-binned feature descriptor histograms • Classify a feature descriptor to k nearest tree nodes instead of just nearest tree node • Soft-binned tree gives improvement in classification accuracy • Disadvantage: • Each database feature now appears in k different inverted lists • Inverted Index is k times as large

  15. Schedule I: VT/II Implementation • Week 1: Research Vocabulary Tree / Inverted Index, Determine which libraries to use • Week 2: Implement Feature Locator/Descriptors • Week 3: Implement Quantization of Descriptors in VT • Week 4: Implement Database scoring scheme using Inverted Index • Week 5: Milestone: Mid Project Presentation, Combine Previous parts, Pairwise Match to retrieve a single image

  16. Schedule II: Compression • Week 6: Inverted Index Image ID storage • Week 7: Fast Decoding • Week 8: Soft Binned Tree, Analysis • Week 9: Final Project Presentation

  17. References • David M. Chen, Sam S. Tsai, Vijay Chandrasekhar, Gabriel Takacs, Ramakrishna Vedantham, Radek Grzeszczuk, Bernd Girod, “Inverted Index Compression for Scalable Image Matching”

More Related