1 / 31

AutoLabel: Labeling Places from Pictures and Websites

AutoLabel enables the automatic labeling of places by matching in-store text with website content, utilizing Wi-Fi AP vectors for accurate localization.

geraldedwards
Download Presentation

AutoLabel: Labeling Places from Pictures and Websites

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. AutoLabel: Labeling Places from Pictures and Websites Rufeng Meng, Sheng Shen, Romit Roy Choudhury, Srihari Nelakuditi

  2. Localization Coordinates  Names

  3. Wi-Fi Based Semantic Localization

  4. Wi-Fi Based Semantic Localization

  5. Core Problem ?

  6. AP Name Mining AP NameMining Best Buyon Street Macy’sin Mall

  7. Opportunity: Stores have two avatars

  8. Opportunity: Stores have two avatars

  9. Mapping AP Vector to Store Name OCR AutoLabelMatching In-store Text with Website Text Matching In-store Textwith Website Text Text From Candidate Webpages Best Buy Whole Foods Panera Starbucks

  10. Candidate Store Names CrowdSourcing m n

  11. CrowdSourcing m n Candidate Store Names

  12. CrowdSourcing OCRed In-store Text 1 Text-based Matching 1 Candidate Store Names

  13. Text Extraction Above Eye-level • Store Text • Text OCRed from In-store Pictures • Web Text • Candidate Store Names • Meta Keywords in Web Pages • Text in Menu/Category Items Eye-level Below Eye-level

  14. Noun/Proper Name Extraction Filter Weight Assignment Bag of Words Similarity Calculation Labeling Noun/Proper Name Extraction Filter Weight Assignment Bag of Words • Weight assignment: TF-IDF Text Matching Store Text Web Text

  15. <AP6, AP7, AP8> <AP1, AP2, AP5> <AP2, AP3, AP4> Cluster (Best Buy) (Dollar Tree) (Starbucks) (Panera Bread) Webs of Candidate Stores AutoLabel In-store Textvs. Website Text

  16. Simultaneous Clustering and Labeling • Two images put in different clusters if no common AP • Generate potential clusterings of images • Similarity of clustering = sum of similarities of its clusters • Pick the clustering with highest similarity • Label the AP vectors as per the best clustering

  17. Evaluation

  18. Data Collection • 40 Stores • 18 from shopping mall @Champaign, IL • 10 from one street area @Champaign, IL • 6 from other places @Champaign, IL • 6 from shopping mall @San Jose, CA • Number of Pics: • 6 ~ 217 pics/store(with readable text)

  19. Text Matching without Store Names Accuracy: 80%

  20. Text Matching with Store Names Accuracy: 87%

  21. Text Matching without Store Names (Mall) 16/18 Correctly Matched

  22. Text Matching with Store Names (Mall) 17/18 Correctly Matched

  23. Text Matching without Store Names (Street) 10/10 Correctly Matched

  24. Accuracy with Varying #Pics/Store

  25. Accuracy with Varying #Pics/Store

  26. Accuracy with Varying #Stores/Area Mean Accuracy: • 5-store: 100% • 20-store: 90%

  27. Performance of Place Recognition with AP Vector– Store Name Mapping

  28. Simultaneous Clustering and Labeling

  29. Related Work • Visual business recognition • Image based matching of the exterior of store • Human assisted positioning using textual signs • Requires user intervention to localize self • Correlate crowdsourced pictures with social network posts • Stronger correlation between store and its website

  30. Limitations & Future Work • WiFi AP tagged in-store pictures • Reduce uncertainty even with a few pictures • Common decorations besides text • Image clustering • Spatial relationship between stores • Color/Theme + AP name mining • Beyond semantic localization • Correlate website and in-store shopping behavior

  31. Summary of AutoLabel

More Related