Business Identification: Spatial Detection

1. Business Identification:Spatial Detection Alexander Darino Week 7

2. Weaknesses to Current Approach Business Name Matching Business Spatial Detection Latitude Longitude Geocoding Reverse Geocoding Nearby Businesses BusinessIdentification Image OCR Detected Text

3. Alternative: Image Matching

4. Alternative: Image Matching • Weaknesses: • Low Availability of Storefront Images (< 50% Avg) • George Aiken area businesses with photos: 18/35 • Brueggers area businesses with photos: 22/40 • Tambellini area businesses with photos: 8/22 • Available Images too small (100 x 100) • Not a viable solution

5. Alternative: Template Matching • Tambellini • Tambellini • Tambellini • Tambellini • Tambellini • Tambellini • Tambellini • Tambellini

6. Alternative: Template Matching • Progress • Able to generate templates • Able to extract SIFT features using Lowe’s implementation • Able to match SIFT features using Lowe’s implementation • Problems • Features are being matched to garbage

7. Alternative: Template Matching

8. Alternative: Template Matching

9. Alternative: Template Matching

10. Alternative: Template Matching

11. Alternative: Template Matching • Currently on-hold • Need to discuss solution with Amir • Currently looking into another alternative…

12. Alternative: Scene Text Recognition • State of the Art: • STR ≠ OCR • Far superior to our ‘naïve’ approaches to STR (ie. OCR, Image matching, SIFT) • OCR only works for highly controlled environments. CEDAR, ICDAR, etc not helpful • STR works for unconditioned environments • Scale invariant • Color/intensity invariant • Font invariant • Lexicon-Assisted

13. Alternative: Scene Text Recognition • No STR implementations readily available • University of Massachusetts specializes in STR • Papers describe enhancements and unification of previous work, but not algorithms • Will email for blackbox implementation • Currently looking into ‘previous work’ • More models • Some algorithms

14. Alternative: Scene Text Recognition • Options • Email authors for implementation • Try to implement STR as per described models • Blackboxes whenever possible (email!) • Code when blackboxes are not available • Try to implement crude STR via blackboxes Increase Contrast OCR Detected Text Text Detection Orthorectification

15. STR Implementation • STR Implementation: “Automatic Detection and Recognition of Signs From Natural Scenes” Multiresolution-based potential characters detection Character/layout geometry and color properties analysis Refined Detection Local affine rectification

16. Multiresolution-based potential characters detection • Laplacian-of-Guassian Edge Detection • Dice image/edges into Patches • Combine patches with similar properties into regions • Obtain bounding box of region as candidate text • Properties include: • Mean • Variance • Intensity(?)

17. Multiresolution-based potential characters detection

18. Multiresolution-based potential characters detection Patches qualify if:

19. Multiresolution-based potential characters detection

20. Multiresolution-based potential characters detection

21. Multiresolution-based potential characters detection

22. STR Implementation • Possible Solutions: • Don’t grow bounding box. Grow non-rectangular region, then obtain bounding box • Or replace with off-the-shelf Text Detector blackbox (?)

23. Next Steps • Email for STR implementations • Backtrack: Implement ‘crude’ STR • Continue with current STR implementation Increase Contrast/Binarization OCR Detected Text Text Detection Orthorectification

24. Thank You