A Sample Recognition Problem

1. A Sample Recognition Problem Joseph Tighe University of North Carolina at Chapel Hill

2. What is recognition?

3. What is recognition?

4. What is recognition?

5. What is recognition? Figure from Shotton et al. (2009) He et al. (2004), Hoiem et al. (2005), Shotton et al. (2006, 2008, 2009), Verbeek and Triggs (2007), Rabinovich et al. (2007), Galleguillos et al. (2008), Gould et al. (2009), etc.

6. What is recognition? • Have some entity • photo • bounding box in photo • pixel • Assign semantic meaning • scene type, tags, sentence • object label, action • semantic class, material, geometric orientation

7. Our Problem • Entity: Images • Goal: Assign 1 of 4 labels (airplane, car, face, motorbike

8. Finding Similar Images

9. Ocean Forest Open Field Mountain Which image is most similar? Highway Inner City Then assign the label from the most similar image Street Tall Building What is depicted in this image?

10. Pixels are a bad measure of similarity Most similar according to pixel distance Most similar according to “Bag of Words”

11. Origin of the Bag of Words model • Orderless document representation: • frequencies of words from a dictionary Salton & McGill (1983) US Presidential Speeches Tag Cloudhttp://chir.ag/phernalia/preztags/

12. What are words for an image?

17. Wing Tail Propeller Building Wheel

18. Wing Building Wheel Propeller Jet Engine Tail

19. Wing Building Wheel Propeller Jet Engine Tail

20. Wing Building Wheel Propeller Jet Engine Tail

21. But where do the words come from?

23. Then where does the dictionary come from?

24. Example Dictionary Source: B. Leibe

25. … … … … Another dictionary Source: B. Leibe

26. Fei-Fei et al. 2005

27. Outline of the Bag of Words method • Divide the image into patches • Assign a “word” for each patch • Count the number of occurrences of each “word” in the image

28. Matlab Demo