Object Recognizing

1. Object Recognizing

2. Object Classes

3. Individual Recognition

4. Object partsFull Interpretation Window Mirror Window Door knob Headlight Back wheel Bumper Front wheel Headlight

5. Action recognition (except 2)

6. Class Non-class

7. Class Non-class

8. Is this an airplane?

9. Features and Classifiers Same features with different classifiers Same classifier with different features

10. Generic Features Simple (wavelets) Complex (Geons)

11. Marr-Nishihara

12. Mental Rotation

13. 3-D Parts • Implementations – poor results • View-specific recognition • fMRI studies • Instead: Using image patches

14. Class-specific Features: Common Building Blocks

15. Optimal Class Components? • Large features are too rare • Small features are found everywhere Find features that carry the highest amount of information

16. Mutual information H(C) F=0 F=1 H(C) when F=1 H(C) when F=0 I(C;F) = H(C) – H(C/F)

17. Mutual Information I(C,F) Class: 1 1 0 1 0 1 0 0 Feature: 1 0 0 1 1 1 0 0 I(F,C) = H(C) – H(C|F)

18. Horse-class features Car-class features Pictorial features Learned from examples

19. Star model Detected fragments ‘vote’ for the center location Find location with maximal vote In variations, a popular state-of-the art scheme

20. Recognition Features in the Brain

21. fMRI Functional Magnetic Resonance Imaging

22. תמונות של פעילות המח

23. LO object recognition V1 early processing

24. Class-fragments and Activation Malach et al 2008

25. Bag of words

26. – Bag of visual words A large collection of image patches

27. – – – Each class has its words historgram Limited or no Geometry Simple and popular Visual words are used, but not for full recognition model

28. HoG Descriptor Dallal, N & Triggs, B. Histograms of Oriented Gradients for Human Detection

29. SIFT: Scale-invariant Feature Transform • MSER: Maximally Stable Extremal Regions • SURF: Speeded-up Robust Features • Cross correlation • …. • HoG and SIFT are the most widely used.

30. DPM Felzenszwalb • Felzenszwalb, McAllester, Ramanan CVPR 2008. A Discriminatively Trained, Multiscale, Deformable Part Model • Many implementation details, will describe the main points.

31. HoG descriptor

32. Using patches with HoG descriptors and classification by SVM Person model: HoG

33. Object model using HoG A bicycle and its ‘root filter’ The root filter is a patch of HoG descriptor Image is partitioned into 8x8 pixel cells In each block we compute a histogram of gradient orientations

34. Dealing with scale: multi-scale analysis The filter is searched on a pyramid of HoG descriptors, to deal with unknown scale

35. Adding Parts A part Pi = (Fi, vi, si, ai, bi). Fi is filter for the i-th part, vi is the center for a box of possible positions for part i relative to the root position, si the size of this box ai and bi are two-dimensional vectors specifying coefficients of a quadratic function measuring a score for each possible placement of the i-th part. That is, ai and bi are two numbers each, and the penalty for deviation ∆x, ∆y from the expected location is a1 ∆x + a2 ∆y+ b1 ∆x2 + b2 ∆y2

36. Bicycle model: root, parts, spatial map Person model