Object Recognition

1. Object Recognition Tom McGrath CIS 601

2. What is object recognition? • Perception of objects is different for humans than for computers. • For humans: perception of familiar items. • For computers: perception of familiar patterns. • Are they really the same thing?

3. What do we mean by ‘objects’ • What we call object recognition may also be called pattern recognition. • A pattern is an arrangement of descriptors. • Descriptors may have more forms, but they are primarily vectors and strings.

4. More generally… • Object recognition is the process whereby observers are able to recognize three-dimensional objects despite receiving only two-dimensional input that varies greatly depending on viewing conditions.(2)

5. 2 main approaches • Decision-theoretic • Patterns described using quantitative descriptors. • Structural • Patterns represented by symbolic information. • Strings, for example.

6. Decision Theoretic • Based on discriminant functions • Let x = (x1, x2, …, Xn)T represent an n-dimensional pattern vector • Let W = (w1, w2,… ,wW) be pattern classes.

7. Basic problem of decision-theoretic • We want to find W decision functions d1(x), d2(x),…,dw(x) with the property: • If a pattern x belongs to class wi, then di(x) > dj(x), where j = 1, 2, …, W; j != i

8. In other words.. • We want to classify x, which is a pattern. • We are given a finite set of classes of objects. • We want to categorize the pattern x into one of the classes. • To do so, we apply x to all decision functions, and categorize x to the class of best fit.

9. Structural • Represents objects as strings, trees, graphs.. • Define descriptors and recognition rules base on the representations.

10. What does finite classification imply? • The idea of a finite set of classes is quite limiting. • Corresponds with industries’ use of object recognition: very application specific. • Indicates that computer object recognition techniques lack some abilities which are simple for humans.

11. Differences in classification • Techniques thus far only classify objects based on their shape, color, texture, etc. These are only representative of the light reflected by an object. • Humans classify objects many ways, including an object’s function.

12. For example… • We classify a ring of rocks with a fire inside as a fire pit. • We classify a board as a joist once it is installed as support for the floor. • We classify our computer as a paperweight once it is more than five years old.

13. Correlation • Given an image, we want to find all places in the image which contain a subimage, also called a template. • Very useful for answering ‘where is the ‘x’ in this picture?’

14. Notice.. • Recognition models typically rely on input from optical sensors. • Such input is represented entirely in two-dimensional space.

15. Is a 3D representation necessary? • DARPA challenge was not successfully completed. • Army’s LADAR sensors, which provide depth data, have demonstrated more capability.

16. 3D Object recognition with neural trees • First stage extracts features from the input range images. • These features are used in the second stage to group image pixels into different surface patches according to the six surface classes proposed by the differential geometry.(4)

17. Invariants • Basic idea: • D(g(A),g(B)) = D(A,B) • For all g in transformation group G Limitations: There are very many possible transformations in G, and computation times becomes a problem.

18. Varying goals of object recognition • Are we looking for “that” object? • Face recognition • Are we looking for “one of those” objects? • Web search for 1987 Chevy pickup.

19. Notice… • Just because an object exists in an image doesn’t mean it is recognizable. • Example from Late Night with Conan O’Brien

20. We don’t know what this is…

21. Recognizable as a human face…

22. Recognizable as the pope…

23. The Punchline

24. Histogram approach… • Vary bad results for images with: • Much noise • Small target objects With tightly controlled conditions, moderate success can be achieved.

25. Noisy histograms

26. Noisy histograms

27. Noisy histograms

28. Correlation example • Find the flower

29. Create template • Actual template size: 32X32

30. Acquire input image Actual image size: 1600X1200

31. Compute correlation image • Actual image size: 1600X1200

32. Show areas of best match • Actual image size: 1600X1200

33. Find flower with more noise • Source image: 1600X1200

34. Correlation image

35. Area of best match?

36. Templates for a coin • Acquire a template:

37. Acquire target image • Actual size: 1600X1200

38. Compute correlation image

39. Display area of best match

40. Finding coin among noise

41. Correlation image

42. Brightest coin – wrong one

43. Among different noise…

44. Correlation image

45. Coin found

46. Structural approach to stapler • Acquire source stapler image

47. Segment and find the stapler edge

48. Compute the boundary • Image recreated from computed boundary:

49. Select boundary points • Boundary points at distance of 8:

50. Image recreated from boundary points: