Computational Vision

1. Computational Vision Jitendra Malik University of California, Berkeley

2. What is in an image? The input is just an array of brightness values; humans perceive structure in it.

3. Water back Grass Tiger Tiger Sand head eye legs tail mouse shadow From Pixels to Perception outdoor wildlife

4. Objects Scenes Pixels Local Neighborhoods Contours Surfaces Water Grass Tiger Sand Mid-level High-level Low-level Recognition Image Processing Grouping Figure/Ground Surface Attributes If visual processing was purely feedforward…(it isn’t)

5. Boundaries of image regions defined by a number of attributes • Brightness/color • Texture • Motion • Binocular disparity • Familiar configuration

8. A B C Grouping is hierarchical Perceptual organization forms a tree: Image BG L-bird R-bird bush far grass body beak body beak eye head eye head Two segmentations are consistent when they can be explained by the same segmentation tree • A,C are refinements of B • A,C are mutual refinements • A,B,C represent the same percept

9. Humans assign a depth ordering to surfaces across a contour • R1 appears in front of R2 • R2 appears in front of R3 This can be done for images of natural scenes …

10. Figure-Ground Labeling • - red is near; blue is far

11. Ground (shapeless) Figure (face) Figure (Goblet) Ground (Shapeless) Figure/Ground Organization • A contour belongs to one of the two (but not both) abutting regions. Important for the perception of shape

12. Good continuation Amodal completion Modal completion Some other aspects of perceptual organization

13. What do we see here?

14. And here?

15. Some Pictorial Cues

16. Support, Size 2 ? 3 ? 1 ?

17. Cast Shadows

18. Shading

19. Measuring Surface Orientation

20. Binocular Stereopsis

21. Optical flow for a pilot

22. Object Category Recognition

23. Shape variation within a category • D’Arcy Thompson: On Growth and Form, 1917 • studied transformations between shapes of organisms

24. Attneave’s Cat (1954)Line drawings convey most of the information

25. Objects are in Scenes

26. Human stick figure from single image Input image Stick figure Support masks

27. This is hard… • Variety of poses • Clothing • Missing parts • Small support for parts • Background clutter

28. Taxonomy and Partonomy • Taxonomy: E.g. Cats are in the order Felidae which in turn is in the class Mammalia • Recognition can be at multiple levels of categorization, or be identification at the level of specific individuals , as in faces. • Partonomy: Objects have parts, they have subparts and so on. The human body contains the head, which in turn contains the eyes. • These notions apply equally well to scenes and to activities. • Psychologists have argued that there is a “basic-level” at which categorization is fastest (Eleanor Rosch et al). • In a partonomy each level contributes useful information for recognition.

29. Visual Control of Action • Locomotion • Navigation/Way-finding • Obstacle Avoidance • Manipulation • Grasping • Pick and Place • Tool use

31. Camera Obscura(Reinerus Gemma-Frisius, 1544)

32. Camera Obscura(Angelo Sala, 1576-1637)