Exploring Visual Perception Modeling in Computer Vision

1. Outline • Theoretical approaches to computer vision • Visual perception as information processing • Problems in Computer Vision • Classification • Segmentation • Recognition • Motion analysis

2. Visual Perception as an Inverse Problem • Retinal images are generated by the light reflected from the 3-D world • The image formation is determined by the laws of optics • The area of image rendering is called computer graphics • Vision as an inverse problem • Get from optical images of scenes back to knowledge of the objects that gave rise to them Visual Perception Modeling

3. Vision as a Heuristic Process • Visual system makes a lot of assumptions about the nature of the environment and conditions under which it is viewed • These assumptions constrain the inverse problem enough to make it solvable most of the time • The resulting solution will be veridical if the assumptions are true • Vision is a heuristic process in which inferences are made about the most likely environmental condition that could have produced a given image Visual Perception Modeling

4. Perception as Bayesian Inference • Images I are observations • Scene properties S are not known • p(S) specifies the prior knowledge about the scene • The knowledge you have without looking at the image • Bayes’ rule Visual Perception Modeling

5. Four Stages of Visual Processing • Image-based stage • Surface-based stage • Object-based stage • Category-based stage Visual Perception Modeling

6. Image-based Stages • Most theorists agree that initial stage is not the only representation based on a two-dimensional retinal organization • It includes image-processing operations • Local edge and line detection • Region detection • Correspondence between left and right eyes • Marr called this representation primal sketches • Raw primal sketch • Full primal sketch Visual Perception Modeling

7. Representation in Early Vision • Local spatial/frequency representation • The representation should be • Local • Orientation-tuned • Frequency-tuned • Gabor filters • Wavelet transformation • Image compression Visual Perception Modeling

8. Gabor Filters Visual Perception Modeling

9. Surface-based Stage • Recovery of intrinsic properties of visible surfaces • Surface layout • The spatial distribution of visible surfaces within the 3-D environment • Explicit surface-based representation • 2.5-D sketch • Intrinsic images • Intrinsic properties to surfaces Visual Perception Modeling

10. Surface-Based Stage – cont. • Surface primitives • Local patches of 2-D surface within a 3-D space • Three-dimensional geometry • Projective geometry • Viewer-centered reference frame Visual Perception Modeling

11. Surface-Based Stage – cont. • Cues for surface representation • Stereopsis • Motion parallax • Shading and shadows • Pictorial properties • Texture • Size • Shape • Occlusion Visual Perception Modeling

12. Object-Based Stage • Some form of true 3-D representation • Includes unseen and occluded surfaces • Explicit representations of whole objects • Two ways of constructing object representation • Extend the surface-based representation • Infer 3-D objects from 2-D images Visual Perception Modeling

13. Object-Based Stage – cont. • Volumetric primitives • Descriptions of truly 3-D volumes • Three-dimensional geometry • Geometry in 3-D space • Object-based reference frame • Spatial relations among the volumetric primitives are represented by intrinsic structures among volumetric structures Visual Perception Modeling

14. Category-Based Stage • Final stage concerns with recovering fully the functional properties of objects • Functional properties through categorization • Properties directly from visible characteristics Visual Perception Modeling

15. Top-down vs. Bottom-up Processes • Bottom-up processing • Data driven processing • Take a lower-level representation as input and create or modify a higher-level representation • Top-down processing • Expectation-driven processing • Processes that take a higher-level representation as input and produce or modify a lower-level representation Visual Perception Modeling

16. Neural Network Approaches • Neural networks are based on the assumptions that human vision depends heavily on the massively parallel structure of neural circuits in the brain • Multiple Layer Perceptrons • Input layer • Hidden layer • Output layer Visual Perception Modeling

17. Problems in Computer Vision • Given a matrix of numbers representing an image, or a sequence of images, how to generate a perceptually meaningful description of the matrix? • An image can be a color image, gray level image, or other format such as remote sensing images • A two-dimensional matrix represents a single image • A three-dimensional matrix represents a sequence of images • A video sequence is a 3-D matrix • A movie is also a 3-D matrix Visual Perception Modeling

18. Image Classification • Given some types through examples, identify the type of a new image Visual Perception Modeling

19. A texture image Image Segmentation • Partition the images into homogenous regions • Widely studied problem • A very difficult problem • An important problem Visual Perception Modeling

20. A cheetah image Object Recognition • Object recognition • Recognize objects in a constrained environment • Identify objects from images Visual Perception Modeling

21. Video Sequence Analysis • Motion analysis • Compute motion from images • Motion segmentation • Video sequence analysis • Derive models automatically • Enhanced TV Visual Perception Modeling