Computational Vision CSCI 363, Fall 2018 Lecture 8 Spatial Frequency II

1. Computational VisionCSCI 363, Fall 2018Lecture 8Spatial Frequency II

2. Visual Psychophysics Psychophysics is an approach used to study human vision. The idea is as follows: Present observers with a well-defined visual stimulus. Examine their ability to perform tasks with respect to that stimulus (e.g. detect a sinewave grating). From the data, we can learn much about the capabilities and requirements of the human visual system.

3. Contrast Sensitivity How much contrast is required to distinguish a sinewave grating from a uniform field of gray? Luminance Contrast = (Lmax - Lmin)/(Lmax + Lmin) Threshold = amount of contrast needed to detect the grating. Sensitivity = 1/Threshold

4. Contrast Sensitivity Function The contrast sensitivity function (CSF) is determined by measuring the contrast sensitivity for many different frequencies.

5. Frequency Adaptation Adaptation of the visual system occurs after prolonged exposure to a given stimulus. If an observer views a single spatial frequency grating for a long time (e.g. 1 minute), he will become less sensitive to that frequency (adaptation). Other frequencies are unaffected.

6. Multiple Spatial Frequency Channels The human visual system appears to have multiple mechanisms (or channels) tuned to different spatial frequencies.

7. Frequency spectrum of a Gaussian function The frequency spectrum of a Gaussian function is also a Gaussian:

8. * I(x) A(w) G(w)A(w) G*I Gaussian smoothing as a band-pass filter • The Fourier transform of f(x)*g(x) (convolution) is F(a)G(a) (multiplication). • When we smooth a function by convolving with a Gaussian function, in the frequency domain we multiply the frequency spectra. Fourier Transform G(x) G(w)

9. Band Pass Filters • Gaussian smoothing has the effect of cutting off the high frequency components of the frequency spectrum, F(a). • Because only a limited range of frequencies remains, this is called band-pass filtering. • High pass filters eliminate low frequencies and leave high frequencies. • Low pass filters eliminate high frequencies and leave the low frequencies.

10. Frequency channels and Marr's multiple spatial scales • Filtering with the Laplacian of a Gaussian at different spatial scales is equivalent to band-pass filtering at different spatial frequencies. • Wide Gaussian functions smooth a lot, for analysis on large spatial scales. These result in a narrow band-pass filter for low spatial frequencies. • Narrow Gaussian functions smooth a little, for analysis at fine spatial scales. These result in a wide band-pass filter which includes high spatial frequencies.

11. Retinal Ganglion Cells as Spatial Frequency filters • Smoothing followed by differentiation leads to filters of different sizes with a center-surround spatial structure, like the receptive fields of Retinal Ganglion Cells. • Marr suggested that Retinal Ganglion cells with different size receptive fields could provide the initial filtering for the spatial frequency channels found by the psychophysicists. • Marr proposed that these fed into the edge detecting V1 cells, leading to edge detection at different spatial scales.

12. Another viewpoint: V1 cells are spatial frequency filters • DeValois (and others) proposed that V1 cells are tuned to spatial frequency. • V1 simple cells can be modeled as Gabor filters. • A Gabor filter is the product of a sinewave and a gaussian. sin(10x) G(x) sin(10x)G(x)

13. Fourier Transform of a Gabor Filter The Fourier Transform of a Gabor filter is a localized set of spatial frequencies. Gabor filters are band-pass filters. They are tuned to spatial frequency. If Striate Cortex cells are like Gabor filters, then they are also acting as band-pass filters. Fourier Transform

14. 2D Gabor filter

15. Simple Cell Receptive Field vs. Gabor Function Solid line: Simple Cell Receptive Field. Dashed line: Best fitting Gabor function. From: DeValois and DeValois, "Spatial Vision", 1988.

16. Contrast Sensitivity of V1 Cells Distribution of tuning bandwidth CSF for individual V1 cells

17. Spatial Profile vs. CSF The spatial profile of the simple cell receptive field is predicted by taking the inverse Fourier transform of the contrast sensitivity function for that cell.

18. Spatial Frequency Columns As with orientation and ocular dominance, spatial frequency shows columnar organization in the cortex.

19. Simple vs. Complex Cells The response of simple cells to drifting gratings shows a big oscillation over time. The complex cell response does not oscillate much.

20. Classifying simple and complex cells • Simple cells have a larger AC (Alternating current) response. • Complex cells have a larger DC (Direct current) response. • The ratio of AC/DC allows classification. AC/DC >1 => simple cells. AC/DC < 1 => complex cells