2D Fourier Theory for Image Analysis

1. 2D Fourier Theory for Image Analysis Mani Thomas CISC 489/689

2. Roadmap • 2D image basis • Fourier basis • Scale-space representation • Gaussian pyramid • Laplacian pyramid • Image mosaicing • Gabor filters

3. Different basis representation • Recall our discussion of basis vectors for coordinate systems: • Describe point as linear combination of ortho-gonal basis vectors: x=a1v1+ ... +anvn • The standard basis for images is the set of unit vectors corresponding to each pixel. A toy example:

4. Hadamard basis • The standard basis is not the only one we can use to describe an image • E.g., the Hadamard basis (basis images shown here for 2 x 2 images, where black = +1, white = -1) • For the previous example, we can express the image with these new (normalized) basis vectors as: • Coefficients of sum = projection of I onto new basis (dot product) • These are the coordinates of the image in “Hadamard space” • We can also say that I has undergone a Hadamard transform H:

5. Sinusoidal Basis • Binary-valued, rectangular wave pattern of Hadamard basis doesn’t capture real image gradients well • Idea: Use smoothly-varying sinusoidal patterns at different frequencies, angles for basis images

6. Fourier Basis • The Fourier basis uses the family of complex sinusoidal functions

7. 2D DFT • Forward 2D DFT • Inverse 2D DFT • (u, v) are the frequency coordinates while (x, y) are the spatial coordinates • M, N are the number of spatial pixels along the x, y coordinates

8. v Fourier Basis Real (cos) part (u, v) (1, 0) (1, 1) (0, 5) Imaginary (sin) part

9. Fourier transform in Matlab • Discrete, 2-D Fourier & inverse Fourier transforms are computed by fft2 and ifft2, respectively • fftshift: Move origin (DC component) to image center for display • Example: >> I = imread(‘test.png’); % Load grayscale image >> F = fftshift(fft2(I)); % Shifted transform >> imshow(log(abs(F)),[]); % Show log magnitude >> imshow(angle(F),[]); % Show phase angle

10. Phase and Magnitude • Output of the Fourier transform is a complex number • Decompose the complex number as the magnitude and phase components • In Matlab: u = real(z), v = imag(z), r = abs(z), and theta = angle(z)

11. Image pyramid representation • Smoothing means removing high frequencies • Smoothing required to avoid aliasing • Fourier transform of a Gaussian is a Gaussian • Convolution is a multiplication Gaussian suppresses high frequencies

12. Gaussian Pyramid • Downsampling: Cut width, height in half at each iteration: • Upsampling S"(I): Double size of image, interpolate missing pixels • Let the base (the finest resolution) of an n-level Gaussian pyramid be defined as P0=I. Then the ith level is reduced from the level below it by: from Forsyth & Ponce Gaussian pyramid

13. Laplacian pyramid • The tip (the coarsest resolution) of an n-level Laplacian pyramid is the same as the Gaussian pyramid at that level: Ln(I) =Pn(I) • The ith level is expanded from the level above according to Li(I) =Pi(I) ¡S"(Pi+1(I)) • Synthesizing the original image: Get I back by summing upsampled Laplacian pyramid levels

14. Gaussian and Laplacian courtesy of Wolfram • Gaussian – Smoothing pyramid • Each level is a smoothed and decimated signal of the previous • Laplacian – Band pass filter of the images • Each level is the difference of a more smoothed and less smoothed image

15. Summation Property • If L0, L1 LN is the sequence of laplacians Li = Gi – EXPAND[Gi+1], 0<i<N LN = GN • The steps used to construct the Laplacian can be reversed to get the original • Expand Li and add it to Li-1 to Gi-1 G0 = i=0N Li

16. Applications – Image Mosaicing Seamless joining of images to get a larger view

17. Multi-resolution spline interpolation

18. Laplacian level 4 Laplacian level 2 Laplacian level 0 left pyramid right pyramid blended pyramid

20. Image mosaicing • Automatic mosaicing • Cross correlation to compute translation between images • Matlab demo – Burt and Adelson’s paper • http://www.cs.huji.ac.il/course/2003/impr/spline83.pdf

21. Jw refine warp + u=1.25 pixels u=2.5 pixels u=5 pixels u=10 pixels image J image J image I image I Pyramid of image J Pyramid of image I Application - Coarse-to-Fine Estimation Slide from CS 223-B L9 by Richard Szeliski

22. Odd (sin) Even (cos) Gabor filters • Gaussian windowed Fourier Transform • Make convolution kernels from product of Fourier basis images and Gaussians £ = Frequency

23. Texture Representation: Filter Responses • Choose a group of filters • Edge/Bar filters: Something like Gabor filters at different orientations, scales • Spot filters: Center-surround filters like a Gaussian/difference of Gaussians at multiple scales • Run filters over image to get a set of response images • Each contains specific texture information

24. Example: Filter Responses Input image Filter bank from Forsyth & Ponce

25. Texture Similarity based on Response Statistics • Collect statistics of responses over an image or subimage • Mean of squared response • Mean and variance of squared response • Euclidean distance between vectors of response statistics for two images is measure of texture similarity

26. Conclusions • 2D Fourier Theory • Image pyramid representation • Gaussian pyramid • Laplacian pyramid • Applications of Image Pyramids • Image Mosaicing • Gaussian + Laplacian pyramids (Burt and Adelson) • Texture statistics • Gabor filters