Download
1 / 51
510 likes | 535 Views
Explore vector space theory applied to image processing and representation problems. Discuss the concept of images as sums of basic images and sparse representations using a smart card. Learn about different bases, orthonormal basis vectors, Fourier transform, bandpass filtering, Log or Dog filter, and more. This study delves into the mathematical models behind image manipulation and analysis techniques.
E N D
Vectors [and more on masks] Vector space theory applies directly to several image processing/representation problems MSU CSE 803 Stockman Fall 2008
Image as a sum of “basic images” What if every person’s portrait photo could be expressed as a sum of 20 special images? We would only need 20 numbers to model any photo sparse rep on our Smart card. MSU CSE 803 Stockman Fall 2008
The image as an expansion MSU CSE 803 Stockman Fall 2008
Different bases, different properties revealed MSU CSE 803 Stockman Fall 2008
Fundamental expansion MSU CSE 803 Stockman Fall 2008
Basis gives structural parts MSU CSE 803 Stockman Fall 2008
Vector space defs., part 1 MSU CSE 803 Stockman Fall 2008
Vector space defs. Part 2 2 MSU CSE 803 Stockman Fall 2008
A space of images in a vector space • M x N image of real intensity values has dimension D = M x N • Can concatenate all M rows to interpret an image as a D dimensional 1D vector • The vector space properties apply • The 2D structure of the image is NOT lost MSU CSE 803 Stockman Fall 2008
Orthonormal basis vectors help MSU CSE 803 Stockman Fall 2008
Represent S = [10, 15, 20] MSU CSE 803 Stockman Fall 2008
Projection of vector U onto V MSU CSE 803 Stockman Fall 2008
Normalized dot product Can now think about the angle between two signals, two faces, two text documents, … MSU CSE 803 Stockman Fall 2008
Every 2x2 neighborhood has some constant, some edge, and some line component Confirm that basis vectors are orthonormal MSU CSE 803 Stockman Fall 2008
Roberts basis cont. If a neighborhood N has large dot product with a basis vector (image), then N is similar to that basis image. MSU CSE 803 Stockman Fall 2008
Standard 3x3 image basis Structureless and relatively useless! MSU CSE 803 Stockman Fall 2008
Frie-Chen basis Confirm that bases vectors are orthonormal MSU CSE 803 Stockman Fall 2008
Structure from Frie-Chen expansion Expand N using Frie-Chen basis MSU CSE 803 Stockman Fall 2008
Sinusoids provide a good basis MSU CSE 803 Stockman Fall 2008
Sinusoids also model well in images MSU CSE 803 Stockman Fall 2008
Operations using the Fourier basis MSU CSE 803 Stockman Fall 2008
A few properties of 1D sinusoids They are orthogonal Are they orthonormal? MSU CSE 803 Stockman Fall 2008
F(x,y) as a sum of sinusoids MSU CSE 803 Stockman Fall 2008
Spatial direction and frequency in 2D MSU CSE 803 Stockman Fall 2008
Continuous 2D Fourier Transform To compute F(u,v) we do a dot product of our image f(x,y) with a specific sinusoid with frequencies u and v MSU CSE 803 Stockman Fall 2008
Power spectrum from FT MSU CSE 803 Stockman Fall 2008
Examples from images Done with HIPS in 1997 MSU CSE 803 Stockman Fall 2008
Descriptions of former spectra MSU CSE 803 Stockman Fall 2008
Discrete Fourier Transform Do N x N dot products and determine where the energy is. High energy in parameters u and v means original image has similarity to those sinusoids. MSU CSE 803 Stockman Fall 2008
Bandpass filtering MSU CSE 803 Stockman Fall 2008
Convolution of two functions in the spatial domain is equivalent to pointwise multiplication in the frequency domain MSU CSE 803 Stockman Fall 2008
LOG or DOG filter Laplacian of Gaussian Approx Difference of Gaussians MSU CSE 803 Stockman Fall 2008
LOG filter properties MSU CSE 803 Stockman Fall 2008
Mathematical model MSU CSE 803 Stockman Fall 2008
1D model; rotate to create 2D model MSU CSE 803 Stockman Fall 2008
1D Gaussian and 1st derivative MSU CSE 803 Stockman Fall 2008
2nd derivative; then all 3 curves MSU CSE 803 Stockman Fall 2008
Laplacian of Gaussian as 3x3 MSU CSE 803 Stockman Fall 2008
G(x,y): Mexican hat filter MSU CSE 803 Stockman Fall 2008
Convolving LOG with region boundary creates a zero-crossing Mask h(x,y) Input f(x,y) Output f(x,y) * h(x,y) MSU CSE 803 Stockman Fall 2008
LOG relates to animal vision MSU CSE 803 Stockman Fall 2008
1D EX. Artificial Neural Network (ANN) for computing g(x) = f(x) * h(x) level 1 cells feed 3 level 2 cells level 2 cells integrate 3 level 1 input cells using weights [-1,2,-1] MSU CSE 803 Stockman Fall 2008
Experience the Mach band effect MSU CSE 803 Stockman Fall 2008
Simple model of a neuron MSU CSE 803 Stockman Fall 2008
Output conditioning: threshold versus smoother output signal MSU CSE 803 Stockman Fall 2008
3D situation in the eye Neuron c has + input to neuron A but - input to neuron B. Neuron d has + input to neuron B but – input to neuron A. Neuron b gives no input to neuron B: it is not in the receptive field of B. MSU CSE 803 Stockman Fall 2008
Receptive fields MSU CSE 803 Stockman Fall 2008
Experiments with cats/monkeys • Stabilize/drug animal to stare • Place delicate probe in visual network • Move step edge across FOV • Probe shows response function when the edge images to receptive field • Slightly moving the probe produces similar signal when edge is nearby MSU CSE 803 Stockman Fall 2008
Canny edge detector uses LOG filter MSU CSE 803 Stockman Fall 2008
