1 / 79

Biologically Inspired Intelligent Systems

Biologically Inspired Intelligent Systems. Lecture 6 R.S. Gaborski. Research Project Topics. Navigation and Spatial Memory Starting point: http://memory.psych.upenn.edu/research/research_spatial_memory.php Motion Detection in MT and MST. Reading. Read Chapters Two and Three.

kenton
Download Presentation

Biologically Inspired Intelligent Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Biologically Inspired Intelligent Systems Lecture 6 R.S. Gaborski

  2. Research Project Topics • Navigation and Spatial Memory • Starting point: http://memory.psych.upenn.edu/research/research_spatial_memory.php • Motion Detection in MT and MST Roger S. Gaborski

  3. Reading • Read Chapters Two and Three Roger S. Gaborski

  4. Four Lobes of Neocortex- Lateral view Dorsal Caudal Rostral Ventral Roger S. Gaborski

  5. Biologically Inspired Vision System • Extract low level features using biologically inspired feature detectors (receptive fields) • Implement a Focus of Attention (FOA) mechanism based on low level features • Modify the low level FOA with high level information • Extract complex features in regions of FOA • Perform classification on objects in FOA area Roger S. Gaborski

  6. Saliency • Definition: • A highlight or striking feature • Focus of Attention is salient Roger S. Gaborski

  7. Bottom Up Saliency • Localization of stimulus – only the ‘where’, not the ‘what’ • Assume: • Early visual structures are topographical feature maps • Constructed with center surround RFs at different spatial scales • The features maps are combined into single saliency map Roger S. Gaborski

  8. Saliency Map • Saliency areas are ranked • After we attend to the most salient region we switch our attention to the next most salient region • The saliency map doesn’t identify the actual object, just the region of interest Roger S. Gaborski

  9. What do you ‘notice’ in the image? Roger S. Gaborski

  10. What do you ‘notice’ in the image? Roger S. Gaborski

  11. Seven Individual Maps Combined to Form Saliency Map • Feature maps at different scales are combined resulting in one map for each feature: • Intensity contrast • Orientation • 0, 45, 90 and 135 degrees • Red-Green opponent color • Blue-Yellow opponent color Roger S. Gaborski

  12. Intensity Contrast • Recall… Roger S. Gaborski

  13. Model -Image made up of pixels -Each pixel can represent the output of a photoreceptor -Digital color images are made up of three planes of data (matrices) -The green rectangle represents a receptive field Roger S. Gaborski

  14. MATLAB Image = imread(‘boat.jpg’); imshow(Image)  displays Image Image is a matrix composed of 3 planes of data; red, green and blue size(Image)  900 1200 3 Image(12,56,1)  red pixel value at position (12,56) Roger S. Gaborski

  15. Low Level Feature Extraction -1 • Form Contrast Image by convolving input image with a set of Difference of Gaussian filters which model center on and center off circular receptive fields in retina • Sizes 8x8, 16x16 and 32x32pixels • Small object will respond strongly to 8x8 DoG, large objects will respond strongly to 32x32 DoG • Uniform areas will not respond Roger S. Gaborski

  16. Difference of Gaussians – 8x8 %Dog8Filter.m %Generates difference of Gaussians 8x8 imageSize = 8; sigmaex = .04*imageSize; %Sigma in Excitatory Gaussian sigmainh = .16*imageSize; [x,y] = meshgrid(-3:4 -3:4); %First Term exp1 = exp( -1*( x .* x + y .* y)./(2*sigmaex*sigmaex)); FirstTerm = 6.0*exp1; %6.0 %9.95 %SecondTerm exp2 = exp( -1*( x .* x + y .* y)./(2*sigmainh*sigmainh)); SecondTerm = 1.0*exp2; Dog8 = .41*(FirstTerm - SecondTerm); figure, subplot(2,2,1), plot(1:8,FirstTerm(4,:)), grid subplot(2,2,2), plot(1:8,SecondTerm(4,:)), grid subplot(2,2,3), plot(1:8, Dog8(4,:)), grid Roger S. Gaborski

  17. Difference of Gaussian Output8x8 Pixels Gaussian 2 Gaussian 1 Note scales Difference of Gaussian 1 and 2 Roger S. Gaborski

  18. DoG8 Roger S. Gaborski

  19. Difference of Gaussian Output64x64 Pixels Roger S. Gaborski

  20. DoG64 Roger S. Gaborski

  21. Convolution • Simple example >> im = imread('fern1.jpg'); >> im = rgb2gray(im); >> im = imresize(im, .5,'bicubic'); >> figure, imshow(im) >> im8 = conv2(double(im),double(Dog8)); >> im16 = conv2(double(im),double(Dog16)); >> im32 = conv2(double(im),double(Dog32)); >> im64 = conv2(double(im),double(Dog64)); Roger S. Gaborski

  22. fern1.jpg Roger S. Gaborski

  23. Roger S. Gaborski

  24. Roger S. Gaborski

  25. Roger S. Gaborski

  26. Roger S. Gaborski

  27. Classical Vision Model Contrast Images imConv8 imConv16 imConv32 Retina Model 8x8, 16x16 and 32x32 circular receptive fields Image Roger S. Gaborski

  28. Input Image Roger S. Gaborski

  29. Convolved with DoG8 Roger S. Gaborski

  30. Convolved with DoG16 Roger S. Gaborski

  31. Convolved with DoG32 Roger S. Gaborski

  32. Intensity Contrast Map • Create intensity image by averaging red, green and blue image planes • Form Contrast Image by convolving gray level input image with a set of Difference of Gaussian filters which model center on and center off circular receptive fields in retina • Sizes 8x8, 16x16 and 32x32pixels • Small object will respond strongly to 8x8 DoG, large objects will respond strongly to 32x32 DoG • Combine resulting intensity contrast maps Roger S. Gaborski

  33. Feature Maps • Intensity contrast • Orientation • 0, 45, 90 and 135 degrees • Red-Green opponent color • Blue-Yellow opponent color Roger S. Gaborski

  34. Directional Receptive Fieldsare Modeled with Gabor FunctionsCosine Grating * 2D Gaussian Roger S. Gaborski

  35. Cosine Grating and Slice Cosine Grating Slice Indicated by Red Bar Roger S. Gaborski

  36. Grating function Gabor_cos = MakeGrating2( orient, numOfSamples, numOfCycles) %parameters: 90,10,2 sd = 12; orient = 2*pi - (orient*pi/180); %create grading step = 1/numOfSamples; [x,y] = meshgrid( -pi:step:pi, -pi:step:pi); ramp = (cos (orient) * x) - (sin(orient)*y); figure, imagesc(ramp), colormap(gray) im = sin(ramp*numOfCycles); im_cos = cos(ramp*numOfCycles); figure, imagesc(im_cos), colormap(gray), title('Grating') Roger S. Gaborski

  37. Gaussian and Gabor %Generate Gabor filtSize = min(size(im)); x = linspace(-1,1,filtSize)*filtSize/2; y = (1/sqrt(2*pi*sd)).*exp(-.5*((x/sd).^2)); filt = (y'*y);filt=filt./max(filt(:)); %Gaussian figure, imagesc(filt), title('filt') colormap(hot) Gabor = im .* filt; figure, subplot(2,1,1), imagesc(Gabor), axis square, colorbar title('Gabor\_sin') Gabor_cos = im_cos .* filt; figure, imagesc( Gabor_cos), axis square title('Gabor\_cos') Roger S. Gaborski

  38. Gaussian – top view Roger S. Gaborski

  39. Gabor Models of Directional Receptive Fields 45 degrees 0 degrees 135 degrees 90 degrees Roger S. Gaborski

  40. Receptive Fields measure in the cat Roger S. Gaborski

  41. Low Level Feature Extraction -2 • Convolve Contrast Image images with Gabor modeled directional receptive fields • 0, 45, 90 and 135 degrees • Size: 7x7, 15x15 and 31x31 • Rectify resulting image (absolute value) • Know as S1 Cells Roger S. Gaborski

  42. DoG8 Contrast Image (On center) 7x7 pixel 0 degree RF Roger S. Gaborski

  43. DoG8 Contrast Image (On center) 15x15 pixel 0 degree RF Roger S. Gaborski

  44. DoG8 Contrast Image (On center)31x31 pixel 0 degree RF Roger S. Gaborski

  45. DoG16 Contrast Image (On center) 7x7 pixel 0 degree RF Roger S. Gaborski

  46. DoG16 Contrast Image (On center) 15x15 pixel 0 degree RF Roger S. Gaborski

  47. DoG16 Contrast Image (On center) 31x31 pixel 0 degree RF Roger S. Gaborski

  48. DoG16 Contrast Image (On center) 31x31 pixel 45 degree RF Roger S. Gaborski

  49. Current State of ModelOrientation Map • For each contrast map convolve with 7x7, 15x15 and 31x31 Gabor orientation filters at 0, 45, 90 and 135 degrees • Combine the resulting orientation maps Roger S. Gaborski

  50. Directional Maps WHERE n = 0, 45, 90 and 135 degrees n degrees 7x7 Gabor n degrees 15x15 Gabor n degrees 31x31 Gabor n degrees 7x7 Gabor n degrees 15x15 Gabor n degrees 31x31 Gabor n degrees 7x7 Gabor n degrees 15x15 Gabor n degrees 15x15 Gabor Contrast Images imConv8 imConv16 imConv32 (Retina Model) Retina Model 8x8, 16x16 and 32x32 circular receptive fields Gray Level Image Roger S. Gaborski

More Related