1 / 52

Introduction of Computer Vision

Introduction of Computer Vision. Pedestrian Detection. Finding People in Images and Videos Navneet DALAL http://lear.inrialpes.fr/people/dalal/NavneetDalalThesis.pdf Chapter 4: Histogram of Oriented Gradients Based Encoding of Images http://lear.inrialpes.fr/pubs/2005/DT05/cvpr2005_talk.pdf

jariah
Download Presentation

Introduction of Computer Vision

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Introduction of Computer Vision

  2. Pedestrian Detection • Finding People in Images and Videos • Navneet DALAL • http://lear.inrialpes.fr/people/dalal/NavneetDalalThesis.pdf • Chapter 4: Histogram of Oriented Gradients Based Encoding of Images • http://lear.inrialpes.fr/pubs/2005/DT05/cvpr2005_talk.pdf • Histograms of Oriented Gradients for Human Detection http://lear.inrialpes.fr/pubs/2005/DT05/hog_cvpr2005.pdf

  3. Datasets • INRIA Person Dataset • http://pascal.inrialpes.fr/data/human/ • Computer Vision Datasets • http://clickdamage.com/sourcecode/cv_datasets.html

  4. Today’s Topic • Motion Detection

  5. Motion Detection • Temporal Differencing • Take the difference between two temporally adjacent frames. The difference is the moving pixels (almost). The static background results in zeros. • Can adapt to changing lighting conditions because the difference frames are only 1/30 of a second apart (typical video 30 frames per second – 30 fps)

  6. Motion Detection • Temporal Differencing Issues • Not all the relevant pixels extracted • Background pixels extracted.

  7. Motion Detection Frame at time t Frame at time t+1 Frame Difference Red Block appears as two separate objects

  8. Sidewalk Scene

  9. Motion DetectionDifference Method Sidewalk 12_5

  10. Processing Video in Matlab %C:\Program Files\MATLAB\R2008b\toolbox\OCR\BackgroundAnalysis02_02_2010 bkg = 20; %frames of video to be processed fname = 'Office1.avi'; vidObj = mmreader(fname); %Play video implay('Office1.avi'); nFrames = vidObj.NumberOfFrames; rw = vidObj.Height; cl = vidObj.Width; numFrames = 1000; CODE TO PROCESS FRAMES HERE

  11. Writing Frames to Directory imwrite(objBox,['.\video\','LabScene','.',num2str(i),'.jpg'],'jpg'); imwrite(objBoxVideo,['.\video\','LabSceneColor','.',num2str(i),'.jpg'],'jpg');

  12. MATLAB vidObj = mmreader(fname); %motionDet.m fname = 'sidewalk11_23indeo.avi'; a = aviread(fname); %%OLD METHOD frameInfo = aviinfo(fname); totalFrames = frameInfo.NumFrames for i = 1:50 %for i = 1:totalFrames-1 currentFrameDiff = abs(im2double(a(1,i+1).cdata)-im2double(a(1,i).cdata)); movDiff(i) = im2frame (currentFrameDiff); end %MATLAB Movie file figure, movie(movDiff) % FOR AVI MOVIE %movie2avi(movDiff,'sidewalk12_05_07.avi','compression', 'none');

  13. Processing .avi files %readWriteAviFiles fname = 'CarsTarget2.avi'; % extracting the frame information. %frameInfo = aviinfo( strcat( pathname, fname )); frameInfo = aviinfo( fname ); disp( frameInfo ); for cnt = 1:20 mov1=aviread(fname,cnt); frame1 = mov1(1,1).cdata; %uint8 image1= im2double(frame1); figure,imshow(image1); %WRITE INDIVIDUAL FRAMES TO DIRECTORY imwrite(image1,['.\video\','CarVideo','.',num2str(cnt),'.jpg'],'jpg'); end

  14. Create Video from Individual Frames • VirtualDub

  15. Motion Detection • Background Modeling • Model background without moving objects • Represent each pixel in the frame with a 3D Gaussian – mean red, green, blue and covariance matrix • For each pixel, collect n pixel triplets. • Use triplets to estimate mean and covariance matrix • Process future frames by determining the probability of each pixel in the new frame • Threshold the probability, p(r,c)>thres is a foreground pixel (moving object) • Compare pixel values in current frame and estimate if pixel is represented by background distribution or more likely from a different distribution (therefore new object not in background)

  16. Updating Gaussian Distributions • Small changes in the environment will result in thresholding errors • Adapt the Gaussian models by calculating a weighted average • Estimate means and covariance matrix from initial frames • Update distributions using pixels identified as background – distributions will adjust for slight changes in lighting conditions

  17. Instead of using estimated covariance matrix use the identity matrix • How does this change affect performance??

  18. Pixel ModelingStationary Camera Sidewalk Threshold

  19. Overpass Object Tracking Overpass

  20. Face Tracking Example – All Objects

  21. Face Tracking Example – Faces Only

  22. Non-stationary Camera • Example: A camera panning a scene • One approach is to register the adjacent frames • Find key points in adjacent frames • Determine offset • Adjust images so that they overlap • Take difference

  23. NEED TO FIND CORRESPONDENCE BETWEEN FEATURE POINTS IN TWO DIFFERENT IMAGES • Cannot match individual pixels • Need to use a window containing many pixels (5x5, 7x7, 21x21, etc)

  24. Matching on a continuum like texture and edges not very robust • Many edges (and parts of edges) will match • At the very least: • Need to find interest points • Extract patches around interest points • Match patches

  25. Feature Points / Correspondence • Points should be extracted consistently over different views • Points should be invariant to scaling, rotation, changes in illumination • Information in the neighborhood of the point should be unique so that it can be matched

  26. Select Window RegionMatch Region in Second Image Calculate difference between the two patches WindowMatching_ACV.m

  27. Convert to Grayscale

  28. Randomly Select Patch patch = I(80:110,200:230); For Demonstration Use Only Strip Containing Patch

  29. Error=Absolute Difference Between Patch and Strip

  30. Normalized Cross Correlation (Refer to equation on page p313)Also MATLAB docs • w is template • w is average value of elements in template • f is the image • f is the average of the image where f and w overlap • Denominator normalizes resulting in an output range of -1, +1 • High value for absolute value of output is a good match

  31. MATLAB Cross Correlation Function g = normxcorr2(template, f)

  32. Find Max Value in |g| d = abs(g); [ypeak, xpeak] = find(d == max(d(:))); %Adjust location by size of template ypeak = ypeak-(size(patch,1)-1)/2; xpeak = xpeak-(size(patch,2)-1)/2; fprintf('\n Center of Patch: ypeak is %d and xpeak is %d \n\n\n', ypeak, xpeak); figure, imshow(Igray) hold on plot(xpeak, ypeak, 'ro')

  33. Red – strongest response Green – second strongest response

  34. But How Do We Select Points? • Junctions or Corners • Stable over changes in viewpoint

  35. Feature Points?

  36. Subtract First Window from Second Window

  37. Moravec’s Corner Detector • Overview: • Select window size • Shift window over image region • If window over uniform region, shifts in all directions will result in small changes • If window over edge, shifts along edge will results in small changes, but shifts across edge will result in large changes • along edge – no change • Perpendicular to edge – large change • If window over corner, than shifts in all directions will result will result in large changes • Detect corner by finding regions that have large changes in all directions

  38. Subtract First Window from Second Window Window moved vertically, no change Window moved horizontally, no change Window moved in either direction, large change

  39. Harris Points Corner Response function, C: C = det(A) – αtrace2(A), where A is the autocorrelation matrix . Fig. 1: Autocorrelation matrix, where w(x, y) is the window function and I(x, y) is the image REF: image from Wikipedia

  40. Corner Detector

  41. Rotated Image

  42. Non-stationary Camera • Example: A camera panning a scene • One approach is to register the adjacent frames • Find key points in adjacent frames • Determine offset • Adjust images so that they overlap • Take difference

  43. Panning a Building Complex

  44. Two Sequential Frames - Color What if you just simply take the difference between two adjacent frames?

  45. Two Sequential Frames -Grayscale

  46. abs(frame11-frame10)

  47. Overall approach

  48. Points of Interest

More Related