340 likes | 580 Views
Computer Vision and Applications. Computer Vision. Computer vision is the study of extracting content from digital image data (my definition) the analysis of digital images by a computer (Shapiro’s definition) the science and technology of machines that see (Wikipedia definition)
E N D
Computer Vision and Applications Computer Vision Set: Applications Slides by D.A. Forsyth , C.F. Olson, J. Ponce, L.G. Shapiro
Computer Vision • Computer vision is • the study of extracting content from digital image data • (my definition) • the analysis of digital images by a computer • (Shapiro’s definition) • the science and technology of machines that see • (Wikipedia definition) • One textbook says: • “The goal of computer vision is to make useful decisions about real physical objects and scenes based on sensed images.” Computer Vision Set: Applications Slides by D.A. Forsyth , C.F. Olson, J. Ponce, L.G. Shapiro
Applications Image databases Computer Vision Set: Applications Slides by D.A. Forsyth , C.F. Olson, J. Ponce, L.G. Shapiro
More applications Robot vision Mars exploration rover Stanley – winner of 2005 DARPAGrand Challenge Computer Vision Set: Applications Slides by D.A. Forsyth , C.F. Olson, J. Ponce, L.G. Shapiro
More applications Face detection and recognition Computer Vision Set: Applications Slides by D.A. Forsyth , C.F. Olson, J. Ponce, L.G. Shapiro
More Applications Modeling for graphics and animation Surveillance Computer Vision Set: Applications Slides by D.A. Forsyth , C.F. Olson, J. Ponce, L.G. Shapiro
More Applications Document analysis Medical imaging liver kidney kidney Computer Vision Set: Applications Slides by D.A. Forsyth , C.F. Olson, J. Ponce, L.G. Shapiro
More Applications Photo tourism Computer Vision Set: Applications Slides by D.A. Forsyth , C.F. Olson, J. Ponce, L.G. Shapiro
More Applications Games Computer Vision Set: Applications Slides by D.A. Forsyth , C.F. Olson, J. Ponce, L.G. Shapiro
More Applications • Industrial inspection • Biometrics (faces, fingerprints) • Motion analysis (including gestures and actions) • Road / traffic analysis • Real-time tracking • Augmented reality • Human-computer interaction • Visual navigation • Image / video indexing and retrieval • Motion capture and entertainment • …
Related fields • Machine vision • Sometimes refers to industrial applications • Image processing • Transforming one image into another • Pattern recognition • Concerned with classification or description of observations • Data could be anything (not necessarily images) • Photogrammetry • Science of obtaining accurate measurements and maps from photographs (images) Computer Vision Set: Applications Slides by D.A. Forsyth , C.F. Olson, J. Ponce, L.G. Shapiro
Why Study Computer Vision? • Images and movies are everywhere • Fast-growing collection of useful applications • Interesting scientific mysteries • how does object recognition work? • Better understanding of human vision Computer Vision Set: Applications Slides by D.A. Forsyth , C.F. Olson, J. Ponce, L.G. Shapiro
Complexity • Computer vision is far from a solved problem • Successful systems exist • Usually for controlled situations • Often dependent on parameter settings • There are many visual tasks that people perform better than computers Computer Vision Set: Applications Slides by D.A. Forsyth , C.F. Olson, J. Ponce, L.G. Shapiro
Imaging Geometry Computer Vision Set: Applications Slides by D.A. Forsyth , C.F. Olson, J. Ponce, L.G. Shapiro
Pinhole Cameras • Abstract camera model - box with a small hole in it • Pinhole cameras work in practice Computer Vision Set: Applications Slides by D.A. Forsyth , C.F. Olson, J. Ponce, L.G. Shapiro
Distant Objects Are Smaller Computer Vision Set: Applications Slides by D.A. Forsyth , C.F. Olson, J. Ponce, L.G. Shapiro
Parallel Lines Meet Common to draw film plane in front of the focal point. Moving the film plane merely scales the image. Computer Vision Set: Applications Slides by D.A. Forsyth , C.F. Olson, J. Ponce, L.G. Shapiro
The equation of projection • Cartesian coordinates: • We have, by similar triangles, that: • (X, Y, Z) ~ (f X/Z, f Y/Z, f) • f is called the focal length. [X, Y, Z] [fX/Z, fY/Z] Computer Vision Set: Applications Slides by D.A. Forsyth , C.F. Olson, J. Ponce, L.G. Shapiro
The reason for lenses We won’t worry much about lenses in this class. Computer Vision Set: Applications Slides by D.A. Forsyth , C.F. Olson, J. Ponce, L.G. Shapiro
Lens distortion • “Barrel distortion” of rectangular grid is common for inexpensive lenses • Precision lenses can be expensive • Zoom lenses often show severe distortion • Fish-eye lenses also have severe distortion Computer Vision Set: Applications Slides by D.A. Forsyth , C.F. Olson, J. Ponce, L.G. Shapiro
Image capture • Images are not continuous • Typically captured with a CCD camera (charge-coupled-device) • The amount of light striking each location on a grid is integrated over some time period • Rows are read out one at a time • For color images, successive • pixels usually correspond to • different colors • High quality color cameras use • a beam splitter and 3 separate • CCD chips Computer Vision Set: Applications Slides by D.A. Forsyth , C.F. Olson, J. Ponce, L.G. Shapiro
Resolution • Resolution often (but not always) refers to the number of pixels in the image. • Lower resolution has fewer pixels. • Interestingly, faces of people you know can usually be recognized at 64x64 (or less) pixels. • Squint and look at the lowest resolution image. Computer Vision Set: Applications Slides by D.A. Forsyth , C.F. Olson, J. Ponce, L.G. Shapiro
Programming in OpenCV In OpenCV, images are represented as matrices (as in linear algebra). Mat image = imread("photo.jpg"); // Most generic declaration The image could have a number of underlying data types for each pixel: uchar – unsigned byte (greyscale image) Vec3b – vector of 3 bytes (color image) Point2f – point in two dimensions, float many others… Computer Vision Set: Applications Slides by D.A. Forsyth , C.F. Olson, J. Ponce, L.G. Shapiro
Creating images Images can be created using a number of methods: • using namespace cv; // all my code assumes this • Mat image; // creates 0x0 image • Mat image = … // uses copy constructor • Mat image(rows, cols, type); // type is CV_8U, for example • Mat image(rows, cols, type, scalarValue); • Example: Mat allBlue(360, 480, CV_8UC3, Scalar(255, 0, 0)); • Mat_<Vec3b> colorImage = imread(“color.jpg”); // Can be convenient, but now limited to Vec3b images (matrices) // Also, must declare as a similar parameter type when passed Computer Vision Set: Applications Slides by D.A. Forsyth , C.F. Olson, J. Ponce, L.G. Shapiro
Copying images Be careful to remember that most image copy and pass by value methods do NOT perform a deep copy. image2 = image1; // shallow copy void someMethod(Mat imageParam); // shallow copy If you want a deep copy, then use clone (or copyTo): image2 = image1.clone(); // deep copy image1.copyTo(image2); // deep copy Computer Vision Set: Applications Slides by D.A. Forsyth , C.F. Olson, J. Ponce, L.G. Shapiro
Memory management Memory management is handled by the Mat class. • This is different from the IplImage class in OpenCV 1 • This works correctly even if multiple images share the same data • A reference count is kept for each image • The data is deallocated only when the reference count goes to zero • However, this can allow privacy leaks unless you are careful Computer Vision Set: Applications Slides by D.A. Forsyth , C.F. Olson, J. Ponce, L.G. Shapiro
Backwards compatibility OpenCV 2 is backwards compatible with OpenCV 1. IplImage*iplIm = cvLoadImage("photo.jpg"); // Do work with image here cvReleaseImage(&iplIm); // necessary to prevent memory leak Can convert to Mat simply: Mat converted(iplIm); // do not release image until finished Computer Vision Set: Applications Slides by D.A. Forsyth , C.F. Olson, J. Ponce, L.G. Shapiro
Image manipulation OpenCV provides many methods to manipulate entire images: Filtering: blur, smooth, median, gradient, laplacian Transformations: resize, affine, perspective, color space, threshold, flood fill Segmentation: grabCut, watershed Feature detection: edges, corners, lines, circles, template matching, SIFT Computer Vision Set: Applications Slides by D.A. Forsyth , C.F. Olson, J. Ponce, L.G. Shapiro