EE 4780: Introduction to Computer Vision

1. EE 4780: Introduction to Computer Vision Introduction

2. EE 4780 • Instructor: Bahadir K. Gunturk • Office: EE 225 • Email: bahadir@ece.lsu.edu • Tel: 8-5621 • Office Hours: MW 10:00 – 12:00

3. EE 4780 • We will learn the fundamentals of digital image processing and computer vision. • Lecture slides, problems sets, solutions, study materials, etc. will be posted on the class website. [www.ece.lsu.edu/gunturk/EE4780] • Textbook is not required. • References: • Gonzalez/Woods, Digital Image Processing, Prentice-Hall, 2/e. • Forsyth/Ponce, Computer Vision: A Modern Approach, Prentice-Hall. • Duda, Hart, and Stork, “Pattern Classification,” John Wiley&Sons, 2001. • Shapiro/Stockman, Computer Vision, Prentice-Hall. • Horn, “Robot Vision,” MIT Press, 1986.

4. Grading Policy • Your grade will be based on • Problem Sets: 30% • Midterm: 30% • Final: 40% • Problem Sets • Mini projects: Theoretical problems and MATLAB assignments • 4-5 Problem Sets • Individually or in two-person teams

5. Digital Image Acquisition Sensor array • When photons strike, electron-hole pairs are generated on sensor sites. • Electrons generated are collected over a certain period of time. • The number of electrons are converted to pixel values. (Pixel is short for picture element.)

6. Digital Image Acquisition • Two types of quantization: • There are finite number of pixels. (Spatial resolution) • The amplitude of pixel is represented by a finite number of bits. (Gray-scale resolution)

7. Digital Image Acquisition Take a look at this cross section

8. Digital Image Acquisition • 256x256 - Found on very cheap cameras, this resolution is so low that the picture quality is almost always unacceptable. This is 65,000 total pixels. • 640x480 - This is the low end on most "real" cameras. This resolution is ideal for e-mailing pictures or posting pictures on a Web site. • 1216x912 - This is a "megapixel" image size -- 1,109,000 total pixels -- good for printing pictures. • 1600x1200 - With almost 2 million total pixels, this is "high resolution." You can print a 4x5 inch print taken at this resolution with the same quality that you would get from a photo lab. • 2240x1680 - Found on 4 megapixel cameras -- the current standard -- this allows even larger printed photos, with good quality for prints up to 16x20 inches. • 4064x2704 - A top-of-the-line digital camera with 11.1 megapixels takes pictures at this resolution. At this setting, you can create 13.5x9 inch prints with no loss of picture quality.

9. Image Resolution Don’t confuse image size and resolution.

10. Bit Depth – Grayscale Resolution 8 bits 7 bits 6 bits 5 bits

11. Bit Depth – Grayscale Resolution 4 bits 3 bits 2 bits 1 bit

12. Matrix Representation of Images • A digital image can be written as a matrix

13. Digital Color Images

14. Color Displays CRT LCD Polarize to control the amount of light passed.

15. Video = vertical position = horizontal position = frame number ~24 frames per second.

16. Why do we process images? • To facilitate their storage and transmission • To prepare them for display or printing • To enhance or restore them • To extract information from them • To hide information in them

17. Image Processing Example • Image Restoration Original image Blurred Restored by Wiener filter

18. Image Processing Example • Noise Removal Noisy image Denoised by Median filter

19. Image Processing Example • Image Enhancement Histogram equalization

20. Image Processing Example • Artifact Reduction in Digital Cameras Original scene Captured by a digital camera Processed to reduce artifacts

21. Image Processing Example • Image Compression Original image 64 KB JPEG compressed 15 KB JPEG compressed 9 KB

22. Image Processing Example • Object Segmentation “Rice” image Edges detected using Canny filter

23. Image Processing Example • Resolution Enhancement

24. Image Processing Example • Watermarking Original image Watermarked image Generate watermark Hidden message Secret key

25. Image Processing Example • Face Recognition Search in the database Surveillance video

26. Image Processing Example • Fingerprint Matching

27. Image Processing Example • Segmentation

28. Image Processing Example • Texture Analysis and Synthesis Photo Computer generated Pattern repeated

29. Image Processing Example • Face detection and tracking http://www-2.cs.cmu.edu/~har/faces.html

30. Image Processing Example • Face Tracking

31. Image Processing Example • Object Tracking

32. Image Processing Example • Virtual Controls

33. Image Processing Example • Visually Guided Surgery

34. Cameras • First camera was invented in 16th century. • It used a pinhole to focus light rays onto a wall or translucent plate. • Take a box, prick a small hole in one of its sides with a pin, and then replace the opposite side with a translucent plate. • Place a candle on the pinhole side, you will see an inverted image of the candle on the translucent plate.

35. Perspective Projection • Perspective projection equations

36. Pinhole Camera Model • If the pinhole were really reduced to a point, exactly one light ray would pass through each point in the image plane. • In reality, each point in the image place collects light from a cone of rays.

37. Pinhole Cameras Pinhole too big - many directions are averaged, blurring the image Pinhole too small - diffraction effects blur the image

38. Cameras With Lenses • Most cameras are equipped with lenses. • There are two main reasons for this: • To gather light. For an ideal pinhole, a single light ray would reach each point the image plane. Real pinholes have a finite size, so each point in the image plane is illuminated by a cone of light rays. The larger the hole, the wider the cone and the brighter the image => blurry pictures. Shrinking the pinhole produces sharper images, but reduces the amount of light and may introduce diffraction effects. • To keep the picture in sharp focus while gathering light from a large area.

39. Compound Lens Systems

40. Real Lenses • Rays may not focus at a single point. Spherical aberration Spherical aberration can be eliminated completely by designing aspherical lenses.

41. Real Lenses • The index of refraction is a function of wavelength. • Light at different wavelengths follow different paths. Chromatic aberration

42. Real Lenses Chromatic Aberration

43. Real Lenses • Special lens systems using two or more pieces of glass with different refractive indeces can reduce or eliminate this problem. However, not even these lens systems are completely perfect and still can lead to visible chromatic aberrations.

44. Real Lenses Causes of distortion • Barrel Distortion & Pincushion Distortion Stop (Aperture) Chief ray (normal)

45. Real Lenses • Barrel Distortion & Pincushion Distortion Corrected Distorted http://www.vanwalree.com/optics/distortion.html http://www.dpreview.com/learn/?/Image_Techniques/Barrel_Distortion_Correction_01.htm

46. Real Lenses Vignetting effect in a two-lens system. The shaded part of the beam never reaches the second lens. The brightness drop in the image perimeter.

47. Real Lenses Optical vignetting example. Left: f/1.4. Right: f/5.6. f-number focal length to diameter ratio

48. Real Lenses Long exposure time Short exposure time

49. Real Lenses Flare Hood may prevent flares

50. Real Lenses Flare