Introduction to Computer Vision

1. Introduction to Computer Vision CS223B, Winter 2005

2. Richard Szeliski – Guest Lecturer • Ph. D., Carnegie Mellon, 1988 • Researcher, Cambridge ResearchLab at DEC, 1990-1995 • Senior Researcher, InteractiveVisual Media Group, Microsoft, 1995- • Research interests: • computer vision (stereo, motion),computer graphics (image-based rendering), parallel programming Introduction to Computer Vision

3. What is Computer Vision?

4. World model Computer vision World model Computer graphics What is Computer Vision? • Image Understanding (AI, behavior) • A sensor modality for robotics • Computer emulation of human vision • Inverse of Computer Graphics Introduction to Computer Vision

5. shape estimation • modeling • shape • light • motion • optics • images • IP motion estimation rendering recognition • modeling • shape • light • motion • optics • images • IP surface design 2D modeling animation Computer Vision user-interfaces Computer Graphics Intersection of Vision and Graphics Introduction to Computer Vision

6. Computer Vision [Trucco&Verri’98] Introduction to Computer Vision

7. image processing graphics Images (2D) Geometry (3D)shape Photometryappearance + vision 3 Image processing 2.1 Geometric image formation 2.2 Photometric image formation 4 Feature extraction 5 Camera calibration 7 Image alignment 6 Structurefrom motion 8 Mosaics 9 Stereo correspondence 11 Model-based reconstruction 12 Photometric recovery 14 Image-based rendering Image-Based Modeling Introduction to Computer Vision

8. Syllabus • Image Transforms / Representations • filters, pyramids, steerable filters • warping and resampling • blending • image statistics, denoising/coding • edge and feature detection Introduction to Computer Vision

9. Image Pyramid Lowpass Images • Bandpass Images Introduction to Computer Vision

10. Pyramid Blending Introduction to Computer Vision

11. Parametric (global) warping • Examples of parametric warps: aspect rotation translation perspective cylindrical affine Introduction to Computer Vision

12. Syllabus • Optical Flow • least squares regression • iterative, coarse-to-fine • parametric • robust flow and mixture models • layers, EM Introduction to Computer Vision

13. Image Morphing Introduction to Computer Vision

14. Syllabus • Projective geometry • points, lines, planes, transforms • Camera calibration and pose • point matching and tracking • lens distortion • Image registration • mosaics Introduction to Computer Vision

15. Panoramic Mosaics • + + … + = Introduction to Computer Vision

16. Syllabus • 3D structure from motion • two frame techniques • factorization of shape and motion • bundle adjustment Introduction to Computer Vision

17. 3D Shape Reconstruction Debevec, Taylor, and Malik, SIGGRAPH 1996 Introduction to Computer Vision

18. Face Modeling Introduction to Computer Vision

19. Syllabus • Stereo • correspondence • local methods • global optimization Introduction to Computer Vision

20. View Morphing • Morph between pair of images using epipolar geometry [Seitz & Dyer, SIGGRAPH’96] Introduction to Computer Vision

21. Z-keying: mix live and synthetic • Takeo Kanade, CMU (Stereo Machine) Introduction to Computer Vision

22. Virtualized RealityTM • Takeo Kanade, CMU • collect video from 50+ stream • reconstruct 3D model sequenceshttp://www.cs.cmu.edu/afs/cs/project/VirtualizedR/www/VirtualizedR.html Introduction to Computer Vision

23. Virtualized RealityTM • Takeo Kanade, CMU • generate new video • steerable version used for SuperBowl XXV“eye vision” system Introduction to Computer Vision

24. Syllabus • Tracking • eigen-tracking • on-line EM • Kalman filter • particle filtering • appearance models Introduction to Computer Vision

25. Syllabus • Recognition • subspaces and local invariance features • face recognition • color histograms • textures • Image editing • segmentation • curve tracking Introduction to Computer Vision

26. Edge detection and editing Elder, J. H. and R. M. Goldberg. "Image Editing in the Contour Domain," Proc. IEEE: Computer Vision and Pattern Recognition, pp. 374-381, June, 1998. Introduction to Computer Vision

27. Image Enhancement • High dynamic range photography[Debevec et al.’97; Mitsunaga & Nayar’99] • combine several different exposures together Introduction to Computer Vision

28. Syllabus • Image-based rendering • Lightfields and Lumigraphs • concentric mosaics • layered models • video-based rendering Introduction to Computer Vision

29. Concentric Mosaics • Interpolate between several panoramas to give a 3D depth effect • [Shum & He, SIGGRAPH’99] Introduction to Computer Vision

30. Applications • Geometric reconstruction: modeling, forensics, special effects (ILM, RealVis,2D3) • Image and video editing (Avid, Adobe) • Webcasting and Indexing Digital Video (Virage) • Scientific / medical applications (GE) Introduction to Computer Vision

31. Applications • Tracking and surveillance (Sarnoff) • Fingerprint recognition (Digital Persona) • Biometrics / iris scans (Iridian Technologies) • Vehicle safety (MobilEye) • Drowning people (VisionIQ Inc) • Optical motion capture (Vicon) Introduction to Computer Vision

32. Projects • Let’s look at what students have done in previous years … • Stanford  http://www.stanford.edu/class/cs223b/winter01-02/projects.html • CMU  http://www-2.cs.cmu.edu/~ph/869/www/869.html • UW http://www.cs.washington.edu/education/courses/cse590ss/01wi/ • GA Tech  http://www.cc.gatech.edu/classes/AY2002/cs4480_spring/ Introduction to Computer Vision