Components of a computer vision system

1. Scene Interpretation Components of a computer vision system Camera Lighting Computer Scene Srinivasa Narasimhan’s slide

2. Computer vision vs Human Vision What we see What a computer sees Srinivasa Narasimhan’s slide

3. A little story about Computer Vision In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to a computer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that. (Szeliski 2009, Computer Vision)

4. A little story about Computer Vision Founder, MIT AI project In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to a computer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that. (Szeliski 2009, Computer Vision)

5. A little story about Computer Vision Founder, MIT AI project In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to a computer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that. (Szeliski 2009, Computer Vision) Professor of Electrical Engineering, MIT

6. A little story about Computer Vision In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to a computer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that. (Szeliski 2009, Computer Vision) Image Understanding

7. A little story about Computer Vision In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to a computer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that. (Szeliski 2009, Computer Vision) Image Understanding Image Sensing

8. Continue on CAPTCHA CAPTCHA stands for "Completely Automated Public Turing test to Tell Computers and Humans Apart". Picture of a CAPTCHA in use at Yahoo. http://www.cs.sfu.ca/~mori/research/gimpy/

9. Breaking a Visual CAPTCHA On EZ-Gimpy: a success rate of 176/191=92%! Other examples http://www.cs.sfu.ca/~mori/research/gimpy/ez/ http://www.cs.sfu.ca/~mori/research/gimpy/

10. Breaking a Visual CAPTCHA On more difficult Gimpy: a success rate of 33%! Other examples http://www.cs.sfu.ca/~mori/research/gimpy/hard/ http://www.cs.sfu.ca/~mori/research/gimpy/

11. Breaking a Visual CAPTCHA YAHOO’s current CAPTCHA format http://en.wikipedia.org/wiki/CAPTCHA

12. Face Detection and Recognition Applications: Security, Law Enforcement, Surveillance

13. Face Detection and Recognition Smart cameras: auto focus, red eye removal, auto color correction

14. Face Detection and Tracking

15. Face Detection and Tracking

16. Face Detection and Tracking Lexus LS600 Driver Monitor System

17. General Motion Tracking Hidden Dragon Crouching Tiger

18. General Motion Tracking Application Andy Serkis, Gollum, Lord of the Rings

19. Segmentation http://www.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/

20. Segmentation using Graph Cuts Application Medical Image Processing

21. Segmentation using Graph Cuts Matting: Soft Segmentation Composition Input

22. Segmentation using Graph Cuts State-of-the-art Tool (videosnapcut.mp4) http://juew.org/projects/SnapCut/snapcut.htm

23. From 2D to 3D http://www.eecs.harvard.edu/~zickler/helmholtz.html

24. Projective Geometry

25. Single View Metrology • http://research.microsoft.com/vision/cambridge/3d/default.htm

26. Single View Metrology • http://research.microsoft.com/vision/cambridge/3d/default.htm

27. Stereo scene point image plane optical center

28. Stereo • Basic Principle: Triangulation • Gives reconstruction as intersection of two rays • Requires • Camera positions • point correspondence

29. Using 3D structure to organize photos http://phototour.cs.washington.edu/

30. Using 3D structure to organize photos http://photosynth.net/

31. Reconstructing detailed 3D models rendered model example input image

32. Reconstructing detailed 3D models rendered model example input image

33. Reconstructing detailed 3D models http://grail.cs.washington.edu/projects/mvscpc/ rendered model example input image

34. Reconstructing detailed 3D models rendered model example input image

35. Reconstructing detailed 3D models rendered model example input image

36. Application: View morphing

37. Application: View morphing

38. From Static Statues to Dynamic Targets MSR Image based Reality Project http://research.microsoft.com/~larryz/videoviewinterpolation.htm …|

39. Spacetime Face Capture System Black & White Cameras Color Cameras Video Projectors

40. System in Action

41. Input Videos (640480, 60fps)

42. Spacetime Stereo Reconstruction

43. Applications Entertainment: Games & Movies Medical Practice: Prosthetics

44. Computational Photography • High Dynamic Range Conventional Image High Dynamic Range Image Nayar et al 2002

45. Computational Photography • High Dynamic Range Modulator Optics Sensor Assorted-pixel camera High Dynamic Range Image Nayar et al 2002

46. Computational Photography • High Dynamic Range Handheld camera Digital Gain Adjustment

47. Computational Photography • High Dynamic Range Handheld camera High Dynamic Range Image Zhang et al 2010

48. Summary • Recognize things • Reconstruct 3D structures • Enhance Photography

49. If you are interested in, Major Conferences: Computer Vision and Pattern Recognition (CVPR) International Conference on Computer Vision (ICCV) European Conference on Computer Vision (ECCV) ACM SIGGRAPH Conference (SIGGRAPH) Faculty: Chuck Dyer, Vikas Singh, Li Zhang Courses: CS766 Computer Vision CS638 Special Topics Computational Photography CS638 Special Topics Computational Methods in Medical Image Analysis