1 / 21

Where computer vision needs help from computer science (and machine learning)

Where computer vision needs help from computer science (and machine learning). Bill Freeman Electrical Engineering and Computer Science Dept. Massachusetts Institute of Technology September 9, 2009. Outline. My background Computer vision applications

jett
Download Presentation

Where computer vision needs help from computer science (and machine learning)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Where computer vision needs help from computer science (and machine learning) Bill Freeman Electrical Engineering and Computer Science Dept. Massachusetts Institute of Technology September 9, 2009

  2. Outline • My background • Computer vision applications • Computer vision techniques and problems: • Low-level vision: underdetermined problems • High-level vision: combinatorial problems • Miscellaneous problems

  3. At Photokina, in Cologne, Germany

  4. Me (Foreign Expert) and my wife (English teacher), riding from the Foreigners’ Cafeteria at the Taiyuan University of Technology, Shanxi, China

  5. While in China, I read this book (to be re-issued by MIT Press this year), and got very excited about computer vision. Studied for PhD at MIT.

  6. Worked for 9 years at Mitsubishi Electric Research Labs, an industrial research lab doing fundamental research across the street from MIT.

  7. 2001 – present, MIT

  8. Infinite images Joint work with: BilianaKaneva Josef Sivic ShaiAvidan Antonio Torralba

  9. A computer graphics application ofbelief propagation for optimal seam finding

  10. The image database • We have collected ~6 million images from Flickr based on keyword and group searches • typical image size is 500x375 pixels • 720GB of disk space (jpeg compressed)

  11. Image representation GIST [Oliva and Torralba’01] Original image Color layout

  12. Obtaining semantically coherent themes We further break-up the collection into themes of semantically coherent scenes: Train SVM-based classifiers from 1-2k training images [Oliva and Torralba, 2001]

  13. Basic camera motions Starting from a single image, find a sequence of images to simulate a camera motion: Forward motion Camera rotation Camera pan

  14. 2. View from the virtual camera 3. Find a match to fill the missing pixels Scene matching with camera view transformations: Translation 1. Move camera 4. Locally align images 5. Find a seam 6. Blend in the gradient domain

  15. Scene matching with camera view transformations: Camera rotation 1. Rotate camera 4. Stitched rotation 2. View from the virtual camera 3. Find a match to fill-in the missing pixels 5. Display on a cylinder

  16. More “infinite” images – camera translation

  17. Forward Rotate (left/right) Pan (left/right) Virtual space as an image graph • Nodes represent Images • Edges represent particular motions: Image graph • Edge cost is given by the cost of the image match under the particular transformation Kaneva, Sivic, Torralba, Avidan, and Freeman, Infinite Images, to appear in Proceedings of IEEE.

  18. Virtual image space laid out in 3D Kaneva, Sivic, Torralba, Avidan, and Freeman, Infinite Images, to appear in Proceedings of IEEE.

More Related