1 / 18

Vision-Based Interaction: Harnessing the Power of Computer Vision for User Interfaces

Explore the evolution of vision-based interfaces from Myron Krueger's Responsive Environments to modern tools like OpenCV and Papier-Mâché. Learn about detecting, tracking, and recognizing elements in a scene, and discover practical applications in user interface design.

karmap
Download Presentation

Vision-Based Interaction: Harnessing the Power of Computer Vision for User Interfaces

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Vision-Based Interaction Scott Klemmer · 28 November 2006

  2. cs547: Blake Ross and Asa Dotzler Mozilla: Creating simple software in a geek-driven culture

  3. The first vision-based interface • Myron Krueger used computer vision to create Responsive Environments (1970s) • “Reaction is the Medium” • http://www.artmuseum.net/w2vr/timeline/videoplace_video.html

  4. How it works • Video and background are separated in analog using chroma key techniques(think broadcast news) • The first and last points of each raster are stored in the computer, and represent the person’s outline

  5. Vision-based UIs: “Verbs” • Detecting and Tracking elements of a certain type in a scene • Capturing contents of detected objects • Recognizing individual members in an object class

  6. Vision-based UIs: “Verbs” • Detecting and Tracking elements of a certain type in a scene

  7. Vision-based UIs: “Verbs” • Capturing contents of detected objects

  8. Vision-based UIs: “Verbs” • Recognizing individual members in a class

  9. Vision-based UIs: “Nouns” • People (one or multiple) • Bodies • Faces • Hands • Documents • Objects

  10. Vision-based UIs: “Nouns” • People (one or multiple) • Bodies • Faces • Hands • Documents • Objects

  11. Vision-based UIs: “Nouns” • People (one or multiple) • Bodies • Faces • Hands • Documents • Objects

  12. I N F R A S T R U C T U R E Background Subtraction

  13. Image Moments (of Inertia) • 0th moment is mass(total number of pixels)

  14. Image Moments (of Inertia) • 1st moment is center

  15. Image Moments (of Inertia) • 2nd moment is orientation

  16. Tools for Vision apps • Intel’s OpenCV • C API to highly optimized image processing functions (threshold, dilate, optical flow, …) • http://www.intel.com/research/mrl/research/opencv • Fast to run! Slow to develop • Great for vision folks; too low-level for app folks • Papier-Mâché • Java API (and to some extent visual UI) for vision (and other physical input) • http://guir.berkeley.edu/papier-mache • Fast to develop! Slow to run • Great for app folks; ~5 fps can sometimes be too slow

  17. Good Vision Books • Computer Vision: A Modern Approach • David Forsyth and Jean Ponce (2003) • Fantastic book; but goal is more theoretical understanding than practical application • Robot Vision • Berthold Horn (1987) • More focused on apps and interactive algorithms • Somewhat out of date

  18. Next Time… Software Tools Past, Present, and Future of User Interface Software Tools, Brad Myers, Scott E. Hudson, Randy Pausch Natural Programming Languages and Environments, Brad A. Myers, John F. Pane, Andy Ko

More Related