1 / 14

Perceptive Context for Pervasive Computing Trevor Darrell Vision Interface Group

Perceptive Context for Pervasive Computing Trevor Darrell Vision Interface Group. Human-centered Interfaces. Free users from desktop and wired interfaces Allow natural gesture and speech commands Give computers awareness of users Work in open and noisy environments

mahon
Download Presentation

Perceptive Context for Pervasive Computing Trevor Darrell Vision Interface Group

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Perceptive Context for Pervasive ComputingTrevor DarrellVision Interface Group

  2. Human-centered Interfaces • Free users from desktop and wired interfaces • Allow natural gesture and speech commands • Give computers awareness of users • Work in open and noisy environments • Outdoors -- PDA next to construction site! • Indoors -- crowded meeting room • Vision’s role: provide perceptive context

  3. Perceptive Context • Who is there? (presence, identity) • What is going on? (activity) • Where are they? (individual location) • Which person said that? (audiovisual grouping) • What are they looking / pointing at? (pose, gaze)

  4. Virtual views • Profile silhouette: • Frontal Head:

  5. Examples: VH-generated views • Faces: • Gait:

  6. Plan view Foreground Range Intensity Range-based stereo person tracking • Range can be insensitive to fast illumination change • Compare range values to known background • Project into 2D overhead view • Merge data from multiple stereo cameras.. • Group into trajectories… • Examine height for sitting/standing…

  7. Audio input in noisy environments • Acquire high-quality audio from untethered, moving speakers • “Virtual” headset microphones for all users

  8. Vision guided microphone array Cameras Microphones

  9. Audio-visual synchrony detection

  10. Audio associated with left face Audio associated with right face AVMI Applications • Image localization from audio + image variance AVMI • Audio weighting from video (detected face)

  11. I I t + 1 I t Z Z t + 1 Z t Brightness and depth motion constraints Parameter space yt = yt-1

  12. Closed-loop 3D tracker Track users head gaze for hands-free pointing…

  13. Head-driven cursor • Related Projects: • Schiele • Kjeldsen • Toyama • Current application for second pointer or scrolling / focus of attention…

  14. “Look-to-talk” Subject not looking at SAM ASR turned off Subject looking at SAM ASR turned on

More Related