410 likes | 424 Views
Active Capture and Folk Computing. Ana Ramírez and Marc Davis ICME 2004 – Taipei, Taiwan 29 June 2004. UC Berkeley - Garage Cinema Research - Group for User Interface Research. Smart Multimedia Acquisition Systems. First two papers – automatic camera calibration Image Audio
E N D
Active Capture and Folk Computing Ana Ramírez and Marc Davis ICME 2004 – Taipei, Taiwan 29 June 2004 UC Berkeley - Garage Cinema Research - Group for User Interface Research
Smart Multimedia Acquisition Systems • First two papers – automatic camera calibration • Image • Audio • Third paper – understand structure of what is being captured to edit in real time • Active capture - smart cameras that interactively guide and capture human action
Outline • Sample applications • Active Capture • Designing Active Capture algorithms • Future work
SampleApplications Automatic Movie Trailers • Video of capture process Play Video
SampleApplications Automatic Movie Trailers • Video of automatically created movie trailer Play Video
SampleApplications Sports Instruction
SampleApplications Telemedicine Rural Town Large City • leishmaniasis
SampleApplications Automated Health Screening Rural Town Large City • leishmaniasis
Active Capture Direction/ Cinematography Capture Interaction Active Capture Human- Computer Interaction Computer Vision/ Audition Processing
Active Capture Direction/ Cinematography Capture Interaction Active Capture Human- Computer Interaction Computer Vision/ Audition Processing
Active Capture Direction/ Cinematography Capture Interaction Active Capture Human- Computer Interaction Computer Vision/ Audition Processing
Active Capture Direction/ Cinematography Capture Interaction Active Capture Human- Computer Interaction Computer Vision/ Audition Processing
Active Capture Direction/ Cinematography Capture Interaction Active Capture Human- Computer Interaction Computer Vision/ Audition Processing
Active Capture • Traditionally, signal processing algorithms avoid interacting with the user • Signal processing + interaction => more sophisticated recognizers • How to design hybrid algorithms that involve capture, interaction, and processing
Components of Active Capture Algorithms • Simple computer vision and audition recognizers / sensors • Motion • Eyes • Sound • Desired action in terms of recognizers • Interaction script
Design Process • Input: • Desired action = head turn • Recognizers = motion, eyes Motion Eyes time
Design Process • Input: • Desired action = head turn • Recognizers = motion, eyes • Step 1: • Express desired action in terms of recognizers No Motion Motion No Motion Motion Eyes No Eyes Eyes time
Design Process • Input: • Desired action = head turn • Recognizers = motion, eyes • Step 1: • Express desired action in terms of recognizers • Step 2: • Design interaction script
Design Process – Step II Play Video
Design Process – Step II Play Video
Design Challenges Step I - Description of action • Approximate timing • Strict and non strict ordering Step II – Interaction script • What to do if something goes wrong – mediation
Step I - Action Description Visual Language • Observations • Commands • Capture • Time constraints • Strict ordering • Non-strict ordering
Step I - Action Description Visual Language • Observations • Commands • Capture • Time constraints • Strict ordering • Non-strict ordering
Step I - Action Description Visual Language • Observations • Commands • Capture • Time constraints • Strict ordering • Non-strict ordering
Step I - Action Description Visual Language • Observations • Commands • Capture • Time constraints • Strict ordering • Non-strict ordering
Step I - Action Description Visual Language • Observations • Commands • Capture • Time constraints • Strict ordering • Non-strict ordering
Step I - Action Description Visual Language • Observations • Commands • Capture • Time constraints • Strict ordering • Non-strict ordering
Step I - Action Description Visual Language • Observations • Commands • Capture • Time constraints • Strict ordering • Non-strict ordering
Step II – Interaction Script Contextual Inquiries • Golf instructor • Aikido instructor • 911 emergency phone operator • Triage nurse • Children’s portrait photographer • Film and theatre directors [Jeffrey Heer, Nathaniel S. Good, Ana Ramirez, Marc Davis, and Jennifer Mankoff. “Presiding Over Accidents: System Direction of Human Action.” In: Proceedings of the Conference on Human Factors in Computing Systems (CHI 2004) in Vienna, Austria. ACM Press, 463-470, 2004. ]
Step II – Interaction Script Direction and Feedback Strategies • External aids Play Video
Step II – Interaction Script Direction and Feedback Strategies • Decomposition and “Show” Play Video
Step II – Interaction Script Direction and Feedback Strategies • Method shift from “Show” to “Tell” Play Video
Step II – Interaction Script Direction and Feedback Strategies
Summary • Active Capture – smart cameras that interactively guide and capture human action • Sample applications • Automated health screening • Automated movie clips • Sports trainer • Design Challenges • Description of action • Interaction script
Future Work • Support design and implementation of Active Capture applications • Evaluate the relative contribution of signal analysis and user interaction in these hybrid algorithms
Questions Ana Ramírez anar@cs.berkeley.edu www.cs.berkeley.edu/~anar Garage Cinema Research http://garage.sims.berkeley.edu Group for User Interface Research http://guir.berkeley.edu