Human-Centered Computing Telepresence, AR, tangible interfaces

1. Human-Centered ComputingTelepresence, AR, tangible interfaces John Canny UC Berkeley

2. Telepresence, AR, tangible interfacesJohn Canny • PRoPs (Personal Roving Presences) and gesturing avatars. • UPM - a universal planar manipulator. • Bearable computers - turning laptops into augmented reality systems and wearable computers.

3. PRoPs: Personal Roving Presences Space Browser + Eric Paulos Surface cruiser

4. PRoPs: Toward being there • A PRoP is a Personal Roving Presence. • PRoPs seek to bridge the gap between telepresence and being there. • There are two essential aspects of remote presence that only telerobotics can provide: • Non-verbal communication, i.e. gaze, proxemics, gesture etc. which are highly spatialized. • A gamut of human behaviors, such as chance encounters, informal meetings, showing or being shown, browsing, exploring etc, which are affordances of a physical presence.

5. Terrestrial PRoPs: Carts • Carts are a good match to human capabilities. • Our strategy was to list important physical communication skills, and to implement as many as possible.

6. The rest of the story:A taxonomy of interaction behaviors • Verbal: symbolic and prosodic • Nonverbal communication theory (psychology, linguistics, anthropology). Main cues are: • Gaze • Proxemics • Facial expression • Body and hand gesture • Posture • Touch • Example: A Handshake depends on all the cues.

7. Differences between verbal and nonverbal communication • Adam Kendon (83) summarizes the difference very well:...gesture has properties different from speech. In particular, it employs space as well as time in the creation of expressive forms, whereas speech can use only time. We find, therefore, that the way information may be preserved in speech, as compared to gesture, tends to be very different.

8. Why nonverbal skills are spatial • Gaze - very sensitive to mutual direction. Center-right-left. • Proxemics - Depends on distance to others and the surroundings. • Facial expression - the least spatial, directed by gaze. • Gesture - deictic gesture is completely spatial, so are other pictographic forms. • Posture - Directional. • Touch - Cant fake it without space.

9. You need social context as well • Communication. • Persuasion. • Trust-building. • These are radically different contexts that rely on different non-verbal cues. We need to study telepresence in representative scenarios.

10. The two-way street: • PRoPs provide selective control over particular cues. They can be turned on and off at will. • So they provide a potentially interesting testbed for fine-grained study of non-verbal cues. • We can create a rich variety of media and study their influence on the overall perception of the other person.

11. The input side • How do you achieve rich, expressive input for PRoPs/avatars with today’s technology? • Cross-modal mapping from pen gesture to body gesture. Both have symbolic and expressive character:

12. Future research? • Gaze cue repair: Face to face is not perfect. There are common breakdowns in face-to-face communication, such as gaze aversion. • Beyond Being There: It may be possible to rectify gaze aversion using image processing and/or a synthetic gaze cue.

13. Project 2:UPM: A universal planar manipulator • A smart desk that can move a large number of objects placed on it independently. • Provides a tangible interface in both directions for 2D layouts: floor plans, landscapes, stage directions etc. • Uses a small numbers of motors (4) and the non-linearity of friction.

14. UPM: A universal planar manipulator • First prototype used linear voice-coils from ancient disk drives, capable of about 2g acceleration. New prototype should manage about 50gs.

15. Bearable Computing (new project) • An exploration of issues in personal, persistent computing (augmented reality, worn interfaces) using ordinary laptop computers. • Avoid head-mounted displays (expensive and low-res) head-tracking, and cables. • The approach: use optics to overlay computer images on reality, but use laptop or pocket-mounted displays. • Testbed: Grad course in HCC next semester.

16. An augmented reality classroom Virtual image with notes, questions, private chat space Glasses to rotatethe laptop image Physical space

17. An AR classroom • Students work in groups of 5-7, communicating silently via pen or keyboard chat. • Each group has one main note-taker, the others add their own comments or questions to the transcript. • Students can mark up the group transcript, or the lecturer’s notes. There is non-archived chat also. • One student per group works as facilitator or TA, posing questions to the others, and testing understanding.

18. Bearable computers on the go • The laptop can serve as a “bearable” computer, attached wirelessly to its owner. Double-mirror element in glasses Heads-up virtual image Pocket-worn portable LCD TV Laptop in briefcase with wireless TV and keyboard TXs. Wireless chordal keyboard or Palmpilot in pocket for input.

19. Goals of this research • Can we enhance attention in class with live group chat/note-taking? • Do we get a richer transcript with collaborative note-taking? • Does the head-up display make note-taking less distracting? • Will people do more tasks with a mobile bearable computer because the threshold for computer use is lower?