Enhancing Communication Through MultiView Display Technology

1. CS260 Lecture 7 Professor John Canny

2. Computer-Mediated Communication • Communication involves at least two people. What are their respective goals?

3. Electronic Media vs. Face-to-face • Knowledge workers have a host of electronic media available, but still travel across town or across the country for face-to-face meetings. • Why?

4. High-stakes communication • What were the most important communications you have had in the last several years?

5. Media richness (Daft & Lengel) • Match the affordances of the medium to the communication task. High richness (often high stakes) Face-to-Face Ambiguity, ephemeral, expressive Video-conf? Telephony IM Clarity, recording, constraint Email/memos Reports Low richness

6. Media choices in the workplace • For routine business communication, we surveyed users about their choice of media: • Email • Instant Messaging • Telephony • Face-to-face • There was one medium that rated higher under several criteria. What do you think it was?

7. Non-verbal communication • In real life, we use a lot more than speech (or sign language) to communicate. • Non-verbal communication includes: • Gaze, eye contact • Facial expression • Gesture • Posture • Touch • Location (proxemics) • Time • Prosody (speech)

8. Non-verbal communication Which of these cues are preserved by: • Email? • Instant messaging? • Telephony? • Video-conferencing?

9. Non-verbal communication Q: What is the role of these cues in normal communication? A: It depends totally on the role of the communication, e.g. • Routine (giving information, coordinating) • Persuading and being persuaded • Trust, deception and negotiation • …

10. Routine communication • Most of what happens in most organizations. • Doesn’t seem to benefit much from non-verbal cues, and in fact there is evidence that people prefer less-rich media such as email and telephone: • Sproull and Kiesler: computer science students did better with email than face-to-face meetings. • Connell et al.: Business employees preferred the phone over face-to-face and email for routine communication.

11. Persuasion • Seems to be strongly influenced by gaze and facial cues (Werkoven et al.). Note:Most non-verbal cues are not consciously processed. We transmit and receive without being aware of what we are doing. Most non-verbal cues are strongly influenced by our personality and emotional state. Facial expression is different however. We consciously manage it, and it shows very little correlation with emotional state.

12. Trust and deception • Most people emit easy-to-read non-verbal cues when they try to deceive. These are the basis of “lie detector” tests. • They include: • Prosodic speech variation • Skin conduction (due to sweating) • Breathing and heart rate changes • Particular body gesture cues

13. Trust and deception • Facial expression on the other hand, since it is consciously managed, is a poor cue to deception. • Most deception cues therefore, are “below the neck”.

14. Trust and deception • A former president:

15. Trust and deception • A former president:

16. Trust and deception • A former president:

17. Its physically impossible with standard video displays to preserve gaze for a group of people on either side of a video connection. Unfortunately, that is the most common case in commercial settings. Gaze distortion B A

18. Only A believes that the other person is looking at them! This is because of the Mona-Lisa effect. Gaze distortion B A

19. 0 10 20 35 50 Mona Lisa Effect

20. Solutions • Props (mobile presences) address some of these issues. They even support exploration.

21. 1 3 2 MultiView Display Light is retroreflected toward the source in the horizontal direction. Each user has their own projector, sees their own image.

22. MultiView Directional Display • Each view is provided by a projector • The projected image is reflected directly back in the direction of the projector • The image can be seen at varying heights only behind the projector • Each user gets video from a unique camera at the other end.

23. Cameras Projectors MultiView Display

24. 1 3 2

25. 1 3 2

26. 1 3 2

27. MultiView Display • The Multiview design fully preserves gaze cues between all pairs of participants, on both sides of the connection. • It also reproduces everything that’s visible above the table at the other end (same deception cues as a face-to-face meeting). • Goal is to see if we can reproduce persuasion and trust cues.

28. CSCW Evaluation • Evaluation of CSCW systems is difficult because of: • Logistics of data collection (separated in time and space, and huge). • Number and complexity of variables. • Validating re-engineered group work.

29. Logistics of data collection • Suggestions? • Get partners for other locations • Self-reports • Instrument! – record everything in the system • Video – get if at all possible • Get context information

30. Number and Complexity of Variables • Suggestions? • Start with some hypotheses – but keep eyes open • Iterate on experiment design • Keep user tasks simple – caveat about considering entire cooperative activity

31. Validating re-engineered group work • Suggestions? • Long-term evaluations • Co-evolutionary design

32. Livenotes: Collaborative Note-taking • Pen and keyboard input • Unique user color • Import background slides • Small peer group • Group awareness • Pages that users are on

33. Client-Server Topology Group 1: Server • 802.11b networking • Large class broken down into many small groups (3-7 students) • One Tablet per group is set to server mode • Other members’ Tablets connect wirelessly to group’s server Clients … Group n: Server Clients

34. Experiment • Spring 2003 undergraduate HCI class • 21 volunteers, randomly partitioned into • Cooperative note-takers • Individual note-takers (control group) • 4 weeks (7 lectures) • Preloaded skeletal PowerPoint slides

35. Previous Observation • From previous deployments, we learned that • Undergraduates were not used to discussing lecture material with one another • Graduate students engaged in group discussions spontaneously • For this experiment (with undergraduates), we held short, live group discussions in the classroom

36. Cooperative Note-Taking:Richer Notes • Average cooperative note-taking group took more than twice as much notes as individual note-taker. • Average cooperative note-taking group has more commentary, question and answer, and reinforcement marks (46.4%) than individual note-taker (7.6%), above simply note-taking • 24.6% of marks attributed to group interaction

37. Cooperative Note-Taking: Richer Notes

38. Augmented Note-Taking: Observed Behaviors • Elaborated on bullets • Appended bullets to list • Concurred and disagreed with bullet • Noted gist of HCI principles • Noted advantage and disadvantage of HCI technique • Answered questions in bullets

39. Elaborating on Bullets

40. Appending Bullets to List

41. Answered Question in Bullet