Video in CSCW

1. Video in CSCW Part I—Michael Boyle Part II—Michael Rounding

2. Part I—Video Mediated Communication • Role of visual channel in collaboration • Video-mediated communication technologies • Opportunities and applications of VMC

3. Role of Visual Channel • Significance • Function and operation of visual behaviours • Technological requirements • Consequences of inadequate support

4. Significance of Visual Channel • Principle means for navigation and orientation in our 3D physical world • Hand-eye coordination and motor skills • Principle means for experiencing our 3D physical world • Other senses are limited • Greatly inconvenienced if disabled

5. Characteristics of Visual Channel • Humans can only focus on small area • Large peripheral visual cone • Humans are adept at handling vast quantities of visual information • Scanning, searching, reacting to changes • Recognizing, classifying • Predominantly diurnal creatures • Large eyes; eyebrows; stereoscopic vision

6. Uses of Visual Channel • Infer much about people from visual information • Gender, age, status • Identity, background, personality • Bulk of human knowledge encoded visually • Arts, aesthetics, entertainment • Communication

7. Visible Behaviours • Gaze • Facial expressions • Gestures • Posture

8. Gaze—Function and Operation • Indicates locus of interest, activity • One generally looks at something when one talks about it or is about to use it • Uses: grounding, workspace awareness • Signals affective state • Interest, disposition, attitude, emotion • Uses: pre-interaction informal awareness, conversational awareness

9. Gaze—Function and Operation • Conveys extremely subtle yet significant content • Infer big meaning from even small physical changes • Although important, not always observed • Observed <7% of conversation time • Mutual gaze (eye contact) <5% of time

10. Gaze—Technological Requirements • Careful camera position • Eyes must be fully visible, aperture at eye-level • Large field of view • See point of interest • High image resolution • Clearly see orientation, percieve minor variations • High frame rate, low latency • Smoothly convey rapid changes in real-time

11. Gaze—Consequences of Inadequate Support • Systemic miscommunication, misinterpretation if undiagnosed • Requires extra effort to explicitly communicate signals, or disambiguate/repair miscommunication • De-referencing focal points is tough!

12. Facial Expressions—Function and Operation • Involves whole of the face • Eyes, eyebrows, forehead, mouth (especial lips and corners), cheeks, ears, nose • Conveys affective information • Blinks, winks, smiles, frowns, furrowed brows, wrinkled up nose, etc. • Uses: conversational awareness, disambiguate meaning of sensitive or easily-misinterpreted content

13. Facial Expressions—Technological Requirements • Tight audio/video synchronization • Face must be in full view with high resolution • Fast frame rate to capture winks, etc.

14. Facial Expressions—Consequences of Inadequate Support • If latencies are too high, then signals may come at inappropriate times • If resolution too low, may not be able to clearly discern expression, adding to ambiguity • Could lead to break-down of conversation if important expression miscommunicated

15. Gestures—Function and Operation • Uses mostly hands, fingers, and arms • Position, orientation, movement • Position relative to other parts of the body, or other objects in environment • Rapid, generally small movements • Context crucial for interpretation • Silent, subtle, implicit long-distance communication

16. Gestures—Classes • Pointing gestures • Grounding: de-reference deictic references • Emblematic or iconic • Convey content • Useful when words don’t come to mind • Sometimes redundant • “Beats” • Convey emphasis

17. Gestures—Technological Requirements • Large field of view • Full body must be visible • High image fidelity • Resolution, frame rate high enough to capture small, rapid motions • Likelihood that there will be obstructions • Fixed, mono-camera configuration insufficient

18. Gestures—Consequences of Inadequate Support • Miscommunication, resulting in explicit communication • Misinterpretation of gestures could lead to conversation breakdown • Missed deictic references impairs workspace awareness • Loss of personable content

19. Posture—Function and Operation • Consequence of work • Involves whole body • Particularly head, neck back, hips • Position, relative orientation are cues to look for • Communicates affective information • Particularly availability: informal awareness

20. Posture—Technological Requirements • Field of view • Must capture whole body yet provide background for reference • Fidelity doesn’t need to be high

21. Posture—Consequences of Inadequate Support • Incorrect assessment of availability during pre-interaction periods • Unnecessary interruptions, missed opportunities • Erroneous assessment of affective information • Impact on smoothness, but likely not so terrible

22. Visible Behaviours—Recap • Subtle yet significant, sometimes redundant • Signal attitude, disambiguate meaning • Valuable personable aspect • Conflicting technological constraints • People can get by without them, but interaction dynamics are fundamentally altered

23. Part I—Video Mediated Communication • Role of visual channel in collaboration • VMC technologies • Opportunities and applications of VMC

24. A/V Infrastructure • Cameras and displays • Digital vs. analog • Microphones and speakers • Digital vs. analog • Half-duplex vs. full-duplex • Microphone arrays, environmental audio • Noise, feedback, attenuation

25. Network Infrastructure • Medium • Digital vs. analog • Dial-up, ADSL, T1, wireless • Quality of service (QoS) factors: • Bandwidth, latency, jitter, delivery guarantees • Packet-switched vs. circuit-switched • Broadcast/multicast vs. unicast

26. Audio Software • Properties: sample size, rate, number of channels • Compression: PCM, µ-Law, ADPCM, GSM/CELP, MP3 • Buffer size: transmission overhead vs. latency • Filtering, mixing

27. Video Software • Lossy vs. lossless compression • RLE (run-length encoding): lossless • replace runs of same-coloured pixels with count • JPEG: lossy • uses signal-processing techniques to change representation of image to bring out redudant aspects • quantization throws away resolution in chroma channel not visible to human eye • entropy (Huffman) coding on result

28. Video Software • Intraframe vs. interframe compression • “M-JPEG:” intraframe • MPEG: intraframe/interframe hybrid • JPEG compress key frames • Encode back and forward differences of non-key frames • Decreases bandwidth, increases latency • Quality: bits-per-pixel

29. Video Software • Frame rate • Frame size

30. General Impact of Inadequacies • Frame size, resolution, fidelity • Subtle facial expressions, gaze not discernable • Lost of critical eye contact • Field of view • Locus of attention not visible • Parts of face, body used in expressions not visible

31. General Impact of Inadequacies • Frame rate • Subtle changes lost • Latency • Cues come at inappropriate time • A/V synchronization • Reduces value of redundant encoding

32. VMC Technology—Recap • Technology never perfect • Bandwidth/quality trade-off • Results: • Content communicated subtly and implicitly via visual channel must be communicated explicitly via some other channel, or lost altogether • People adapt, but personability of FTF is lost

33. Part I—Video Mediated Communication • Role of visual channel in collaboration • VMC technologies • Opportunities and applications of VMC

34. Opportunities and Applications • Tele-presentation/distance learning • Augmenting shared workspaces • Supporting informal interaction • Tele-presence and mediating contact • Function, operation, example systems, and lessons learned

35. Tele-presentation • Presenter/audience with presentation matter • Numerous configurations: • with/without local audience • multiple participants at each remote site • multiple presenters at same/different sites • Formal • Information dissemination applications

36. Tele-presentation • Example systems: • Forum • Telep • TV call-in show

37. Tele-presentation—Lessons Learned • Audience likes it • Enables participation • Integrates well with other work tasks • Presenters not so favourable • Value of back-channels • Reduced local attendance • Sub-conversations

38. Augmented Shared Workspaces • Coordinating activities in shared workspace requires communication channels • E.g., text-chat or telephone • Integrate shared workspace and shared communication channels to create a shared tasks environment • Traditional meeting tasks

39. Augmented Shared Workspaces • Example systems: • Montage • ClearBoard • TeamWorkstation • NetMeeting

40. Augmented Shared Workspaces—Lessons Learned • Diminished priority of video relative to audio • Call setup: minimize time, provide sense of approach • Requirements vary with task • Critical mass • Know your users

41. Casual Interaction Support • Promote social interactions of serendipitous nature • Based on value of social interactions to productivity • “Seamless” integration of distant sites into a signal interaction space • Hope to overcome effect of distance on social interactions and collaboration

42. Casual Interaction Support • Example systems: • VideoWindow • Vkitchen • CRUISER

43. Casual Interaction Support—Lessons Learned • Public reaction mixed • Not 100% transparent: technology still a barrier to forging new relationships • Difficult to diagnose failures • Requires extended installation/observation periods

44. Tele-presence and mediating contact • Video makes for a subtle, unobtrusive means of delivering tele-presence for accruing distributed informal awareness • Easy, natural for people to capitalize on visual information to coordinate interactions

45. Summary • Role of visual channel in collaboration • Technological foundations • Opportunities for video in CSCW • Challenges

46. Summary—Role of Video • Communication medium • Coordination medium • Regulation and mediation of interactivity • Subtle, implicit • Important, but not in the obvious ways • Importance varies with task

47. Summary—Video Technologies • Present technology not very robust • Bandwidth/fidelity trade-off • Issues: latency, field of view • Consequences of inadequate technology • Vary from minor inconvenience to conversation breakdown • Must understand when video is crucial • Design should reflect this

48. Summary—Opportunities • Tele-presentation • Meeting support • Communication channel for shared work activities • Casual interaction • Mediating interaction

49. Summary—Challenges • Understanding where video succeeds and fails • Success is not always obvious • Some failures are inevitable • Basic technological issues • Deployment, ubiquity, acceptance, incorporation • Social issues: privacy, accountability