Collaboration via the Info. Utility

1. Collaboration via the Info. Utility John Canny Endeavour Mini-retreat 1/20/2000

2. Consequences of Ubiquitous Computing and Calm Technology: • As computing becomes ubiquitous and invisible, it also ceases to be the focus of activity. • The focus shifts instead to people and the tasks they are working on. • This implies a Human-Centered Computing approach (HCC) to designing applications. • Our work involves collaborations with several faculty outside CS who are expert in aspects of human behavior, and especially group behavior, via a HCC center.

3. Three takes on collaboration • Design media for natural (physical) interaction: Desks, PRoPs, Bearables. • Model behavior: Activity-Based Computing (ABC): • uses tacit data to infer structure in group members’ activities. • Learning as collaboration: • Large classes create opportunity via group cooperation and student perspectives to help other learners.

4. Three takes on collaboration • Design media for natural (physical) interaction: Desks, PRoPs, Bearables. • Model behavior: Activity-Based Computing (ABC): • uses tacit data to infer structure in group members’ activities. • Learning as collaboration: • Large classes create opportunity via group cooperation and student perspectives to help other learners.

5. Designing Media: Distributed Cognition Theory • People use artifacts in the world to share their thoughts (Norman, Hutchins, Pea). • Computer-enhanced artifacts have richer behaviors and be shared across distance.

6. A desk that loses entropy... • The UPM (Universal Planar Manipulator) is a desk that can move many objects placed on it. It is a tangible input/output device.

7. Applications of the UPM • Teaching Chemistry: A tangible simulation of planar organic chemistry. Colored disks for atoms, simulation of bond forces and reactions, movement of atoms by the desk. • Urban planning. Phycons of buildings moved by initially by hand, then aligned by the desk according to a set of rules. • Other layout problems: web site design (moving postit-like notes).

8. UPM principle of operation

10. PRoPs: Proxys for remote collab. • There is still a huge gap between face-to-face and videoconferencing. • First difference: Physical cues: gaze, posture, proxemics... are missing or distorted. • Much collaboration involves “being in” a remote space. You need to have a body there...

11. PRoPs: Proxys for remote collab. • PRoPs are internet-controlled robot avatars that allow you to have a second presence somewhere. • The user sits at a computer console watching video from the PROP, and controlling it with mouse and joystick.

12. Theoretical underpinnings • Right now, much of the theory driving PROP design comes from outside CS: • Non-verbal communication (D. Keltner psychology). • Theory of interpersonal psychology (G. Mendelsohn). • Spatial reasoning and spatial memory.

13. Double-mirror element in glasses Heads-up virtual image Pocket-worn portable LCD TV Laptop in briefcase with wireless TV and keyboard TXs. Wireless chordal keyboard or Palmpilot in pocket for input. Bearable computers • The laptop can serve as a “bearable” computer, linked wirelessly to its owner.

14. Advantages of Bearables vs. Wearables • Short term: • Leverage higher performance of laptops and mini-TV displays compared to wearable components. • Long term: • No wires between components. • No power source to the head. • The laptop display is preferable sometimes. • Some optical benefits for augmented reality.

15. Three takes on collaboration • Design media for natural (physical) interaction: Desks, PRoPs, Bearables. • Model behavior: Activity-Based Computing (ABC): • uses tacit data to infer structure in group members’ activities. • Learning as collaboration: • Large classes create opportunity via group cooperation and student perspectives to help other learners.

16. ABC: Activity-Based Computing • How do you build a UI when the computer is invisible, e.g. a smart space? • What is context and how do you use it? • How does ABC help collaboration?

17. ABC: Activity-Based Computing • How do you build a UI when the computer is invisible, e.g. a smart space? • Build a conceptual model instead. • What is context and how do you use it? • How does ABC help collaboration?

18. ABC: Activity-Based Computing • How do you build a UI when the computer is invisible, e.g. a smart space? • Build a conceptual model instead. • What is context and how do you use it? • Use Activities as the context. • How does ABC help collaboration?

19. ABC: Activity-Based Computing • How do you build a UI when the computer is invisible, e.g. a smart space? • Build a conceptual model instead. • What is context and how do you use it? • Use Activities as the context. • How does ABC help collaboration? • Pro-active document sharing; Understanding roles; Document context;

20. ABC: Activity-Based Computing • Activities are clusters with: • Users • Documents • Tools • The model can be implemented as a GUI, or with speech, etc. • Nearness encodes awareness.

21. ABC: Motivation • Based on “activity theory”, a rich psychological theory of human behavior, especially collaboration. • AT Emphasizes on tools (such as computers) as mediators of activity. • Includes the role of symbol systems (language, documents) as mediators. • Ties together development (learning) and everyday activity. • Rich models of context.

22. ABC: Uses • Context (disambiguation) of person, object, tool. • Pro-active sharing of information. • Skill mining (tracking user expertise). • Document visibility and access rights. • Understanding individual’s roles in activities. • Managing work relationships. • Inferring quality of documents/tools. • Sensemaking for documents. • Perspectives on a document or meeting. • Attention management/awareness.

23. ABC: Under development now • Context (disambiguation) of person, document, tool. • Pro-active sharing of information. • Skill mining (tracking user expertise). • Document visibility and access rights. • Understanding an individual’s roles in activities. • Managing work relationships. • Inferring quality of documents/tools. • Sensemaking for documents. • Perspectives on a document or meeting. • Attention management/awareness.

24. ABC: Tacit data sources • Direct communication: 1-1 email, phone, F2F. • Topical discussions, forums, F2F meetings. • Document access, reading, doc. markup. • Document search and browsing, who, when.

25. Knowledgescapes: Info Retrieval with tacit data Search Engine Query(terms) Document1 Document2 Information Need Document3 Document4 Rankings from reading time. Treat as probability of interest or E(interest)

26. Info Needs and tacit rankings Infoneed1 P(D1|N1) Document1 Document2 Infoneed2 Document3 Infoneed3 Document4 Document5 P(D5|N3)

27. First use: Most popular hits • Start with a query string and use all the information need nodes recorded for that query. • Return highest weighted docs. Document1 Document2 Infoneed2 Document3 Query Infoneed3 Document4 Document5

28. Tacit knowledge in user queries • The IN is usually more specific than the query, and this is encoded in the user’s tacit rankings. US to France Infoneed1 Document1 Document2 Infoneed2 “Airlines” Document3 Document4 Infoneed3 Document5 West to East Coast

29. Tacit knowledge in user queries • Tacit user ranking while browsing boosts most probable info needs, improving recall. US to France Infoneed1 Document1 Document2 Infoneed2 “Airlines” Document3 Document4 Infoneed3 Document5 West to East Coast

30. Tacit knowledge in user queries • Tacit user ranking while browsing boosts most probable info needs, improving recall. US to France Infoneed1 Document1 Document2 Infoneed2 “Airlines” Document3 Document4 Infoneed3 Document5 West to East Coast

31. Tacit knowledge in user queries • Tacit user ranking while browsing boosts most probable info needs, improving recall. US to France Infoneed1 Document1 Document2 Infoneed2 “Airlines” Document3 Document4 Infoneed3 Document5 West to East Coast

32. Current testbeds: • Berkeley DigLibs (environmental) collections. Knowledgescapes on Dlib is running at: http://indios.cs.berkeley.edu/knowledgescapes.html • IMDB • Philip Stark’s statistics course. • NEEDs: the ME courseware database. Future: • My groups web and campus library access.

33. Three takes on collaboration • Design media for natural (physical) interaction: Desks, PRoPs, Bearables. • Model behavior: Activity-Based Computing (ABC): • uses tacit data to infer structure in group members’ activities. • Learning as collaboration: • Large classes create opportunity via group cooperation and student perspectives to help other learners.

34. Education as collaboration:Opportunities of scale. • In lecture format classrooms, attention is the critical resource (Norman, Papert...). • Attention is best gained by interaction: with artifacts (the LOGO model) or with other students (the Stanford TVI model).

35. An augmented reality classroom Virtual image with notes, questions, private chat space Glasses to rotatethe laptop image Physical space

36. Livenotes in the classroom • Students work in groups of 4; communicate silently via pen or keyboard chat. • Each group has one main note-taker; others add their own comments or questions to the transcript. • Students can mark up a group transcript, the lecturer’s notes, or a non-archived window. • One student per group works as facilitator or TA, posing questions to the others.

37. Using Livenotes • The group transcripts include notes, plus student comments and questions. • Remote live participants should be equally engaged in the note-taking.

38. Initial feedback • 4 Students used Livenotes in a grad course in F99 on IBM laptops running Netmeeting on a wireless net. • Reactions: • Overlay touch screens were bad, everyone used keyboard chat. • Difficulty in listening and chatting simultaneously only in first lecture. • After that, attention level higher. No chance of falling asleep. • Many notes: two parallel threads, the note-taker and the group chat. Group chat periodically comes back to lecture content as new notes appear from the note-taker.

39. Next Step • Move to Vadem Clios with wireless: • Support both keyboard and pen note-taking. • Cheaper, lighter, batteries last all day. • Develop custom software based on feedback: • Support conversation threads. • Include hyperlinks (or hyper-ink). • Include timestamping to allow synchronization with MM transcript of the lecture. • In-class “lecture rewind” was proposed as a useful feature.

40. Further along... • Offline students lose interaction, but gain through access to live student perspectives (their notes). • By organizing live student notes according to expertise and a few dimensions of perspective, we can provide offline students with an ideal “scaffolding” perspective.

41. The End..

42. PROPs and Bearables are complementary • A user with a bearable display can see a video overlay of the PROPs pilot.

43. PROPs and Bearables are complementary • A user with a bearable display can see a video overlay of the PROPs pilot. • This display can cover a much larger area than the PROPs built-in display, and provide body language cues.

44. PROPs and Bearables are complementary • A user with a bearable display can see a video overlay of the PROPs pilot. • This display can cover a much larger area than the PROPs built-in display, and provide body language cues. • In this display, the pilot is standing up, implying he is controlling the PROP with a wearable computer.