R. Alterman Brandeis Unviersity Review of Human Factors Discovery and Invention Projects

1. Cognitive Model of Computer-Mediated Joint Activity R. AltermanBrandeis UnviersityReview of Human FactorsDiscovery and Invention Projects Code 342Cognitive and Neural SciencesOffice of Naval Research 5-6 September 2001

2. Objective • Develop a model of how participants in a computer-mediated cooperative activity coordinate and jointly make sense of the unfolding situation. • Develop a methodology and an analyst’s tool that supports a development team in constructing secondary representations that improve the performance of the users in staying coordinating and maintaining a common viewpoint during a dynamic situation.

3. Problem/Deficiency Being Addressed • The Navy continues to move into an era of distributed, computer-mediated planning, problem solving, and joint activity. The joint activity among ship, air, submarine, and shore personnel is largely mediated by computer. • A significant task is to develop tools and interface technology that structures the activities of distributed planners and actors in a way that simplifies ongoing coordination and reduces the work involved in sharing meanings, understandings, and assessments, given the press of time in an ongoing crisis situation.

4. Technical Approach • Coordinating representation is an external representation. It is structured and shared amongst a group of cooperating actor. Its designated purpose is to simplify the interactions among participants at the critical junctures of a joint cooperative activity. • Use coordinating representation as a signaling system to improve coordination and communication during the performance of computer-mediated tasks. • Develop a methodology and an analyst’s tool for building a set of effective coordinating representations for a given computer-mediated cooperative task.

5. Participants and Staffing • Rick Alterman, PI (1 month) • Mix of Ph.D., Masters, Undergraduates, Post Docs, and Consultant • Roughly equivalent to 3 Ph.d’s • Jim Storer (1 month)

6. Current Progress • Completed an experiment that confirmed the utility of the coordinating representation for improving performance. • A significant part of the analysis of the data was to determine the methods used among the participants to jointly make sense of a shared dynamic situation. • Developed a VCR-like device that allows an analyst to replay the activities of a set of users in a computer-mediated problem-solving session. • Began to develop an analyst’s tool for constructing a set of effective coordinating representations for a given computer-mediated cooperative activity.

7. Accomplishments • Completed initial experiment to confirm utility of coordinating representations. • Also analyzed data to develop model of how participants maintain a consistent view of the world. • Coordination of Talk; Coordination of Action. To appear in special issue of HCI Journal on “Talking about Things” • Conference paper appeared in Proceedings of Twenty-Fourth Annual Meeting of the Cognitive Science Society, 2001

8. Future Directions/Payoffs • Continue development of analyst’s tool • Model interaction among participants • Apply methodology and use tool to develop coordinating representations for another domain. • The methodology and the analyst’s tool we develop will have wide application and is likely to significantly impact the development of the next generation of groupware applications.

9. Outline • Testbed (VesselWorld) • Joint Sensemaking • Secondary Structures • Examples of Coodinating Representations • Experiment • Methodology & Tools

11. Report Review Repair Confirm Review 1. Crane1: center is clean – nothing in the center 4 quads 2. Crane2: ok, we forgot the west, I didn’t get down that far 3. Tug1: 600-400 square is clean, I don’t have to worry about tit? 4. Crane2: correct, that’s where I started 5. Crane2 about 100 N of sb 6. Tug1: okay cool, grabbing the sb Joint Sensemaking

12. Secondary Structures • Created by participants to simplify talk at critical junctures of cooperative activity • Indicate problem areas where the introduction of a coordinating representation could be productive

13. 1. Co-ordination of talk a. Adjacency pairs to propose and confirm next action b. Expectations that adjacency pairs will occur for each of the actions in an extended sequence of tightly coupled cooperative behaviors 2. Coordination of action 1. Crane1: sub lift 2. Crane2: LL 3. Crane2: k 4. Crane2: sub load 5. Tug1: the next XL needs nothing 6. Crane1: k 7. Crane2: ok, then XLD right? 8. Crane2: sub Lift 9. Tug1: yep 10. Crane1: k 11. Crane2: sub load 12. Crane1: k 13. Crane2: sub sep 14. Crane1: sep Close Coordination

14. Coordinating Representations (Two Examples) • Reduce work in timing closely couple joint actions • Maintain consistent information about shared domain objects

15. Timing of Joint Actions

16. Object List

17. Experiment • Three groups with Coordinating Representations; Three groups without Coordinating Representations • Training + 10 hours of problem-solving • 49% improvement in clock time • 38% reduction in the number of events generated • 57% reduction in the amount of electronic chatting • 61% reduction in total errors

18. Tools & Methodology • VCR-Tool • Replay online activities of participants in a computer-mediated cooperative activity • Analyst’s Tool • Segment and sort the conversational threads. • Profile and model interactions among participants

20. Analysis tool, part 1 • Analyst can to tag and classify utterances • Mark up text with standard tags (errors, conventions, 2ndry structures) for profiling later • Allows note-taking on a per-utterance basis

21. Analysis tool, part 2 • Analyst can cluster utterances and mark conversation threads • Emergent categorization of utterances for flexibility in analysis • Threaded view reveals conversational structure and allows tracking of errors due to causes like participants' mismatched understanding of conversational state

22. Selected Publications • Alterman, R., Feinman, A., Introne, J., and Landsman, S., Coordination of Talk; Coordination of Action. To appear in special issue of HCI Journal on “Talking about Things”. • Alterman, R., Feinman, A., Introne, J., and Landsman, S., Coordinating Representations in Computer-Mediated Joint Activities. Proceedings of Twenty-Fourth Annual Meeting of the Cognitive Science Society, 2001. • Alterman, R. and Garland, A. Convention in Joint Activity. Cognitive Science 25:4, 611-657, 2001. • Garland, A. and Alterman, R. Learning Procedural Knowledge to Better Coordinate. International Joint Conference on Artificial Intelligence, 2001.