Distributed Cognition: Towards a New Foundation for Human-Computer Interaction Research

1. Distributed Cognition: Towards a New Foundation for Human-Computer Interaction Research Authors: James Hollan, Edwin Hutchins, David Kirsh Presented by: Luis Bathen

2. Outline • Theory behind Distributed Cognition • A Distributed Cognition Approach at HCI • Socially Distributed Cognition • Embodied Cognition • Culture and Cognition • Ethnography of Distributed Cognitive Systems • Integrated Framework for Research • Ethnographies • Beyond Direct Manipulation • History-Enriched Digital Objects • Intelligent Use of Space • Conclusions

3. Outline • Theory behind Distributed Cognition • A Distributed Cognition Approach at HCI • Socially Distributed Cognition • Embodied Cognition • Culture and Cognition • Ethnography of Distributed Cognitive Systems • Integrated Framework for Research • Ethnographies • Beyond Direct Manipulation • History-Enriched Digital Objects • Intelligent Use of Space • Conclusions

4. Theory behind Distributed Cognition • The term cognition comes from the Latin term cognoscere, which means “to know” and it is commonly referred to as the human ability to process information • The traditional view of cognition in HCI (not limited to HCI, also in other fields like psychology, AI, etc.) limits it the individual processes within a mind • This limits us to think of only a user level and how he/she will interact with a system • This model is insufficient to really examine today’s complex systems because they do not take into account interaction between individuals • As systems become more complex, and they scale to larger and larger systems, an individual will no longer have complete control over it, rather, it requires multiple individuals to collaborate in order to accomplish a goal • Navigational Systems (Ships and Airplanes) • Distributed Cognition tries to extend the concept of what is cognitive beyond the individual

5. Theory behind Distributed Cognition (Cont.) • There are two main principles that separate the classical view of cognition and distributed cognition • Boundaries: The classical view of cognition limits it to just an individual. Distributed cognition looks for cognitive processes wherever they occur on the basis of functional relationships of elements that participate together in the process • Range of mechanisms participating in a cognitive process: The traditional view of cognition looks at the manipulation of symbols within an individual, where as Distributed Cognition looks at a broader class of cognitive events that are not necessarily encompassed by the skin or skull • When these principles are applied to human behavior in “the wild” there are at least three interesting kinds of cognitive processes • Cognitive processes may be distributed across the members of a group • Cognitive processes may involve coordination between internal and external (material or environment) structure • Processes may be distributed through time in such a way that the products of earlier events can transform the nature of later events

6. Outline • Theory behind Distributed Cognition • A Distributed Cognition Approach at HCI • Socially Distributed Cognition • Embodied Cognition • Culture and Cognition • Ethnography of Distributed Cognitive Systems • Integrated Framework for Research • Ethnographies • Beyond Direct Manipulation • History-Enriched Digital Objects • Intelligent Use of Space • Conclusions

7. A Distributed Cognition Approach • Socially Distributed Cognition • Anthropologists and sociologists studying knowledge and memory, AI researchers building systems that do distributed problem solving, social psychologists studying small group problem solving, etc., have all arrived at the same idea • Social organization is itself a form of cognitive architecture • Cognitive processes involve trajectories of information transmission and transformations, so the patterns of these information trajectories reflect some underlying architecture • Rationale: Since social organization – plus the structure added by the context of activity – largely determines which way information flows through a group, then social organization may itself be viewed as a form of cognitive architecture • Distributed cognition means more than that processes are socially distributed across members of a group, rather, it encompasses the group members as well as their interactions with other people as well as with their environments

8. A Distributed Cognition Approach (Cont.) • From the Society of Mind: “…each brain contains hundreds of different types of machines, interconnected in specific ways which predestine that brain to become a large, diverse society of partially specialized agencies.” • Implication: the cognition of an individual is also distributed • Social organization in a ship: there is a captain, navigators, radio operators, engine engineers, etc., all working to move the ship

9. Embodied Cognition • The only way to understand the mind, and how it works is to consider the body and what helps the body and mind to function • Tools and work materials are not just stimuli for a cognitive system, but at times, they become part of the system itself • A blind person’s cane is part of his/her world • A cell biologist’s microscope is a central part of the way they view the world • A tool can be integrated in the way people think, see, and control activities and part of the distributed system of cognitive control

10. Culture and Cognition • The study of cognition cannot be separated from the study of culture because agents live in complex cultural environments • Culture emerges out of the activity of human agents in their historical contexts, as mental, material and social structures interact • Culture in the form of history of material artifacts and social practices, shapes cognitive processes, particularly cognitive processes that are distributed over agents, artifacts and environments • By moving the boundaries on cognition from the individual, it becomes part of culture • Cognitive sciences traditionally view culture as a body of content on which the cognitive processes of individual persons operate • From distributed cognition’s perspective, culture shapes the cognitive processes of systems that transcends individuals

11. Culture and Cognition (Cont.) • The basic idea behind the connection between culture and cognition is in that culture, particularly, a person’s environment serves as a reservoir of resources for learning, problem solving and reasoning • Culture is a process that accumulates partial solutions to frequently encountered problems • Without these ‘partial solutions’ we as agents would have to find solutions to these problems from scratch, hence, culture allows us humans to learn from our ancestor’s experiences • Of course, culture is not without its ‘evils’ since it programs us to a way of thinking, and there might be problems that could be considered ‘unsolvable’ using this one way of thinking where the problem may in fact be solvable if one was to take a different approach

12. Ethnography of Distributed Cognitive Systems • Ethnography: by definition, ethnography refers to the genre of writing that presents qualitative description of human social phenomena, based on fieldwork • Earlier ethnographic methods on cognitive focused on understanding the meaning of words • Ethnography of distributed cognitive systems retains the interest of the individual mind, but adds to that a focus on the material and social means of the construction of action and meaning • It is important to emphasize the ethnography of distributed cognitive systems focuses not only on the meaning of things or what people know but also, action, or how people go about doing things, how they use what they know to do what they do • Action was ignored by previous approaches, this is no longer the case in the new model • Cognitive ethnography brings together many specific data collection and analysis techniques such as interviewing, surveys, participant observation, and audio and video recording • Because understanding both the individual, what he/she knows and what actions he/she takes based on the knowledge he/she has, it is imperative to use experts and observe their behavior while they work in their environments • Their interactions with their surroundings as well as how data is used need to be observed in order to design more useful tools

13. Outline • Theory behind Distributed Cognition • A Distributed Cognition Approach at HCI • Socially Distributed Cognition • Embodied Cognition • Culture and Cognition • Ethnography of Distributed Cognitive Systems • Integrated Framework for Research • Ethnographies • Beyond Direct Manipulation • History-Enriched Digital Objects • Intelligent Use of Space • Conclusions

14. Integrated Framework for Research • The proposed approach for human-computer interaction research integrates distributed cognition with methods for design of digital work materials • Distributed cognition theory identifies a set of core principles that widely apply • People establish and coordinate different types of structure in their environment • It takes effort to maintain coordination • People offload cognitive effort to the environment whenever practical • There are improved dynamics of cognitive of load-balancing available in social organization Distributed Cognition Ethnography Work Places Experiments Work Materials Integrated Research Activity Map

15. Integrated Framework for Research (Cont.) • These principles serve to identify classes of phenomena that merit observation and documentation • Cognitive ethnography has methods for observing, documenting and analyzing such phenomena, particularly, information flow, cognitive properties of systems, social organizations, and cultural processes • Cognitive ethnography seeks to understand what things mean to the participants in an activity and to document the means by which the meanings are created • Unfortunately, there are times when tools are used for other means than for what they were originally intended, expert users make opportunistic use of environmental use to simplify tasks, and this phenomena can only be captured through observation • Consider the following example: while observing a group of pilots, it was observed that they routinely displayed the test pattern on the weather radar as a reminder that a final fuel transfer was in progress • Fork as a knive

16. Integrated Framework for Research (Cont.) • While the study of cognition in ‘the wild’ can answer many kinds of questions about the workplace, the richness of real world settings places limits on the power of observation, this is where focused experiments come in place • After observing certain phenomena in ‘the wild’ tests can be conducted in order to fully understand the behavior • Ethnography offers clever ways of getting things done that can be incorporated into a new design. • Experiments can be run in order to refine the results obtained from ethnography which in turn can be introduced into the design • This can all be done in a form of feed back look, where on each pass, new uses for different things as well as ways of doing things can be refined and introduced into the design

17. Ethnographies • During the 80s, Professor Hutchins conducted an extended cognitive ethnography of navigation aboard U.S. Navy ships. • It was based on this ethnography that the very notion of distributed cognition and the need for cognitive ethnography arose from the observation that the outcomes that matter to the ship were not determined by the cognitive properties of a single individual but instead were a product of the interaction of many individuals and their tools • From this ethnography, distributed cognition was developed as well as the introduction of tools into the cognitive processes • It documented the social organization of work and showed how learning happened both in individuals and at the organizational level • It talked about how navigators had mental models of the ships and how they were able to see themselves in these models, by doing so, they were able to detect errors in their tools. • Since error detection is a key cognitive property of this system, it would be nice to know how navigators perceived these models in their minds • Although observation detected this behavior, it alone was not enough to understand it, hence the need for expert users and focused experiments to understand it

18. Beyond Direct Manipulation • One of the key focus of research based on distributed cognition is the nature of representations and the ways that people use representations to do work • This goes back to the use of icons, and metaphors to represent things • Desktop/PC, Files, Folders, etc. • Ethnography results showed that people often go back and forward between representations of items and the actual things they represent • For instance, when we move the icon of a file from a folder to another, we don’t think we are just moving an icon, but rather, we are moving the actual contents of the files • There are limits to how well a representation can resemble the thing that it represents. For instance, in a mac, if we drag a CD or a Floppy disk to the trashcan, it does not actually mean that we are throwing the items away, rather, we are going to ‘eject’ them

19. Beyond Direct Manipulation (Cont.) • Because we manipulate icons in space, it is possible to exploit them. For instance: • We can leave files we want to delete later near the ‘trashcan’ • We can group files we need for a project in a single location • As users become more familiar with an environment, they become expert users, adapt to the environment, and find new ways of achieving their goals • As system complexity grows, it is very important for the field of HCI to understand how agents become closely coupled with their environments

20. History-Enriched Digital Objects • In the real world, history of use is sometimes available and informs us of ways to interact with objects • A well-worn section of a door handle suggests where to grasp it • A used book with highlights might suggest important subjects • In the office, the most used pieces of paper are usually at the top of stacks of paper • We can translate this history of use notion from the real world to the digital world • Digital objects can contain usage information and history data like who used it last, when it was used and for how long • For instance, a website that logs usage, can keep track of what subjects were browsed the most, who looked at them, for how long, and so on • It is beneficial for users who are looking for the most recent data or for the most commonly viewed pages • Of course, privacy issues come into place, but users can be allowed to keep control of what data is logged • History in systems environments can be used by the users to become more efficient users, and to build upon the experiences of other users

21. Intelligent Use of Space • By observing people’s behavior we can see what things are important to them, and how they arrange items to fit their needs • A left handed hot-dog vendor can position his ketchup/mustard/etc. to his left while a right handed hot-dog vendor can do the opposite • It is important to understand how users arrange items because it is part of their thinking process • In Kirsh (1995) functions of space were classified into three main categories: • Spatial arrangement that simplify choice: things that need more attention next to each other, order of things to be processed, helps makes decision problems less complex • Spatial arrangement that simplify perception: chef differentiates between identical spoons by placing them near different spices, jigsaw puzzles where similar pieces are grouped together thereby exploiting capacity of the visual system • Spatial arrangement that simplify internal computation: largest spaghetti noodle can be singled out by striking the bundle on the table, the max is computed by using the material and the properties of the world

22. Conclusions • The authors proposed a new methodology for the study of cognition, or the interaction between humans and systems • Their work extends the study of cognition from the individual mind to the individual and its surroundings including the tools that aid them with their daily activities and their interactions with their peers • They propose a research framework that consists of distributed cognition, ethnography, experiments and the user’s environment to design complex systems

23. Questions?