Measuring Team Shared Understanding: Using Analysis-Constructed Shared Mental Model Methodology

1. Measuring Team Shared Understanding: Using Analysis-Constructed Shared Mental Model Methodology Tristan E. Johnson Learning Systems Institute, Florida State University, Tallahassee, FL, USA tjohnson@lsi.fsu.edu International Workshop and Mini-conference on Extending Cognitive Load Theory and Instructional Design to the Development of Expert Performance August 29-30, 2005 Open University of the Netherlands

2. Background • Team Performance • Team Cognition • Link between SMM and Team Performance • Shared Understanding and Shared Mental Models Development of SMM and its relation to team performance

3. Team Cognition Elaborated view of team cognition including team interactions and SMM development

4. Shared Knowledge Types • Task Knowledge—domain specific • Team Knowledge—5 factors

5. Team Knowledge Factors • Team Knowledge • Knowledge about team members and tasks that they need to perform • Teammates knowledge, Task knowledge • Team Skills • Abilities associated with successful job performance • Communication skills, Interpersonal skills, Leadership skills, Skills to deal with conflict and team cohesion • Team Attitudes • Internal state that influences team members’ choices or decision to act in a certain way under particular circumstances • Shared belief, Shared value • Team Dynamics • Combination of dynamic processes of team coordination and team cohesion • Team coordination, Team cohesion • Team Environment • External conditions affecting the foundation of the team mental model • Technology, Organization, Synchrony & Geographic dispersion

6. Measuring Task Knowledge • Measuring Shared Understanding—measuring concept relatedness • Card sorting, cognitive interviewing, MDS, Pathfinder, surveys, casual maps (Langan-Fox, Code, Langfield-Smith, 2000; Trochim, 1989) • Concept Mapping • Statistical analysis • Descriptive analysis • Analysis Constructed - Shared Mental Model (AC-SMM)

7. SMM Elicitation Techniques • TmC-SMM—Whole team elicitation (1 map) • AC-SMM—Individual elicitation with aggregation (n maps) • SMM i— desired shared mental model state • TmC-SMM — involves team negotiation and interaction • SMM∂— altered team shared mental model state • AC-SMM —retains the initial ICMM state

8. AC-SMM Methodology Rationale • Knowledge Elicitation • Process allows simultaneous consideration of concepts • Reflection and changes during elicitation • Analysis • Allows for explication of implicit relationships—considering 1) logic, 2) structure, and 3) spatial orientation • Relatedness • Specific to three levels • Concepts • Links • Clusters • Appropriate for studying shared understanding in applied settings

9. AC-SMM Methodology Overview • Instrument Design • Structured/Semi-Structured/Unstructured • Task Analysis (Generate Concepts) • Data Collection • Guided Practice • Individually Constructed Mental Model (ICMM) Elicitation • Data Analysis • Phase I: ICMM Analysis/Coding • Relatedness at concepts, links, clusters levels • Allows for explication of implicitrelationships • Implicit coding has [logic and spatial] or [logic and structural] support • Phase II: Shared Analysis • Determine sharedness level—number or percentage of team members • Phase III: AC-SMM Construction • Generates SMM

10. Phase I: ICMM Analysis Factor 1: Concepts • Explicit individual nodes Factor 2: Links • Two concepts joined explicitly [connector] or implicitly Factor 3: Clusters • Two or more connectors explicitly bridging three or more concepts • May have implicit connections with evidence • Combination of clusters—Sub- and Super- clusters Factor 4: Emphasis and Sequence • Explicit notation of node emphasis or node order

11. ICMM Coding—Links

12. ICMM Coding—Clusters

13. ICMM Coding—Emphasis and Sequence

14. ICMM Coding Example

15. Phase II: Shared Analysis • Determine Sharedness Level Criterion—Number or Percentage • Shared Data Used for AC-SMM Construction

16. Phase III: AC-SMM Construction

17. Research • General Research Focus • What task knowledge is shared? • How does shared understanding change over time? • What are the patterns of change? • What is the affect of task performance on the shared understanding of the team?

18. Data Collection Timeline

19. Concepts

20. Team Profiles Findings

25. Shared Data Findings, Team 1 Only

28. Shared Data Summary Per Team

37. Cross Case Findings

40. Pre, Mid, Post Analysis

41. ACSMM Scores

42. General Findings • Similarity among ICMMs tends to increase as does the number of clustered concepts, the tendency is for the number of concepts used to decrease. • ICMMs were becoming more structured and more representative of the team task • These ideas are not yet proven. • We have designed a set of studies to try and validate our hypothesis • This work is intended to not only learn about teams that work in the various settings, but to validate the AC-SMM analysis model

43. Summary • Richer description of shared understanding in teams • AC-SMMs compared over time to determine change in shared understanding • Lacks weighted measures and precise distances between concepts, but future work will include descriptive statistics of the key factors • Lack of prepositional descriptors • As we become more precise and descriptive we can utilize this new knowledge to better explain and understand team cognition • Facilitate team training with intent to improve team performance outcomes

44. Thanks for your attention.Questions?