Collaborative Time-Sensitive Targeting

1. Collaborative Time-Sensitive Targeting Mary (Missy) Cummings, Stacey Scott Humans and Automation Laboratory http://halab.mit.edu {missyc, sdscott}@mit.edu

2. Current HAL Research Projects Research in the Humans and Automation Lab (HAL) focuses on the multifaceted interactions of human and computer decision-making in complex sociotechnical systems. • Time-Sensitive Operations for Distributed Teams • Measurement of Display Complexity • Collaborative Human Computer Decision Making • Integrated Sensor Decision Support • Reduced Manning for Unmanned Vehicle Teams • Single Operator Control of Multiple Unmanned Aerial Vehicles

3. Human Supervisory Control in NCO

4. Supervisory Control & Teams Supporting Supervisors of Teams of HSC Operators Supporting the Control of Teams of Heterogeneous Unmanned Vehicles

5. Inverting Current UAV Control Schemes • How do operators manage multiple supervisory control processes? • What effects do human performance limitations have on the overall system? • Reduced Manning • Combining Shadow positions • Multiple UV control for a single operator • Distributed control Multi-Modal Watch Station

6. Collaborative Decision Making • Tomahawk Strike Planning: missions and missiles • Determine how humans and computer optimization algorithmscan complement each other • Cognitive strategies

7. Aiding Human-Computer Path Planning • Decision Support for Planetary Exploration • Astronaut traversal replanning task • Path replanning with increasingly complex constraints • Affect of level of automated decision support

8. Goal of this Research Project To identify or develop technologies to facilitate collaborative decision-making in the time-sensitive targeting task domain

9. Time-Sensitive Targeting is Highly Collaborative

10. Teamwork in Network-Centric Operations • Highly connected operators, often working remotely • Heavy reliance on commercially available collaboration technologies • email • instant messaging (‘chat’) • video & desktop conferencing

11. Benefits of Collaboration Technologies • Facilitate collaboration with remote colleagues • Facilitate sharing of and access to digital media during group work (in both co-located and distributed settings) • Provide opportunity and support for casual interactions • Promote ad hoc collaboration & communication • Enhance team cohesion

12. Disadvantages of Collaboration Technologies • Require explicit effort to access / share information • implicit or more subtle communication / interaction is often missing • lack of non-verbal cues • Typically convey information about completed activities, ongoing progress/status information is also important: • Has anyone seen my request for help? • Is someone working on the problem? What’s their status? • Associated costs with using collaboration technologies: • distraction • time & mental effort of asking for or sharing information

13. Enhancing Teamwork in Non-Military Domains • Research in the CSCW domain has been focused on providing technological support for teamwork in many collaborative settings: • corporate, educational, research • Reviews of the CSCW literature has revealed common threads across many new approaches to CTs: • Minimizing overhead costs of sharing information about remote collaborators • Increased use of automation to help provide this information • Many approaches are aimed at supporting the shared activity process

14. Virtual School Project Timeline (Carroll et al., 2003) IBM’s Activity Explorer (Millen et al., 2005) Supporting the Shared Activity Process • Focused on providing team member’s activity awareness (Carroll et al., 2003): • awareness of project work that supports group performance in complex tasks • implies an awareness of other people’s plans and understandings Project View IM (Scupelli et al., 2005)

15. Supporting the Shared Activity Process (cont’d) Approaches for supporting activity awareness: • Feedthrough of action • Visibility of action Project View IM (Scupelli et al., 2005) Scupelli, P., Kiesler, S., Fussell, S.R., & Chen, C. (2005). Project View IM: A Tool for Juggling Multiple Projects and Teams. In Extended Abstracts of CHI 2005: ACM Conference on Human Factors in Computing Systems, pp. 1773-1776.

16. Supporting the Shared Activity Process (cont’d) Approaches for supporting activity awareness: • Feedthrough of action • Visibility of action • Important to situate this information within the context of the shared activity process Virtual School Project Timeline (Carroll et al., 2003) Carroll, J.M., Neale, D.C., Isenhour, P.L., Rosson, M.B. & McCrickard, D.S. (2003). Notification and Awareness: Synchronizing task-oriented collaborative activity. International Journal of Human-Computer Studies, 58(5), pp. 605-632.

17. Challenges Introduced by the TST Domain • Determining what constitutes relevant activities when ‘shared objects’ are not necessarily ‘files’ • Reducing the cost of providing activity-related information to team members • Balancing awareness and information overload • Balancing individual performance and team performance

18. Methodology for Developing Activity Awareness Support for Collaborative TST Operations • Develop representative collaborative TST task scenario • Conduct task analysis on scenario to develop collaborative technology requirements, focusing on the shared activity process • Develop initial team display designs for collaborative TST scenario incorporating CT requirements • Implement team display designs • Develop evaluation protocol and teamwork metrics for evaluation of team displays • Design and build team testing environment • Evaluate team displays • Iterate team display designs based on study results

19. Representative Task Scenario: GTT Task Generalized Team TST (GTT) Task • A convoy is scheduled to passthrough a hostile region • Team of UAV operators must protect convoy until convoy exits region • Stakeholders: • Convoy • UAV Operations Team + UAVs • Strike Team • Intelligence • Enemy

20. report to upper command TST alert UAV OperationsTeam intel TST ISR TST BDA convoy location BDA detects location orders engagement strike team ISR requests BDA controls engages TST ISR target pinpoint Enemy (TSTs) UAVs TST BDA 1A 2A 3A 1B 2B 3B ISR 1C 2C 3C 1D 2D 3D detects convoy Activity Awareness Information Analysis for GTT Task Scenario: Inter-team Interaction

21. Activity Awareness Information Analysis for GTT Task Scenario: Intra-team Interaction TST alerts (convoy) UAV Operations Team TST ISR (intel) TST BDA (intel) convoy location (intel) COM BDA (strike team) TST ISR (UAVs) TST BDA (UAVs) BDA assignments target allocation ISR – STR – BDA report – control request engagement UAV Ops mCDR authorize engagement situation update report to command controls orders engagement requests BDA

22. Activity Awareness Requirements for the GTT Task Scenario • Types of intra-team shared activity information • relevance to current tasks / team member • available task resources • priority of current task activity • operators’ availability (progress, workload, tasking availability) • operators’ interruptibility (status, cognitive availability) • This information helps team members address the questions: • when / how can I help somebody else? • when / how can I be helped? • how do my actions impact others’ activities?

23. Relevant Activity Awareness Information is Role Dependent • COM and mCDR need status and workload information of the UAV Ops to ensure individuals and team are not overwhelmed • mCDR can help UAV Ops prioritize task activities or request support from external resources • helps COM distributed TST and BDA assignments among UAV Ops • UAV Ops need operator & resource availability and interruptibility information • helps minimize disruptions when someone is involved in critical tasking • helps operators know who can feasibly help them, or who might need their help

24. Supporting Team Supervision: Status and Workload Information on a Situation Display for Battlefield Context

25. Supporting Team Supervision: Status and Workload Information on a Team Status Display CONVOY 5% Z1 Z2 Z3 Z1 OK UAV OP 1 OP 1 OP 2 OP 3 total 1A ISR 1B route: 10% 25% 10% 15% total: 20% 50% 90% 53% 1C targets ISR 5 1D UAV OP 2 STR 3 2A BDA: 2 2B 2C Comms Connectivity Status 2D SATELLITE AWACS UAV OP 3 UAVTEAM 3A 3B 3C STRIKE TEAM 3D

26. Ongoing Work: Display Design & Implementation • Iterating on the status and workload information displays • Developing activity awareness displays to integrate into our existing UAV Op simulation environment, MAUVE, that provides operator & resource availability and interruptibility information MAUVE Decision Support Display MAUVE Tactical Display

27. Ongoing: Developing Team Testing Lab HAL Complex Operation Center • 3 Large-screen displays (A) • 4 Reconfigurable operatorstations (B) • 2 Mobile operatorstations (tabletPCs &handhelds) (C) • Wireless headsetswith microphonesfor ‘always on’ audiovia Internet software

28. Ongoing: Developing Team Testing Lab HAL Complex Operation Center • 3 Large-screen displays (A) • 4 Reconfigurable operatorstations (B) • 2 Mobile operatorstations (tabletPCs &handhelds) (C) • Wireless headsetswith microphonesfor ‘always on’ audiovia Internet software

29. Project Summary • Enhancing activity awareness in collaborative TST environments seems to help promise for helping teams coordinate their actions and help reduce the costs of collaborating via technology • Overarching goal of this project is to understand the tradeoffs between supporting individual operator performance and team performance to provide overall mission effectiveness during TST operations

30. This research is funded by: Boeing Phantom Works&Air Force Research Lab For more information: http://halab.mit.edu or sdscott@mit.edu Thanks!