Boeing-MIT Collaborative Time-Sensitive Targeting Project

1. Boeing-MIT Collaborative Time-Sensitive Targeting Project July 28, 2006 Stacey Scott, M. L. Cummings (PI) Humans and Automation Laboratory http://halab.mit.edu

2. Time-Sensitive Targeting is Highly Collaborative • Many parties involved in TST: • ISR, verifying intel, determining asset availability, coordinating strikes, prosecuting targets • Current collaboration tools such as ‘chat’ introduce overhead costs to communication / coordination: • time/effort needed to request information or respond to information requests • Automatically providing TST operators information related to their collaborators’ current and ongoing activities may: • help teams coordinate their actions • reduce the costs of obtaining activity-related information

3. Increasing Reliance on UAVs in TST Operations • Unmanned aerial vehicles (UAVs) have become an essential part of modern military command and control and TST operations • Increasing autonomy of UAV platforms is changing the role of UAV operations teams Predator Ground Control Station Predator UAV

4. New Human-System Interaction Roles with Increased UAV Autonomy Autonomous UAV Operations2-5 UAVs : 1 Operator Current UAV Operations1 UAV : 2-5 Operators

5. New Human-System Interaction Roles with Increased UAV Autonomy Autonomous UAV Operations2-5 UAVs : 1 Operator Current UAV Operations1 UAV : 2-5 Operators Future UAV Teams

6. Project Goal: Develop Technological Support for Futuristic UAV Teams Engaged in TST Operations Future UAV Teams

7. Project Goal: Develop Technological Support for Futuristic UAV Teams Engaged in TST Operations Future UAV Teams Current Focus: Technology to Support the Team Supervisor

8. Assisting Team Supervisors of UAV Teams • Research Goal: • Develop activity awareness display technology for team supervisor of UAV team engaged in TST operations • Research Approach: • Develop representative TST task to explore UAV team operations • Conduct cognitive task analysis (CTA) to generate requirements for supervisory decision support and activity awareness displays: • Implement supervisory displays in a simulated task environment • Conduct experiment to evaluate effectiveness of interface displays

9. Generalized TST Team (GTT) Task Representative UAV Operations Team Task Scenario “Surveillance support for a ground convoy through a hostile region” Task Details: • UAV team consists of: • 1 Mission Commander • 3 UAV operators, each controlling multiple UAVs • UAVs have camera sensors only • UAV ops must coordinate with external strike team to destroy identified threats • UAV team can communicate with convoy & external intelligence sources

10. Hybrid Cognitive Task Analysis: 4 Step Process: 1 2 4 3 Generating Requirements for Assisting Team Supervisionin the GTT Task Step 1: Scenario Task Overview Step 2: Event Flow Diagram Step 3: Situation Awareness (SA) Requirements Produces Functional & Information Requirements Step 4: Decision Ladders & Display Requirements

11. TST Supervisory Displays & Simulated Task Environment Large-Screen Wall Displays Situation Map Display Mission Status Display TabletPC Display TST Mission Commander Decision Display

12. The Experiment Research Questions: • Do the proposed activity awareness displays enable effective decision making in the supervision of collaborative TST operations? • Do these displays: • facilitate decision making that result in successful mission performance? • facilitate good decision making strategies? • enable appropriate prioritization of emergent team problems? • Does operational tempo impact the ability of the proposed displays to facilitate decision making? • Do the proposed activity awareness displays provide the necessary information to make the appropriate decisions? • If so, is this information presented in a form that is easily understandable or accessible when it is needed? Is this information presented at an appropriate level of detail? • If not, what information is missing?

13. Experimental Task: GTT Team Supervision Task Primary Task: • Monitor the ongoing (simulated) mission progress using the team supervision displays & make certain intervention decisions when necessary to successfully complete their mission (i.e., ensure safe, expeditious passage of the convoy through their area of responsibility). • Available command interventions: • Request the convoy hold its current position or resume is progress, and • Request AWACS surveillance data. Secondary Task: • Target Buffer Violation Detection: • Record all occurrences of targets being scheduled for a strike within 30 seconds of the start of their threat envelop (i.e., the period of time that the target is within weapons range of the convoy, based on its location and its weapons capability).

14. Experimental Design • 2(Targets) x 2(Operators Present) Mixed Design, with repeated measures on the Targets factors Independent Variables • Targets: low and high. • i.e., number of targets the TST team has to identify and destroy • Varying the level of this factor enables the investigation of the impact of operational tempo on the decision making support provided by the team supervision displays. • Operators Present: present or not present. • i.e., whether the team of UAV operators is present or not • Varying the presence of the operators will hopefully help to determine: • if important information is missing, or is difficult to interpret, from the team supervisor displays • whether the displays can help the supervisor keep track of the high mission priority issues when operators are communicating sometimes lower priority, local problems to the team supervisor

15. Experimental Design (cont’d) Dependent Variables I. Quantitative • Mission time (mission ends when convoy clears team’s AOI or convoy is destroyed) • Targeting performance (targets discovered, target discovery rate: total time to find all discovered targets) • Target buffer violation detection performance (detections, omission errors) • Convoy safety performance (convoy health level, convoy safety violations, convoy attacks) • Decision making (number of each type of decision made, number & occurrence pattern of decisions of each type (reactive/proactive)) II. Subjective / Qualitative • Cummings-Myers Display Quality Rating Scale will be used to collect participants’ perceptions the team supervisor displays • Semi-structured interview with retrospective video review to elicit participants information gathering & decision making strategies • Communication Analysis of team communications for ‘Operators Present’ conditions

16. Experiment Timeline • Experimental setup currently being pilot tested • Experiments to commence in August and continue into September/October, depending on subject availability

17. Future Project Directions: Test-bed for Exploring Other Collaborative TST Issues Future UAV Teams

18. Future Project Directions: Test-bed for Exploring Other Collaborative TST Issues Future UAV Teams 1. Activity awareness interfaces for the UAV operators

19. Future Project Directions: Test-bed for Exploring Other Collaborative TST Issues Future UAV Teams 1. Activity awareness interfaces for the UAV operators 2. Interruption recovery tools for the team supervisor

20. Future Project Directions: Test-bed for Exploring Other Collaborative TST Issues Future UAV Teams 1. Activity awareness interfaces for the UAV operators 2. Interruption recovery tools for the team supervisor 3. Interface support for operator hand-off