Modeling Capabilities and Workload in Intelligent Agents for Simulating Teamwork

1. Modeling Capabilities and Workload in Intelligent Agents for Simulating Teamwork Thomas R. Ioerger, Linli He, Deborah Lord Dept. of Computer Science, Texas A&M University Pamela Tsang Dept. of Psychology, Wright State University

2. Teamwork and Team Training • Examples: fire fighters, air traffic controllers, military, sports, businesses • Characteristics of effective teams: • commitment to shared goals • communication, information sharing • coordination, synergy, non-interference • back each other up in case of failure (robustness) • flexible, adaptive

3. Training methods • practice; learn team structure (roles) and process (procedures, policies) • build shared mental model; cross-training • stress innoculation (adaptive, load-balancing) • Intelligent Agents can help! • distributed simulations (with cognitive fidelity) • act as role players (teammates or others) • can monitor/evaluate teamwork and act as coach by giving feedback (dynamic or AAR) • Our goal is to develop psychologically-sound methods for using agents to train teams

4. The importance of reasoning about capabilities and workload in teams • load-balancing, adapting to shifting task demands • self-assessment: when to ask for help? • who to ask for help? • requires knowledge of capability of others • also depends on their workload (“best available”) • awareness of load on others: • when not to interfere, disrupt; suppress communication • e.g. air traffic controllers • can also offer to help (proactiveness) • How to get agents to understand this?

5. Toward A Computational Model • Motivating observations: • humans can perform multiple tasks in parallel • humans have internal limits on processing capacity • humans can often get better performance by applying more effort (within limits) • humans can complete tasks faster by more effort • Therefore... • a human is “capable” of doing a task within a deadline if there is enough reserve capacity for the effort required • definition of “capability” for set of tasks depends on finding a schedule such that overlaps do not exceed capacity • options are to delay processing, or “stretch out” a task over longer time to make more resources available for other tasks

6. We assume there is a single resource (similar to attention) • We assume it is bounded: umax • umax may differ between individuals, or with fatigue, etc. • effort=total resource required over time=e(T)=t1..t2 et(T) • let e be average momentary effort: e(T)=(t2-t1).e • in some cases, greater effort leads to better performance • Performance Resource Functions • resource-limited tasks • workload is sum of effort being applied to all tasks at a given moment: wt=Si et(Ti) • must satisfy capacity constraint at all times: umaxwt Umax effort iso-curve average resource utilization resource utilization e task 1 time duration q=F(e) quality or performance effort (res. util.)

7. start end • A schedule is an assignment of a start time, end time, and average effort level for each task in a set: {<tstart,tend,e>i} • Main Definition: an entity is said to be capable of doing a set of tasks T1...Tn if there exists a schedule {<tstart,tend,e>i} such that: 1) meet each deadline: dl(Ti)  tend(Ti) 2) enough effort for required quality: F(ei.(tend-tstart)) q(Ti) 3) never exceed capacity: t umax  wt=Si et(Ti) Umax resource utilization T1 T3 T2 e T4 time

8. Comments • Interruptibility of current tasks? • Computational Complexity • scheduling is NP-hard • exponential in # tasks to solve exactly • in practice, # tasks is small • do humans use heuristics, like longest-first? • Extension to Multiple Resources (ala Wickens) • multiple resource pools, each with own limit • tasks use “profile” of resources; scales with effort • explains differential interference by task types

9. An Agent Could Use this Model of Capabilities... • to monitor/assess a subject’s workload • to explain observed performance decrements • to select training events to push a student’s abilities • to interact appropriately as a teammate • offer to help those who need it • minimize communication to avoid distraction • to evaluate team performance (load-balancing) • to augment the shared mental model of team

10. Weaknesses and Limitations • Integration with discrete tasks? • How valid are additivity and tradeoff assumptions? • Managing priorities: which task to drop, if no feasible schedule exists? • How to determine parameters? • empirically, e.g. via secondary task interference • evaluation method (work-in-progress)