Recognizing Teamwork Activity in Observations of Embodied Agents

1. Linus J. Luotsinen School of Electrical Engineering and Computer Science University of Central Florida B. Sc. University of Dalarna, Sweden, 2002 M. Sc. University of Central Florida, 2004 Ph.D. Dissertation Defense November 06, 2007 Recognizing Teamwork Activity in Observations of Embodied Agents

2. Committee Members • School of Electrical Engineering and Computer Science • Dr. Ladislau Bölöni • Dr. Avelino Gonzalez • Dr. Kenneth Stanley • Department of Statistics and Actuarial Science • Dr. Liqiang Ni

3. Outline Covered in candidacy • Introduction • Data Acquisition and Knowledge Engineering • Teamwork Activity Recognition using Hidden Markov Models • Team-Oriented Feature Extraction • Role-Based Teamwork Activity Recognition • Tuning the Performance of Teamwork Activity Recognition • Conclusions

4. Team merge (t=0) Goal (t=200) Convoy (t=100) Introduction • Recognize teamwork activity in a stream of positional agent traces, and annotate them with the recognized actions • Video demonstration

5. Applications • Surveillance • Recognize multi-agent activity in surveillance video feeds • Training • Identify discrepancies and deviations from the actions performed by an expert team • Smarter agents • Model the opponent team to imitate or countermeasure its actions • Automated annotation • Automatically index large databases for fast content retrieval • After Action Review • Digital Video Recorder

6. Challenges • Observation noise • Position traces can be distorted by inaccuracies in sensors and localization algorithms • Alignment problems • Movement performed at different location and orientation • Scaling problems • Movement performed at different physical scale • Temporal scaling • Movement happens slower or faster in time • Terrain distortion • Movement distorted because of adaptation to the terrain

7. Challenges • Movement variants • Movement in alternative ways that map to the same label • Uncertainty regarding the role of the agents in the team • Role count variants • Movement with different number of agents in the team • Agents changing their roles during the team action

8. Outline • Introduction • Data Acquisition and Knowledge Engineering • Software tools we developed • Datasets we acquired, segmented and labeled • Teamwork Activity Recognition using Hidden Markov Models • Team-Oriented Feature Extraction • Role-Based Teamwork Activity Recognition • Tuning the Performance of Teamwork Activity Recognition • Conclusions

9. Software Tools we Developed • Teamwork Scenario Editor (TSE) • Interactive knowledge engineering tool • Video editor interface • Visualize large datasets and geographical areas • All knowledge engineering work in this thesis was performed using the TSE • Acquire the datasets • Find teamwork activities • Label the activities

10. Datasets we Acquired, Segmented and Labeled • Real-world warfare exercise • Recorded over a three days • Data collected from hundreds of soldiers and tanks equipped with GPS devices, laser range finders and laser range detectors

11. OTBSAF Mini-MOUT Datasets we Acquired, Segmented and Labeled • Military Operations in Urban Terrain (MOUT)

12. Military Movement • We extract military movement patterns from the warfare databases • In the military domain the movement techniques and formations are selected based on the situational awareness of the team • Traveling • Enemy contact not expected • Fast movement speed • Traveling overwatch • Enemy contact possible • Medium movement speed • Characterized by continuous movement of lead unit and alternating advancement of rear units • Bounding overwatch • Enemy contact expected • Slow movement speed • Alternating or successive bounds

13. Military Movement • Formations are used in combination with movement techniques • Formation is selected based on visibility needs, firepower focus and so on • Column • Leader followed by rear units • Fire power in all directions (flanks, front and rear) • Line • Fire power in the front • Wedge • Fire power in the front and in the flanks • Echelon • Enemy contact expected in the front or in the echeloned flanks • Used when one flanks are secured by obstacles

14. Outline • Introduction • Data Acquisition and Knowledge Engineering • Teamwork Activity Recognition using Hidden Markov Models • Team-Oriented Feature Extraction • Role-Based Teamwork Activity Recognition • Tuning the Performance of Teamwork Activity Recognition • Conclusions

15. Teamwork Activity Recognition using Hidden Markov Models • A spatio-temporal pattern recognition problem • Recognize teamwork from X and Y coordinates over time for multiple agents • Why Hidden Markov Models? • Mathematically sound • Temporal by nature • Successfully applied in the past (e.g. speech recognition) • Ways of encoding teamwork activity in HMMs • Knowledge engineering • Learn from a set of representative examples

16. Teamwork Activity Recognition Teamwork Scenario Editor External database Teamwork Database Import Learning Export Visualization Identification Classification Representative Examples Teamwork Activity Models Real-time data stream Annotated Teamwork Behavior Workflow Overview

17. The Hidden Markov Model • The HMM consists of a number hidden states • Transition probabilities • Emission probabilities • Gaussian PDF • Initial probabilities HMM with 3 hidden states

18. The Hidden Markov Model • Learning algorithms • Baum-Welch • Optimize by maximizing • Segmental K-means • Optimize by maximizing , where is the optimum hidden sequence given by the Viterbi decoding algorithm • Classification • Determine the probability that the input sequence was generated by the HMM • Forward evaluation algorithm

19. Baseline Input Format for the HMM + Basic Preprocessing • The input is a vector of the agent positions • VT = {v1, v2, v3,…, vt} • vt = {x1, y1, x2, y2, x3, y3, …, xn, yn} • It is very unlikely that the team action will be repeated in the same location and position! • We perform a pre-processing of the input data which allows us to recognize team actions happening at arbitrary locations and orientation: • Translation • Align with team centroid • Rotation • Align with x-axis • This is not sufficient, many other distorting factors can happen: scaling, terrain distortion, different ordering of the agents – we are dealing with these later in this presentation

20. Experimental Setup • Test problem • Real-world military warfare exercise • Train and test data • Six activities (extracted using TSE) • Artificial activities added for testing

21. Results: Recognition Accuracy • Classification accuracy is 82% • Matches the performance of the knowledge engineering approach As presented at the AAMAS-07 conference

22. Results: Real-Time Analysis • HMM with 4 states and 6 classes  ~9.4ms

23. Shortcomings: A Lot of Assumptions! • We assumed that the teamwork activities were performed at the same scale • We assumed that there is no interaction with the environment • Observation input is of fixed arrangement, hence, we assumed that recognition is performed on the same team the HMM was trained for • We assumed that all activities can be modeled using the same number of hidden states in the HMMs • Dimensionality and state space for larger teams will quickly grow out of control • We do not recognize the roles of the agents in the teamwork activity • Who is the leader and who is the follower? • The recognizer is not practical!

24. Outline • Introduction • Data Acquisition and Knowledge Engineering • Teamwork Activity Recognition using Hidden Markov Models • Team-Oriented Feature Extraction • Role-Based Teamwork Activity Recognition • Tuning the Performance of Teamwork Activity Recognition • Conclusions

25. Team-Oriented Feature Extraction • Improve recognition accuracy and robustness by replacing the “vector of positions” input with a collection of team-oriented features • The features are extracted through pre-processing • Some of the features replace the existing input vector • Some of the features are the input of the role recognizer (and there is some overlap) • It requires a more complex recognition workflow (which will be shown later, together with the role recognition module) • Extract semantically rich features from the agent position traces • Discretization process • Intuitive descriptions of teamwork activities • For classifiers using discrete input • Three feature function classes • Agent-oriented features • Environment-oriented features • Team-oriented features • Calculated over a sliding window in time • Features are matched with the ways humans would understand the scene

26. Agent-Oriented Feature Functions • Focus on individual agents • Enhance performance of teamwork recognition • Used to recognize (likely) roles

27. Curvature • The rate at which a curve changes direction Finite difference approximation with central difference

28. Obstacle Angle Orientation A1 LOS Path t=2 t=1 t=0 Environment-Oriented Feature Functions • Agent and team interactions with environment • Environmental objects • Physical or virtual • Static or dynamic • Domain specific examples • Frontline in war • Offside line in soccer • Line-of-scrimmage in football

29. Team-Oriented Feature Functions • Extract features relative to the team • Specifically designed for teamwork activity recognition

30. CRPV Discretization Centroid-Relative Position Vector (CRPV) • Positions are calculated relative the centroid position and orientation • Dimensionality is reduced (compared to previous approach) • Translation, rotation and scale invariant • An evolution is the Role-Relative Position Vector (RRPV) • Privileged agent

31. Cohesion • Measures the bonding together of the team • Derived from the Principal Component Analysis • PCA: Dimensionality is reduced by restricting attention to the directions along the scatter cloud that are the greatest • Calculate eigenvalues and eigenvectors from the position-based scatter matrix • In the 2D case there are two eigenvalues and two eigenvectors • Cohesion is the maximum eigenvalue • CohesionDirection is the direction of the eigenvector with maximum eigenvalue • CohesionGradient is the change in cohesion over a sliding window

32. Cohesion • Position-based scatter matrix

33. Team-Oriented Feature Functions • Agent-oriented features can be used by replacing the team with a virtual agent following the team centroid

34. Outline • Introduction • Data Acquisition and Knowledge Engineering • Teamwork Activity Recognition using Hidden Markov Models • Team-Oriented Feature Extraction • Role-Based Teamwork Activity Recognition • Tuning the Performance of Teamwork Activity Recognition • Conclusions

35. Role-Based Teamwork Activity Recognition • The goal is to improve robustness of the teamwork recognizer • What are the role assignments in the team? • The baseline HMM input we presented previously is a fixed arrangement • {1, 2, 3, 4} not the same as {2, 3, 4, 1} • Brute force solution • Inefficient: One HMM for each permutation • Role recognition module • Represent teamwork activity based on roles, rather than by agents

36. Teamwork Scenario Editor Teamwork Activity Recognition Framework Teamwork Activity Models Role Models Role Recognition Teamwork Data Format Import Learning Export Visualization Identification Role-Oriented Feature Extraction Recognition Representative Examples Assignment Team-Oriented Feature Extraction External database Classification Mapping Real-time data stream Annotated Teamwork Activity Extended Workflow Overview

37. Learning Role Models • Role models are represented by decision trees • Trained from observations with ID3 algorithm • Pruned to minimize effects of overfitting • Intuitive (white-box) • Visualizes the exact features which were used in classification • Follow leader uses CRPV • Feature functions for role recognition: • Agent-oriented feature functions • CRPV feature (assuming that bystander agents are filtered out)

38. NorthWest SouthWest f={2, 4, 0, 0} f={0, 0, 25, 0} SouthEast NorthEast f={0, 0, 17, 0} f={4, 6, 0, 0} South North f={0, 0, 277, 319} Velocity Low High f={66, 113, 0, 0} Acceleration CRPV Constant Accelerating Decelerating f={107, 80, 0, 0} f={34, 7, 0, 0} f={106, 109, 0, 0} Role Recognition • Calculate the role assignment probability Pr( ai , rj ) • The probability that agent aiplays role rj • Extract observation sequences from the movement trace of each agent • Input each observation in the sequence to the decision tree classifier • The output is a sequence of class frequency vectors • Role assignment probability:

39. Role Assignment and Mapping • Identify the best match of role to agent assignments by searching the role assignment probabilities • Re-map the team-oriented feature vectors • Multiple role assignment • Each agent can play multiple roles

40. Role Assignment and Mapping • Unique role assignment • Each agent can play one role

41. Experimental Setup • Warfare exercise dataset • Same as used previously • Extended with seven activities • Four agents and four roles • Teamwork Activity Recognizer • Gaussian Mixture Model (GMM) • Multiple internal HMMs for each activity to accommodate for complexity variations in teamwork activities • GPS readings are not always available (e.g. positions from opponents) • Simulate noisy observations • Offset position with a randomly generated number following the Gaussian distribution multiplied with a noise magnitude • Mean accuracy and standard deviation was calculated using stratified 10-fold cross-validation

42. Results: Creating the Idealized Team Actions • Assume perfect role recognition • Best parameter configuration • Four hidden states • Three mixture components

43. Results: Performance Evaluation with Unknown Team-Organization • Shuffled input

44. Results: Performance Evaluation with Unknown Team-Organization • Accuracy with noise and unknown team-organization • 92.62% with standard deviation 5.53% • Previous accuracy was 82% (without noise) • Role recognition improves robustness • Team-oriented feature functions improves accuracy

45. Outline • Introduction • Data Acquisition and Knowledge Engineering • Teamwork Activity Recognition using Hidden Markov Models • Team-Oriented Feature Extraction • Role-Based Teamwork Activity Recognition • Tuning the Performance of Teamwork Activity Recognition • Conclusions

46. Tuning the Performance of Teamwork Activity Recognition • Many components in the recognizer’s workflow are customizable • HMM: Number of hidden states and choice of emission probability representation • Feature extraction: Width of sliding window and discretization thresholds • Role recognition module: Choice of features, parameters in the ID3 learning algorithm and the choice of learning algorithm • The probability density function (PDF) has a significant impact on the accuracy and robustness of the recognizer

47. Simple Hidden Markov Model • Discrete (histogram-based) probability density function • Inputs are sequences of discrete values • Vector quantization • Compress observations • Training of a codebook • Cluster assignment • Advantages • Multi-modal distribution • Disadvantages • Extra cluster parameter to train codebook • Can not estimate unseen observations • Requires more training data

48. Hidden Markov Model with Gaussian PDF • Used in the initial recognizer • Advantages • Continuous • Can estimate unseen observations • Disadvantages • Unimodal distribution • Example: How to model U-turn to the left and right in the same HMM? • Matrix inversion of covariance matrix can be problematic • Reduce expressiveness by enforcing diagonal covariance matrices

49. Hidden Markov Model with GMM PDF • Used in the extended recognizer • Advantages • Continuous • Can estimate unseen observations • Multi-modal distribution • Disadvantages • Extra parameter to determine number of mixture components • Parameter estimation is computationally expensive

50. Best recognition accuracy 60 clusters with 18 hidden states Accuracy is 75.05% ± 14.89% Difficulties with Bounding overwatch Team split Team merge Results: Simple HMM