Towards Verified Agents using Sequential Analysis

1. Thinking Metal Towards Verified Agents using Sequential Analysis Lake Arrowhead April, 2007 James Girard Thinking Metal LLC JPGirard@thinkingmetal.com 505 983 6333

2. “You can observe a lot just by watching” – Yogi Berra • Contract: “..produce a verification of the observed video scenarios.” • Completely empirically “grounded”/qualitative • Start without preconceptions • Data first • Not perfect, just more data driven than an ad-hoc approach • Glaser & Strauss, Grounded Theory 1967

3. Data of similar events (videos, audios, etc.) Define ME&E behavior sets Code times and actions Choose individual agent model Implement behavior set Accuracy statistic Data First‘Grounded’ Approach Sequential data Stats analysis, find dependencies MODEL “rules”

4. Specifics of our problem • Physical crowd (observable) • Protesting • Non-lethal weapon analysis • Original model • 42 parameters • ~ad-hoc • Unknown distributions/dynamics

5. ‘Grounded’ ME&E States/Events definition • Qualitative • Behaviors should be obvious to multiple observers • Agree on behavior set • Do not guess subject’s thoughts • Result: • Mutually exclusive, exhaustive sets of behaviors • >= 1 class of these • Plus events

6. Behavior Classes Example • Locomotion: stand, run, walk, jog, hiding, evade • Direction: still, towards, away, sideways, diagonals • Arms: Up, down, protect, object play • Events: jump, spin, duck, pickup, throw, exasperated, point • Location: front, middle, back

7. Generating Sequential Data From video: * Noldus IT

8. Sequential Data Example (subject A, locomotion class) ‘throw’ event

9. Possible Models for Agents – add Quantitative • Several approaches – similar – distributions and dependencies • Markovian • Bayesian • Stimulus / response • Find minimal version which provides a good fit. Data driven. • “grounded” • Existence proofs for limited scenarios • Cohn & Tronick, Mother-Infant Interaction 1987 • Bishop, Fienberg and Holland Discrete Multivariate Analysis • Not guaranteed • ABMs : err on the side of simplicity – low cognition Bakeman & Gottman, Introduction to Sequential Analysis

10. Choose many random t’s Stimulus window Response window Stimulus/Response t X2 test for independence (P<0.01) Prob(response occurs|stimulus) from this table also

11. Stim/Response Contingency Details • Self and others (interactions) as stimuli • Analyze states and transitions as stimuli • Aggregate across subjects within agent types

12. Chi^2 output (Test for independence) • Pvalue 0.00059 dof 1 chi2 stat 11.78 for stimulus self state pickup against response run • Pvalue 0.0031 dof 1 chi2 stat 8.69 for stimulus self transition run against response throw • Pvalue 0.0092 dof 1 chi2 stat 6.81 for stimulus other throw against response throw

13. As far as the data will allow Independent behavior simple dependent behavior (single) multiple state dependencies complex

14. Rule Syntax • Independent rules (time spent in each behavior, per class) • <agentType> I <class> <state1> <prob1> <meanTime1> <stddevTime1> <state2> <prob2> <meanTime2> <stddevTime2> • Dependent rules. P(response | stimuli) • <agentType> D <ruleName> <stimWindow> <responseWindow> <responseState> <responseProb> <stimulusCondition1> <state|transition|other> <stimulusCondition2> <state|transition|other>

15. Classes on top of RePast for RuleProcessing • The reverse of the stats analysis • Separate self/others rules • Handle dependencies based on states and transitions • Within a time window • Probability action occurs • Some classes with descriptions • Behavior : a particular behavior • Condition : state or transition, other or self • DependentRule: set of conditions. Window. Probability action occurs

16. Agent Rule processing Loop Do forever { UpdateRules(current behaviors); If (rule fires) { t = uniformRandom(window) + currentTime add action to UpcomingActions for time t If (time of action in UpcomingActions) > currentTime { remove action from UpcomingActions Add action to current behaviors Begin action } }

17. Data of similar events (videos, audios, etc.) Define ME&E behavior sets Code times and actions Choose individual agent model Accuracy statistic The Approach Sequential data behavior set rule proc.* “rules” Stats analysis, find dependencies * Model environment * generic

18. implementations

19. GazeFOV • walkSpeed • runSpeed • crowdSize • maxSightDistance • initialDistance • NLWlocation • NLWmeanTargetDistance • Other NLW characteristics • 42 Total Parameters • -> now 6 Total Parameters Previous Variable Set • AttackProbability • AttackDistanceMeanFraction • DesireThreshold • DesireChangeObservedMean • DesireChangeObservedStanDev • DesireChangeSelfMean • DesireChangeSelfStanDev • PainDecay • PainThresholdMean • PainThresholdStanDev • RetreatTimeMean • ThrowDistanceRatio • WaitTimeRetreatMean • WaitTimeReattackMean Only variables which remain can be “researched”!

20. Current issues • Serial actions – intent – need sufficient behavioral codes • Increase complexity of model as data warrants • Dependencies on multiple stimuli • Dependencies on previous states • Search for proper window size for each dependency. • Remove extraneous dependencies with G2 test. • A->B->C will show A->C • A->B, A->C will show B->C • Stationarity • Need to recode: location resolution; locomotion+direction

21. Abstracting the problem • Pretty general approach • Need: • Constrained sets of behaviors • Observable behaviors • Endogenous interactions: Assume single agent behavior is dependent on current and previous behaviors and observations of those in others • Empirical sequential data or raw data source (ie: video, transcriptions, action data, trade data, etc.)

22. Thanks to…. • Robert Holmes (Qforma) • Frank Wimberly (Redfish) • George Duncan (CMU) • Jason Bruns (Conceptual Mindworks) • US Air Force HEDR • US Joint NLW Directorate

23. Summary • Simplistic models for sequential data exists – good fit for ABMs • Qualitative -> then quantitative • Data first / data guided • Generic tools and classes for Stimulus/Response agent model • Not perfect – one step more validated • One more possible tool/technique