1 / 72

Compressing Mental Model Spaces and Modeling Human Strategic Intent

Compressing Mental Model Spaces and Modeling Human Strategic Intent. Prashant Doshi University of Georgia, USA. http://thinc.cs.uga.edu. Yifeng Zeng Reader, Teesside Univ. Previously: Assoc Prof., Aalborg Univ. Yingke Chen Doctoral student. Muthu Chandrasekaran Doctoral student.

bianca
Download Presentation

Compressing Mental Model Spaces and Modeling Human Strategic Intent

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Compressing Mental Model Spaces and Modeling Human Strategic Intent

  2. Prashant Doshi University of Georgia, USA

  3. http://thinc.cs.uga.edu

  4. Yifeng Zeng Reader, Teesside Univ. Previously: Assoc Prof., Aalborg Univ. Yingke Chen Doctoral student Muthu Chandrasekaran Doctoral student Hua Mao Doctoral student

  5. What is a mental behavioral model?

  6. How large is the behavioral model space?

  7. How large is the behavioral model space? General definition A mapping from the agent’s history of observations to its actions

  8. How large is the behavioral model space? 2H (Aj) Uncountably infinite

  9. How large is the behavioral model space? Let’s assume computable models Countable A very large portion of the model space is not computable!

  10. Daniel Dennett Philosopher and Cognitive Scientist Intentional stance Ascribe beliefs, preferences and intent to explain others’ actions (analogous to theory of mind - ToM)

  11. Organize the mental models Intentional models Subintentional models

  12. Organize the mental models Intentional models E.g., POMDP  =  bj, Aj, Tj, j, Oj, Rj, OCj BDI, ToM Subintentional models Frame (may give rise to recursive modeling)

  13. Organize the mental models Intentional models E.g., POMDP  =  bj, Aj, Tj, j, Oj, Rj, OCj BDI, ToM Subintentional models E.g., (Aj), finite state controller, plan Frame

  14. Finite model space grows as the interaction progresses

  15. Growth in the model space Other agent may receive any one of |j| observations |Mj|  |Mj||j| |Mj||j|2  ...  |Mj||j|t 0 1 2 t

  16. Growth in the model space Exponential

  17. Absolute continuity condition (ACC)

  18. ACC Subjective distribution over histories True distribution over histories

  19. ACC is a sufficient and necessary condition for Bayesian update of belief over models

  20. How do we satisfy ACC? Cautious beliefs (full prior) Grain of truth assumption Prior with a grain of truth is sufficient but not necessary

  21. General model space is large and grows exponentially as the interaction progresses

  22. It would be great if we can compress this space! • No loss in value to the modeler • Flexible loss in value for greater compression Lossless Lossy

  23. Expansive usefulness of model space compression to many areas: • Sequential decision making (dt-planning) in multiagent settings • Bayesian plan recognition • Games of imperfect information

  24. Interactive POMDP framework (Gmytrasiewicz&Doshi05) 1. Sequential decision making in multiagent settings Include models of the other agent in the state space Update beliefs over the physical state and models

  25. General and domain-independent approach for compression Establish equivalence relations that partition the model space and retain representative models from each equivalence class

  26. Approach #1: Behavioral equivalence (Rathanasabapathy et al.06,Pynadath&Marsella07) • Intentional models whose complete solutions • are identical are considered equivalent

  27. Approach #1: Behavioral equivalence Behaviorally minimal set of models

  28. Approach #1: Behavioral equivalence Lossless Works when intentional models have differing frames

  29. Approach #1: Behavioral equivalence Impact on dt-planning in multiagent settings Multiagent tiger Multiagent MM Multiagent tiger

  30. Approach #1: Behavioral equivalence Utilize model solutions (policy trees) for mitigating model growth Model reps that are not BE may become BE next step onwards • Preemptively identify such models and do not update all of them

  31. Approach #2: -Behavioral equivalence (Zeng et al.11,12) Redefine BE

  32. Approach #2: RevisitBE (Zeng et al.11,12) Intentional models whose partial depth-d solutions are identical and vectors of updated beliefs at the leaves of the partial trees are identical are considered equivalent Lossless if frames are identical Sufficient but not necessary

  33. Approach #2: (,d)-Behavioral equivalence • Two models are (,d)-BE if their partial depth-d solutions are identical and vectors of updated beliefs at the leaves of the partial trees differ by  Models are (0.33,1)-BE Lossy

  34. Approach #2: -Behavioral equivalence • Lemma (Boyen&Koller98): KL divergence betweentwo distributions in a discrete Markov stochastic process reduces or remains the same after a transition, with the mixing rate acting as a discount factor • Mixing raterepresents the minimal amount by which the posterior distributions agree with each other after one transition • Property of a problem and may be pre-computed

  35. Approach #2: -Behavioral equivalence Given the mixing rate and a bound, , on the divergence between two belief vectors, lemma allows computing the depth, d, at which the bound is reached Compare two solutions up to depth d for equality

  36. Approach #2: -Behavioral equivalence Impact on dt-planning in multiagent settings Discount factor F = 0.5 Multiagent Concert On a UAV reconnaissance problem in a 5x5 grid, allows the solution to scale to a 10 step look ahead in 20 minutes Multiagent Concert

  37. Approach #2: -Behavioral equivalence What is the value of d when some problems exhibit F with a value of 0 or 1? F=1 implies that the KL divergence is 0 after one step: Set d = 1 • F=0 implies that the KL divergence does not reduce: Arbitrarily set d to the horizon

  38. Approach #3: Action equivalence (Zeng et al.09,12) Intentional or subintentional models whose predictions at time step t (action distributions) are identical are considered equivalent at t

  39. Approach #3: Action equivalence

  40. Approach #3: Action equivalence Lossy Works when intentional models have differing frames

  41. Approach #3: Action equivalence Impact on dt-planning in multiagent settings AE bounds the model space at each time step to the number of distinct actions Multiagent tiger

  42. Approach #4: Influence equivalence (related to Witwicki&Durfee11) Intentional or subintentional models whose predictions at time step tinfluence the subject agent’s plan identically are considered equivalent at t Regardless of whether the other agent opened the left or right door, the tiger resets thereby affecting the agent’s plan identically

  43. Approach #4: Influence equivalence Influence may be measured as the change in the subject agent’s belief due to the action Group more models at time step t compared to AE Lossy

  44. Compression due to approximate equivalence may violate ACC Regain ACCby appending a covering model to the compressed set of representatives

  45. Open questions

  46. N > 2 agents Under what conditions could equivalent models belonging to different agents be grouped together into an equivalence class?

  47. Can we avoid solving models by using heuristics for identifying approximately equivalent models?

  48. Modeling Strategic Human Intent

  49. Adam Goodie Professor of Psychology, UGA Yifeng Zeng Reader, Teesside Univ. Previously: Assoc Prof., Aalborg Univ. Yingke Chen Doctoral student Xia Qu Doctoral student Roi Ceren Doctoral student Muthu Chandrasekaran Doctoral student Matthew Meisel Doctoral student Hua Mao Doctoral student

  50. Computational modeling of probability judgment in stochastic games Computational modeling of human recursive thinking in sequential games

More Related