Modeling Reasoning in Strategic Situations

1. Modeling Reasoning in Strategic Situations Avi Pfeffer MURI Review Monday, December 17th, 2007

2. Strategic Situations • Scenarios involving multiple agents, all of which make decisions, and receive rewards based on their decisions • May be competitive, cooperative, or anything in between • May have interesting structure • May be extended over time

3. Strategic Situations Abound • Countering terrorist threats • Robotic soccer • Disaster response • Diplomatic relations • Auctions • Trading

4. A Key Question Can we model how agents reason in strategic situations?

5. Why this Question is Important • Modeling how agents reason will allow us to: • predict their behavior • develop counter-strategies • develop computer systems that help people in their strategic decision making • analyze situations to determine optimal strategies • allow people to explain their reasoning

6. Possible Approaches • Classical game theory • Opponent modeling • Behavioral economics • Psychological theories

7. Our Approach • Identify the basic reasoning patterns that can be used to justify decisions • underlies sophisticated behavior such as sacrificing, retaliation and tempting • Model and learn the factors underlying decision-making in particular games • Use the models to develop strategies that work well

8. Characterizing Reasoning Patterns • Informally, a reasoning pattern is a form of argument that leads to a decision • We characterize reasoning patterns as structures in a graph describing a strategic situation • the reasoning patterns capture the way information is used and manipulated

9. Reasoning Pattern #1: Direct Effect Drill • An agent takes a decision because of its direct effect on its utility • without being mediated by other agents’ actions Profit

10. Reasoning Pattern #2: Manipulation Offer to Read Brush Teeth • Child knows about parent’s action • Parent does not care about reading, but wants child to brush teeth • Child dislikes brushing teeth but likes being read to • Parent can manipulate child Parent Child

11. Reasoning Pattern #3: Signaling • A communicates something that she knows to B, thus influencing B’s behavior Better Restaurant Recommendation Choice Alice Bob

12. Reasoning Pattern #4: Revealing/Denying • Driller cares about oil • Tester receives fee if driller drills • Tester causes driller to find out (or not) about information tester herself does not know Seismic Structure Test Test Result Oil Drill Tester’s Profit Driller’s Profit

13. A Question • For each reasoning pattern, we provide a graphical criterion to determine if the pattern holds • Intuitively, a node is motivated if the agent owning the node cares about its decision If a node is motivated, does the graphical criterion characterizing one of the reasoning patterns necessarily hold?

14. Answering the question • Answer: it depends what strategies we allow for other agents • If we allow arbitrary strategies, any directed path from a decision node to a utility causes the node to be motivated • But if we restrict attention to a “highly justifiable” class of strategies, we get a more interesting answer

15. Well-Distinguishing (WD) Strategies: Intuition • A strategy is well-distinguishing if all distinctions that it makes really make a difference • whenever the strategy distinguishes between two states of parents, the agent should receive different utility in the different states

16. Completeness Result Theorem: If other agents are playing WD strategies, then a node is motivated only if at least one of the reasoning patterns holds • i.e., the four patterns of reasoning are sufficient to characterize all cases in which an agent cares about a decision

17. Relationship with Game Theory Theorem: The set of WD strategies always includes a Nash equilibrium • We can view WD equilibrium as refinement of Nash • Completeness theorem holds for all WD strategies, not just equilibria • different assumption from rationality WD WD equilibria Nash

18. Learning how People Reason • Reasoning patterns give us a theoretical basis for what arguments a person might make • Can we learn what people actually do in particular games? • Can we use what we learn to develop automatic strategies that perform well?

19. Learning How to Negotiate • Can we learn how people trade off factors such as self-interest, altruism, etc. in negotiations? • Yes • we developed a computer agent using learned models • it performed much better than game-theoretic agents, and also better than people

20. Learning Reciprocal Reasoning • Do people use reciprocal reasoning in repeated interactions? • retrospective reasoning • prospective reasoning • Yes • models that factor in reciprocal reasoning perform better than those that don’t • but prospective may not be as important as retrospective

21. Reasoning Under Uncertainty • When people have uncertainty about other players, do they use models of the other players? • Yes • modeling people as reasoning about the potential actions of others leads to better performance • but recursive modeling has diminishing returns

22. Distinguishing Beliefs from Preferences • A person’s behavior may be influenced by both beliefs and preferences • can we distinguish between them? • Yes • we have created models that are uniquely identifiable • in this scenario, people have almost correct beliefs

23. Next Steps • Algorithms and analysis tools for identifying relevant arguments in particular situations • Analyze arguments for key behaviors • recruitment to terrorism • diplomatic relations, e.g. North Korea

24. Questions?