Beyond Behavior

1. An Introduction to Knowledge Representation Beyond Behavior Damián Isla, Naimad Games Peter Gorniak, Rockstar

2. Knowledge Representation We spend a lot of time on what our AIs do but very little time on what they know One of the great neglected problems of [game] AI This talk: • Introduce techniques • Agitate

3. “Behavioral” Knowledge? • Behavioral knowledge • When to run away, when to shoot, when to flank left, etc. • Does an ant “know” where the anthill is? • State Knowledge • What is true about the world

4. The Point of KR Perception of a thing != the thing itself

5. The Point of KR Agent Object 1 Pathfinding Behavior Object 2 Animation Object 3

6. The Point of KR Agent Object 1 KR Pathfinding Behavior Object 2 Animation Object 3 Decisions about action Decisions about perception and interpretation

7. Why is KR Interesting? • Fun • exploit mistakes / limited perception • new modes of interaction • Lifelike • reason about AI as thinking perceiving creatures • emotional reactions • We’re doing it already anyway • Search for better representations == Search for more expressive power • build behavior out of better primitives

8. Timescales “Guy X is behind the crate” “I have three bullets left” “That car is coming towards me” “dogs are animals” “birds have wings” “pushing the button calls the elevator” ∞ “Bobby is 5 years old” “Jane is spending the semester in France.” This instant

9. 3 Key Concepts • Confidence • How sure am I in the knowledge I have? • Salience • How important is the sensory data I’m getting? • Prediction • What do believe will happen given what I’ve seen and what I know?

10. Demo

11. Behavior update void s_agent::behavior_update() { if (!confused()) { s_pos2d pos; omap.get_target_position(&pos); move_to(pos); } }

12. Behavior update void s_agent::behavior_update() { if (!confused()) { s_pos2d pos; omap.get_target_position(&pos); move_to(pos); } } +

13. Expectation-related Emotions • Confusion • Surprise <Something I was confident in is confirmed FALSE> <Something I thought unlikely is confirmed TRUE>

14. Expectation-related Emotions • Confusion • Surprise

15. Target Lists

16. Target Lists Agent Object 1 KR Pathfinding Behavior Object 2 Animation Object 3

17. Target Lists Agent Object 1 Target 1 Pathfinding Behavior Object 2 Target 2 Animation Object 3 Target 3

18. Target Lists Target Perceived data location (x,y,z) action shoot hitpoints 44 Derived data Threat 0.8 Target weight 0.9 “Intentions” hurt_me Allows AI to make mistakes 0.99 0.99 0.99 0.8 0.95 0.98 0.6 0.9 0.98 Shared computation + expressive power

19. shoot_at_target shoot_at_target Example switch_to_knife switch_to_knife search_for_target search_for_target !

20. Phil Carlisle Memorial Memory Slide

21. PCMMS Volatile behavior state • Working memory • Short-term • Episodic • ??? Remember that Target Perceived data location (x,y,z) Target Perceived data location

22. Challenge #1:Representational Versatility Solution: Polymorphism Nazi Truck Wheel Fence Grass My hand

23. Polymorphism Percept DAG (Synthetic Characters, MIT Media Lab, circa 2002)

24. Challenge #2: Performance

25. Challenge #2: Performance Object 1 KR Agent Agent Object 2 Agent Object 3 Shared KR

26. Challenge #2: Performance X:location: <x,y,x> crates Object 1 Agent KR KR Agent Object 2 KR Agent Object 3 X:weapon: “pistol” KR enemies Hybrid KR

27. Challenge #2: Performance Enemy 1 Agent Enemy 2 O A Agent Grass Salience Threshold O x A

28. Benefits of Target Lists • Reasonable mistakes / limited perception • Shared computation • Expressive power

29. Limitations of Target Lists • Relational information • Where does the notion of “behind” live? • Wholes and parts • Does a car’s wheel deserve it’s own representation? • A guy’s arm? • What about a mob of guys?

30. Head Enemy has-a next-to Arm Enemy next-to Arm Representational Wankery behind holding Car has-a Hood Gun Wheel Wheel Wheel Wheel

31. Wild Speculation Lazy Representation??? • Perception is active • Behavioral / emotional / motivational state changes the way you see the world • And WHAT you see in the world.