Agent Behaviour and Knowledge Representation

1. Agent Behaviour andKnowledge Representation K. V. S. Prasad Dept. Of Computer Science Chalmers Univ 23 March 2012

2. The twobranches of TCS • Semantics • Whatdo you want the system to do? • Specify a sort • Howdo you know (what) it does? • Build the right system; build the system right • Specification (syntheticbiologyexample) • Testing • Complexity • Howmuchtime/place • Superfast? Program a sort by shuffling

3. Wherewe are • You havealreadyseen AI in action • Search, Natural Language, BayesianInference • Hard problems, butalreadyclearlyposed • That is, mostlyefficiency (central to AI) • Whataboutsemantics in AI? Specify humans?

4. Semantics for AI • Today: what problems dowewant to solve? • Evolve the language(s) in which to pose the problem • Chapter 2: Intelligent Agents • Chapter 7: Logical Agents • Chapter 8: First-OrderLogic • Chapter 9: Inference in First-OrderLogic • Chapter 12: Knowledge Representation • Harder to get to grips with • But ”softer” than the hard problems above • Oftentidying up earlier work, seeingcommonalities, …

5. The meaning of ”meaning” • Whatdoes ”a tree” mean? • Point to a tree, ”ett träd”, ”grown up plant”, … • Onlyworksif you alreadyknowwhat a tree is, or ”träd”, or ”plant” • Semanticbootstrapping! (Babies are smart!)

6. Whatdoes a treemean? • Food, shade, wood, paper, medicine, wildlife, ecology, … • Thesemeanings are in ourheads • The treeitself is all of these and more • The University is split intodepartments, not the world • So wechoosehow to look at the tree • So, is anymodel at all OK? • No, it must be right at least in itsown terms • Wefindout the hardwayifwetake a narrowview • Thenwehave to change the model

7. Models and Reality • Natural Sciences modelreality • Different models for different purposesuse different abstractions(physics, chemistry,biology, economics…) • Look for predictive and explanatory power • Naturallanguageitself is a model of the world • Eerilygood: ”Ifwedon’thurry, we’ll miss the train” • Howdid it get that good? Evolution • Artificialworlds (games) • Measure of the model • Not correspondence to otherreality: there is none • Instead: Howinteresting this world is • Abstractionsusuallyobvious: plastic = woodchesspieces

8. Book sees AI = Science of agent design • So define agents • How to specify performance • Describe environment • Classify environments • Basic design skeletons for agents • Learning (where it fits in) • All very high level, lots of definitions • So goal: to get you into the book • Chapter 2 is the framework • boring but needed, like the “how to use” of dictionary

9. Chapter 2: intelligent agents Notes to textbook and Russell’sslides

10. What are agents? • AI used to look only at agent components • Theorem provers, vision systems, etc. • Robots are agents • So are controllers in control theory • Agents have sensors, actuators and programs • Ordinary (reactive) programs? No, • Limited pre-knowledge of actual situations • Rational – takes best action as it appears at the time, not what actually is best • Autonomy • Learning

11. Are we agents? • Seem to fitdescription! • For whom or what are we agents? • As animals, agents of our genes! (Dawkins) • Higherlevelemergentstructuresallowus to be • Agents of ideology (sacrificialactions) • Agents of society (monogamy, contraception) • Agents of our ”selves” (gratificationseeking)

12. Semantics of Sequential Agents • Agent = Sequentialprogram? • Old view (CSALL liveshere, largely) • Functionalview: Input/output table is the specification • Languageview: recognizers for Chomsky hierarchy • Twomachines are ”the same” iftheyrecognize the same language • Eventhoughonemight be smaller or faster • One canreplace the other • Same spec, different implementations • Spec is oneabstraction, speed/sizeanother

13. ReactiveAgents: some features • Conversation with environment • I don’tknow my second question till you answer my first • So can’t be function from input sequence • Non-determinism (in ad-hoc notation) • a! ((b? e!) Or (c? f!)) willalways accept b • (a! b? e!) Or (a! c? f!) won’talways accept b • The twomachine are ”may-equal” but not ”must-equal”

14. Agents interact with environments

15. Chap 2 from Russell • Russell’sslides

16. Agents and Environment(S4-6, B34-36) • Action candepend on percept history • Tabulate the agent function • Externally, notewhat it does for eachsequence • Infinite table, ifunbounded input sequence • Internally, implemented by (finite) program • The external table of the agent function • Sayswhat the agent actuallydoes • A similaronecansaywhat it shoulddo (specification) • Whatdowewant the agent to do? • Is everything an agent? A bridge? A calculator?

17. Problems with R&N definition • Agent behaviour = f(percept history) • Assumes agent responses are deterministic • If not • percept at n+1 depends on the agent’sresponse at n • So behaviourdepends on percept history intertwined with agent responses • Evenifdeterministic • somehistories make no sense • Agent has moved right, perceptsays ”left” • Possibleifsomeexternal force moved the agent • Asymmetricbetween agent and environment

18. Good behaviour: rationalityB 36-37 • What should the agent do? • Decide by consequences • As decided by change in environment • By change in agent? Sour grapes, etc. • What should we measure? • Amount of dirt cleaned? • Clean, dump cycle can maximise this • Point per clean square per time unit • Same average from wild swings and always mediocre • Distinguish by energy use? • Design spec from point of view of environment • not how you think agent should behave

19. Rationality (B37-38) • What is the rational action? Depends on • What actions the agent can perform • Performance measure (to be maximised) • Agent’s knowledge • Built in • Percept (input) sequence to date • Agent S6 optimises “clean square/time” • If moves incur penalty? • Can clean square become dirty again?

20. Living with limits of rationality(B38-39, S7) • Spec. can’t demand what will turn out best • Unless we are omniscient (crystal balls) • Hence only precept history • So rational agent can only max. expected outcome • But in AI • Look if you can • Learn from what you see (look at results!) • Built-in knowledge = lack of autonomy • Needed at start, but with learning • Can go beyond built-in knowledge • Succeed in many environments • Randomisation can be rational: ethernet

21. Environments(B 40-42, S 8 -11) • The environment is the task (spec.) • To which the agent is the solution • The automated taxi is an open-ended problem • Real life can be simple (robots on assembly lines) • Artificial life can be complex (softbots) • Show me news items I find interesting • Need some NLP, ML, and cope with dynamic world • Many human and artificial agents in this world • Environments can be agents (process calculus)

22. Why classify environments?(S 12 – 18) • We design agents for a class of environments • Environmentsimulator produces many of given class • To test and evaluate agent • Designing for single scenario = unconscious cheating • Classification is often debatable • Assembly robot can see many parts wrong: learn • Game players usually know rules • Interesting to see what they do if they don’t

23. Observability of Environments • Observable? Fully or partially? • Those aspects relevant to performance • Partial because of noise or data unavailable • Is there dirt in other squares? What will other car do? • Observablescan be re-readratherthan stored • Known =/= observable • Know rules of solitaire, but can’t observe cards • New phone – see buttons, but what do they do?

24. Multi-agentEnvironments • Single agent? Multi-agent? • Chess is twoagent, Solitaireis one-agent • Othervehicles on road, but not the road itself • Agents eithercooperate or compete • Eachtriesto maximiseitsperformance • Competing: chess, parking • Cooperation: avoid collision • When is the other thing an agent? • Always, for careful animals (and superstitious humans?) • Another car? The wind? • When its behaviour is best seen as responding to me • Communication with me

25. (Non)deterministicEnvironments(B 43) • Deterministic? • If next state (of environment) depends only on • current state and action of agent • Otherwise stochastic • Unexpected events: tiresblowingout, etc. • incomplete info might makeenvironmentlook stochastic • “stochastic” implies probabilities (eg. chemistry) • Non-deterministic • Like stochastic, but no probabilities • Performance requires success for all outcomes • Examples from process calculus, sorting, 8 queens

26. Episodic vs.Sequential Environments (B 43-44) • Sequential • History matters • Chess is sequential • This action affects subsequent ones (env. state change) • Episodic = actions indep. of previous ones • Spot faulty parts on assembly line (same rules) • Don’t need to think ahead • Soapoperas largelyepisodic

27. Static vsDynamic Environments • Dynamic • Environment change while agent thinks(driving) • Chess is partly so (clockrunsout) • Like real time; no action is a “pass” • Semi-dynamic • no change in environment, but in score

28. (Un)known Environments(B 44) • Agent knows rules? Doesn’t? • i.e., Knows outcomes for all actions • Doesn’t know -> agent has to learn what happens • Known? • Refers to rules. Card face down is not observable, butrulesknown. • New phone? Don’tknowwhatsomebuttons do.

29. ContinuousEnvironments • Discrete vs continuous • Can apply to states, time, percepts and actions

30. The structure of agents • Agent = architecture + program • We don’t consider in our course • physical actuators (robots) • Physical sensors (cameras, vision) • So “architecture = general purpose computer” • So agent = program

31. Agent program skeleton • function AGENT (INPUT) returns ACTION persistent: STATE, STATEFUN, ACTFUN STATE := STATEFUN(STATE, INPUT); return ACTFUN(STATE, INPUT) Here STATE keeps track of history. The two FUNs could be just table lookups, but these tables would be huge or even unbounded. Can we replace the tables by programs?

32. Tables and Programs • Log tables and square root tables (ca 1970) • Replaced by programs in calculator • Not always the best: • Learn your “times tables”, don’t add 4 seven times • Don’t keep re-deriving (a+b)^2 = a^2 +2ab +b^2 • “Facts at fingertips” often sign of a professional • Reflexes faster than thought is more hard vs soft • “Intelligence” = facts, to a shocking extent • See Cyc project (Doug Lenat)

33. Simple Reflex Agents(B48-50, S19 – 21) • Reflex agents from behavioural psychology • Stimulus/response • Functionalism introduced state of agent • Vacuum cleaner (S20, 21) • Depends only on current location and “dirty?” • Small program because no history • Also, if dirty, action indep of location • Also, only works if env. fully observable • i.e. current percept is only info needed • If vacuum cleaner can’t see where it is, … • “Left” fails if already in Left • Try flipping coin to decide LEFT/RIGHT; don’t need to see location • Note LISP code • historically important, John McCarthy 1959, SAIL, etc.

34. Reflex agents with state(B50-52, S22-23) • You see only one page of a book at a time • But you can read because you remember • History can compensate some partial observability • Drivers (and secret service agents) • Can’t see everyone at once, but • I remember where they were when I last looked • Need to predict (guess) where they would be now • “how world evolves”, “effect of actions” in S22 • Hence “model-based” reflex agents

35. Goal-based agents(B52-53, S24) • In reflex agents, current percept selects action • If you have a goal • Easy if one step gets you there • Search and planning if you need sequence of steps • Need to represent what happens if I do “X” • This explicitly represented knowledge can be updated • If it’s raining, update taxi brake info (less efficient) • Reflex agent would need many condition-rule pairs • Easy to change goal, not hard coded as in reflex

36. Utility (B53-54, S25) • Goals are binary – happy, unhappy • Utility more general – how happy is the agent? • For conflicting goals, utility specifies trade-off • Several uncertain goals • weigh chances against importance of goal • Performance grades sequence of env. states • Distinguishes between better and worse routes • Agent has utility function that should match this

37. Learning (B54-57, S26) • Learner may be simpler to build than learned • “Performance element” • takes in percepts, decides actions • Previously was whole agent • Learning element uses feedback from critic • Modifies performance element to do better • Ask “what kind of PE do I need now?” • Percept says “won”, critic (spec) says this is good • Problem generator suggests experiments • Local suboptimal may be optimal in long run

38. Learning taxi example • PE allows cutting across other cars • Critic sees bad effects • LE makes new rule, modifying PE • Problem generator says “try brakes on wet road” • LE modifies “how world evolves” and “effects of my actions” • Given brake pressure, how much deceleration? • Part of input may directly grade agent • (No) tips from (dis)satisfied customers

39. Knowledge and AI • To actintelligently, machinesneed • reasoning and creativity • glamorous • butalso (mostly?) knowledge • boring? • A smart assistant/teacher/expert • Has the facts at their fingertips • Don’thave to be toldthings • Can tell you the relevant facts faster than you canfindthem with Google and Wikipedia • Whydowe still go to the doctor or a lawyer?

40. PropositionalLogic (Chap 7) • From Russell’sslides

41. Counter-culturalcurrents • Cycproject • to teach computer millions of facts • How to structure this knowledge? • E. D. Hirsch’s ”CulturalLiteracy” • Long tradition sayslet student discover • My childrenknownothingabout the Civil War! • No matter, weteachthemhow to findoutaboutanything • So theywilllearnabout it whentheywant to • Buttheymightneverwant to • Thenremainshutout of muchdebate

42. Machines are not human • Human memory is leaky • Storieseasier to rememberthandisjointedfacts • Biologyused to be onedamnfact after another, perhapsgrouped or organised, but no ”how” or ”why” • With evolution, makes sense • Machinememorydoesn’tleak • All structuresequally easy to remember • Choosebased on speed of lookup • Studyingbirds is different from building planes: lesson for AI

43. What is knowledge? • Structuresthroughwhich to filter experience • Is that just a bunch of stripes or is it a tiger? • Evolved to get this right! • What a pretty red! • It’s a traffic light, so stop! • Inborn • Babies smile at twodots on a whitesheet (a face!) • Babies and birdscancount to three • Learned • Put the milk in the fridge or it willspoil • 4 * 2 = 8 (No 4’s or 2’s in nature, but 4 boys and 2 apples; brace, couple, pair, … all mean 2)

44. Stored knowledge • Whylearn the multiplication table? • Lookup faster thanre-computation • Breadthbetterthandepth for fast search • Deep description vs flat nameaddressstyles • 45, 3rd Cross, 2nd Main, Jayanagar IV Block, Bangalore. • 4532D Jayanagar? • Rännvägen 6, Gbg. (Butmostdon’tknowwhere this is) • So you need a description: how to get there • Whydolanguageshave so manywords? • Becausenames are faster thandescriptions • Twowords of titleenough to identifybook

45. Wumpus Axioms in Prop. Logic • KB = Axioms + perceptsentences • Axioms • not P(1,1) and not W(1,1) • B(1,1) iff P(1,2) or P(2,1) … similarly for B(n,m) • S(1,1) = W(1,2) or W(2,1) … similarly forW(n,m) • W(1,1) or W(1,2) or … or W(2,1) or W(2,2) … • There is at leastonewumpus • not (W(1,1) and W(1,2)) … for all pairs; 1 wumpus

46. WumpusPercepts in Prop. Logic • Shouldweadd ”Stench” to KB? • No, depends on when. Else contradiction. • Index all percepts, position, directionby time • L(t,x,y) => (Breeze(t) iffB(x,y)) etc. • Actions? • L(t,x,y) and East(t) and Forward(t) => L(t+1,x+1,y) and not L(t+1,x,y) • For eachpossible t, x and y, and each action • Alsostatewhatremainsunchanged: frame problem

47. What a mess, but … • PredicateLogic gets rid of the separate propositions for eachvalue of t, x and y • Situation Calculus and otherplanninglanguagestakecare of the frame problem • But you cansee that plans can be made by propositionalinference. Prove • Init and action axioms => HaveGold and Out • Programs from proofs!

48. Formal Languages (Chap 8) • Prog. languages (C++, Java, …) are formal. • representcomputation steps directly • Knowledge ad hoc (in data or algorithms) • Facts from otherfacts? Hard in Java, etc. • Partial info hard. Howsay ”W(2,3) or W(3,2)”? • Databases = domainindep. store/read KB • ”Declarative” • KB and inference are separate • Func. Programming: script and eval separate

49. First Order Logic (FOL) • From Russel’sslidesChap 8

50. Alternative Semantics for FOL • ”Richard has twobrothers, John and Geoff”. • B(J,R) and B(G,R) and J=/=G and forall x. B(x,R) => (x=J or x=G) • What a pain! So people make mistakes. • Popular database solution • Uniquenameassumption. • J=/=G assumed, different names. • Closed world assumption • Atomic sentences not statedtrueassumedfalse. • Domainclosure • There are no domain elements otherthanthosenamed