CS 416 Artificial Intelligence

1. CS 416Artificial Intelligence Lecture 2 Agents

2. Review • We’ll study systems that act rationally • They need not necessarily “think” or act like humans • They need not “think” in rational ways • The domain of AI research changes over time • AI research draws from many fields • Philosophy, psychology, neuroscience, mathematics, economics, mechanical, linguistics • AI has had ups and downs since 1950

3. What is an agent? • Perception • Sensors receive input from environment • Keyboard clicks • Camera data • Bump sensor • Action • Actuators impact the environment • Move a robotic arm • Generate output for computer display

4. Perception • Percept • Perceptual inputs at an instant • May include perception of internal state • Percept Sequence • Complete history of all prior percepts • Do you need a percept sequence to play Chess?

5. An agent as a function • Agent maps percept sequence to action • Agent: • Set of all inputs known as state space • Repeating loop: • We must construct f( ), our agent • It must act rationally

6. The agent’s environment • What is known about percepts? • Quantity, range, certainty… • If percepts are finite, could a table store mapping? • What is known about environment? • Is f (a, e) a known function and predictable? • More on this later

7. Evaluating agent programs • We agree on what an agent must do • Can we evaluate its quality? • Performance Metrics • Very Important • Frequently the hardest part of the research problem • Design these to suit what you really want to happen

8. Performance vis-à-vis rationality • For each percept sequence, a rational agent should select an action that maximizes its performance measure • Example: autonomous vacuum cleaner • What is the performance measure? • Penalty for eating the cat? How much? • Penalty for missing a spot? • Reward for speed? • Reward for conserving power?

9. Learning and Autonomy • Learning • To update the agent function, ,in light of observed performance of percept-sequence to action pairs • Does the agent control observations? • What parts of state space to explore? • Learn from trial and error • How do observations affect agent function? • Change internal variables that influence action selection

10. Adding intelligence to agent function • At design time • Some agents are designed with clear procedure to improve performance over time. Really the engineer’s intelligence. • Camera-based user identification • At run-time • Agent executes complicated equation to map input to output • Between trials • With experience, agent changes its program (parameters)

11. How big is your percept? • Dung Beetle • Almost no perception (percept) • Rational agents fine-tune actions based on feedback • Sphex Wasp • Has percepts, but lacks percept sequence • Rational agents change plans entirely when fine tuning fails • A Dog • Equipped with percepts and percept sequences • Reacts to environment and can significantly alter behavior

12. Qualities of a task environment • Fully Observable • Agent need not store any aspects of state • The Brady Bunch as intelligent agents (lost in Hawaii) • Volume of observables may be overwhelming • Partially Observable • Some data is unavailable • Maze • Noisy sensors

13. Qualities of a task environment • Deterministic • Always the same outcome for environment/action pair • Stochastic • Not always predictable – random • Partially Observable vs. Stochastic • My cats think the world is stochastic (lack of perception) • Physicists think the world is deterministic

14. Qualities of a task environment • Markovian • Future environment depends only on current environment and action • Episodic • Percept sequence can be segmented into independent temporal categories • Behavior at traffic light independent of previous traffic • Sequential • Current decision could affect all future decisions • Which is easiest to program?

15. Qualities of a task environment • Static • Environment doesn’t change over time • Crossword puzzle • Dynamic • Environment changes over time • Driving a car • Semi-dynamic • Environment is static, but performance metrics are dynamic • Drag racing (reward for reaching finish line after 12 seconds is different from reward for reaching it after 14 seconds)

16. Qualities of a task environment • Discrete • Values of a state space feature (dimension) are constrained to distinct values from a finite set • Blackjack: • Continuous • Variable has infinite variation • Antilock brakes: • Are computers really continuous?

17. Qualities of a task environment • Towards a terse description of problem domains • Environment: features, dimensionality, degrees of freedom • Observable? • Predictable? • Dynamic? • Continuous? • Performance metric

18. Building Agent Programs • The table approach • Build a table mapping states to actions • Chess has 10150 entries (1080 atoms in the universe) • I’ve said memory is free, but keep it within the confines of the boundable universe • Still, tables have their place • Discuss four agent program principles

19. Simple Reflex Agents • Sense environment • Match sensations with rules in database • Rule prescribes an action • Reflexes can be bad • Don’t put your hands down when falling backwards! • Inaccurate information • Misperception can trigger reflex when inappropriate • But rules databases can be made large and complex

20. Simple Reflex Agents w/ Incomplete Sensing • How can you react to things you cannot see? • Vacuum cleaning the room w/o any sensors • Vacuum cleaning room w/ bump sensor • Vacuum cleaning room w/ GPS and perfect map of static environment

21. Model-based Reflex Agents • So when you can’t see something, you model it! • Create an internal variable to store your expectation of variables you can’t observe • If I throw a ball to you and it falls short, do I know why? • I don’t really know why… • Aerodynamics, mass, my energy levels… • I do have a model • Ball falls short, throw harder

22. Model-based Reflex Agents • Admit it, you can’t see and understand everything • Models are very important! • We all use models to get through our lives • Psychologists have many names for these context-sensitive models • Agents need models too

23. Goal-based Agents • Overall goal is known, but lacking moment-to-moment performance measure • Don’t exactly know what performance maximizing action is at each step • Example: • How to get from A to B? • Current actions have future consequences • Search and Planning are used to explore paths through state space from A to B

24. Utility-based Agents • Goal-directed agents that have a utility function • Function that maps internal and external states into a scalar • A scalar is a number used to make moment-to-moment evaluations of candidate actions

25. Learning Agents • Desirable to build a system that “figures it out” • Generalizable • Compensates for absence of designer knowledge • Reusable • Learning by example isn’t easy to accomplish • What exercises do you do to learn? • What outcomes do you observe? • What inputs to your alter?

26. Learning Agents • Performance Element • Selecting actions (this is the “agent” we’ve been discussing) • Problem Generator • Provides suggestions for new tasks to explore state space • Critic • Provides learning element with feedback about progress (are we doing good things or should we try something else?) • Learning Element • Making improvements (how is agent changed based on experience)

27. A taxi driver • Performance Element • Knowledge of how to drive in traffic • Problem Generator • Proposes new routes to try to hopefully improve driving skills • Critic • Observes tips from customers and horn honking from other cars • Learning Element • Relates low tips to actions that may be the cause

28. Review • Outlined families of AI problems and solutions • I consider AI to be a problem of searching • Countless things differentiate search problems • Number of percepts, number of actions, amount of a priori knowledge, predictability of world… • Textbook is divided into sections based on these differences

29. Sections of book • Problem solving: Searching through predictable, discrete environments • Knowledge and Reasoning: Searching when a model of the world is known • a leads to b and b leads to c… so go to a to reach c • Planning: Refining search techniques to take advantage of domain knowledge • Uncertainty: Using statistics and observations to collect knowledge • Learning: Using observations to understand the way the world works and to act rationally within it