CPSC 322 Introduction to Artificial Intelligence

1. CPSC 322Introduction to Artificial Intelligence December 3, 2004

2. Things... Slides for the last couple of weeks will be up this weekend. Did I mention that you have a final exam at noon on Friday, December 10, in MCML 166? Check the announcements part of the web page occasionally between now and then in case anything important comes up.

3. This should be fun Artificial Intelligence and Interactive Digital Entertainment First Annual Conference June 1-3, 2005 in Marina Del Rey, California www.aiide.org

4. Learning Definition: learning is the adaptive changes that occur in a system which enable that system to perform the same task or similar tasks more efficiently or more effectively over time. This could mean: • The range of behaviors is expanded: the agent can do more • The accuracy on tasks is improved: the agent can do things better • The speed is improved: the agent can do things faster

5. Learning is about choosing the best representation That’s certainly true in a logic-based AI world. Our arch learner started with some internal representation of an arch. As examples were presented, the arch learner modified its internal representation to either make the representation accommodate positive examples (generalization) or exclude negative examples (specialization). There’s really nothing else the learner could modify... the reasoning system is what it is. So any learning problem can be mapped onto one of choosing the best representation...

6. Learning is about search ...but wait, there’s more! By now, you’ve figured out that the arch learner was doing nothing more than searching the space of possible representations, right? So learning, like everything else, boils down to search. If that wasn’t obvious, you probably will want to do a little extra preparation for the final exam....

7. Same problem - different representation The arch learner could have represented the arch concept as a decision tree if we wanted

8. Same problem - different representation The arch learner could have represented the arch concept as a decision tree if we wanted arch

9. Same problem - different representation The arch learner could have represented the arch concept as a decision tree if we wanted do upright blocks support sideways block? no yes not arch arch

10. Same problem - different representation The arch learner could have represented the arch concept as a decision tree if we wanted do upright blocks support sideways block? no yes not arch do upright blocks touch each other? no yes arch not arch

11. Same problem - different representation The arch learner could have represented the arch concept as a decision tree if we wanted do upright blocks support sideways block? no yes not arch do upright blocks touch each other? no yes is the not arch top block either a rectangle or a wedge? no yes not arch arch

12. Other issues with learning by example The learning process requires that there is someone to say which examples are positive and which are negative. This approach must start with a positive example to specialize or generalize from. Learning by example is sensitive to the order in which examples are presented. Learning by example doesn’t work well with noisy, randomly erroneous data.

13. Reinforcement learning Learning operator sequences based on reward or punishment Let’s say you want your robot to learn how to vacuum the living room iRobot Roomba 4210 Discovery Floorvac Robotic Vacuum $249.99 It’s the ideal Christmas gift!

14. Reinforcement learning Learning operator sequences based on reward or punishment Let’s say you want your robot to learn how to vacuum the living room goto livingroom vacuum floor goto trashcan empty bag Good Robot!

15. Reinforcement learning Learning operator sequences based on reward or punishment Let’s say you want your robot to learn how to vacuum the living room goto livingroom goto trashcan vacuum floor vacuum floor goto trashcan goto livingroom empty bag empty bag Good Robot! Bad Robot!

16. Reinforcement learning Learning operator sequences based on reward or punishment Let’s say you want your robot to learn how to vacuum the living room goto livingroom goto trashcan vacuum floor vacuum floor goto trashcan goto livingroom empty bag empty bag Good Robot! Bad Robot! (actually, there are no bad robots, there is only bad behavior...and Roomba can’t really empty its own bag)

17. Reinforcement learning Should the robot learn from success? How does it figure out which part of the sequence of actions is right (credit assignment problem)? goto livingroom goto trashcan vacuum floor vacuum floor goto trashcan goto livingroom empty bag empty bag Good Robot! Bad Robot!

18. Reinforcement learning Should the robot learn from failure? How does it figure out which part of the sequence of actions is wrong (blame assignment problem)? goto livingroom goto trashcan vacuum floor vacuum floor goto trashcan goto livingroom empty bag empty bag Good Robot! Bad Robot!

19. Reinforcement learning However you answer those questions, the learning task again boils down to the search for the best representation. Here’s another type of learning that searches for the best representation, but the representation is very different from what you’ve seen so far.

20. Learning in neural networks The perceptron is one of the earliest neural network models, dating to the early 1960s. weights w1 x1 w2 w3 x2 S x3 . . . sum threshold xn wn inputs

21. Learning in neural networks The perceptron can’t compute everything, but what it can compute it can learn to compute. Here’s how it works. Inputs are 1 or 0. Weights are reals (-n to +n). Each input is multiplied by its corresponding weight. If the sum of the products is greater than the threshold, then the perceptron outputs 1, otherwise the output is 0. weights w1 x1 w2 w3 x2 S x3 . . . sum threshold xn wn inputs

22. Learning in neural networks The perceptron can’t compute everything, but what it can compute it can learn to compute. Here’s how it works. The output, 1 or 0, is a guess or prediction about the input: does it fall into the desired classification (output = 1) or not (output = 0)? weights w1 x1 w2 w3 x2 S x3 . . . sum threshold xn wn inputs

23. Learning in neural networks That’s it? Big deal. No, there’s more to it.... weights w1 x1 w2 w3 x2 S x3 . . . sum threshold xn wn inputs

24. Learning in neural networks That’s it? Big deal. No, there’s more to it.... Say you wanted your perceptron to classify arches. weights w1 x1 w2 w3 x2 S x3 . . . sum threshold xn wn inputs

25. Learning in neural networks That’s it? Big deal. No, there’s more to it.... Say you wanted your perceptron to classify arches. That is, you present inputs representing an arch, and the output should be 1. weights w1 x1 w2 w3 x2 S x3 . . . sum threshold xn wn inputs

26. Learning in neural networks That’s it? Big deal. No, there’s more to it.... Say you wanted your perceptron to classify arches. That is, you present inputs representing an arch, and the output should be 1. You present inputs not representing an arch, and the output should be 0. weights w1 x1 w2 w3 x2 S x3 . . . sum threshold xn wn inputs

27. Learning in neural networks That’s it? Big deal. No, there’s more to it.... Say you wanted your perceptron to classify arches. That is, you present inputs representing an arch, and the output should be 1. You present inputs not representing an arch, and the output should be 0. If your perceptron does that correctly for all inputs, it knows the concept of arch. weights w1 x1 w2 w3 x2 S x3 . . . sum threshold xn wn inputs

28. Learning in neural networks But what if you present inputs for an arch, and your perceptron outputs a 0??? weights w1 x1 w2 w3 x2 S x3 . . . sum threshold xn wn inputs

29. Learning in neural networks But what if you present inputs for an arch, and your perceptron outputs a 0??? What could be done to make it more likely that the output will be 1 the next time the ‘tron sees those same inputs? weights w1 x1 w2 w3 x2 S x3 . . . sum threshold xn wn inputs

30. Learning in neural networks But what if you present inputs for an arch, and your perceptron outputs a 0??? What could be done to make it more likely that the output will be 1 the next time the ‘tron sees those same inputs? You increase the weights. Which ones? How much? weights w1 x1 w2 w3 x2 S x3 . . . sum threshold xn wn inputs

31. Learning in neural networks But what if you present inputs for not an arch, and your perceptron outputs a 1? weights w1 x1 w2 w3 x2 S x3 . . . sum threshold xn wn inputs

32. Learning in neural networks But what if you present inputs for not an arch, and your perceptron outputs a 1? What could be done to make it more likely that the output will be 0 the next time the ‘tron sees those same inputs? weights w1 x1 w2 w3 x2 S x3 . . . sum threshold xn wn inputs

33. Learning in neural networks But what if you present inputs for not an arch, and your perceptron outputs a 1? What could be done to make it more likely that the output will be 0 the next time the ‘tron sees those same inputs? You decrease the weights. Which ones? How much? weights w1 x1 w2 w3 x2 S x3 . . . sum threshold xn wn inputs

34. Let’s make one... First we need to come up with a representation language. We’ll abstract away most everything to make it simple.

35. Let’s make one... First we need to come up with a representation language. We’ll abstract away most everything to make it simple. All training examples have three blocks. A and B are upright blocks. A is always left of B. C is a sideways block. Our language will assume those things always to be true. The only things our language will represent are the answers to these five questions...

36. Let’s make one... yes = 1, no = 0 Does A support C? Does B support C? Does A touch C? Does B touch C? Does A touch B?

37. Let’s make one... yes = 1, no = 0 Does A support C? 1 Does B support C? 1 Does A touch C? 1 Does B touch C? 1 Does A touch B? 0 C A B arch

38. Let’s make one... yes = 1, no = 0 Does A support C? 1 Does B support C? 1 Does A touch C? 1 Does B touch C? 1 Does A touch B? 1 C A B not arch

39. Let’s make one... yes = 1, no = 0 Does A support C? 0 Does B support C? 0 Does A touch C? 1 Does B touch C? 1 Does A touch B? 0 A B C not arch

40. Let’s make one... yes = 1, no = 0 Does A support C? 0 Does B support C? 0 Does A touch C? 1 Does B touch C? 1 Does A touch B? 0 A B C not arch and so on.....

41. Our very simple arch learner x1 x2 S x3 x4 x5

42. Our very simple arch learner - 0.5 x1 0 0.5 x2 S x3 x4 0.5 0 x5 -0.5

43. Our very simple arch learner - 0.5 1 0 C 0.5 1 A B S 1 arch 1 0.5 0 0 -0.5

44. Our very simple arch learner - 0.5 1 0 C 0.5 1 A B S 1 arch 1 0.5 0 0 -0.5 sum = 1*-0.5 + 1*0 + 1*0.5 + 1*0 + 0*0.5

45. Our very simple arch learner - 0.5 1 0 C 0.5 1 A B S 1 0 arch 1 0.5 0 0 -0.5 sum = -0.5 + 0 + 0.5 + 0 + 0 = 0 which is not > threshold so output is 0

46. Our very simple arch learner - 0.5 1 0 C 0.5 1 A B S 1 0 arch 1 0.5 0 0 -0.5 sum = -0.5 + 0 + 0.5 + 0 + 0 = 0 which is not > threshold so output is 0 ‘tron said no when it should say yes so increase weights where input = 1

47. Our very simple arch learner - 0.4 1 0.1 C 0.6 1 A B S 1 0 arch 1 0.5 0.1 0 -0.5 sum = -0.5 + 0 + 0.5 + 0 + 0 = 0 which is not > threshold so output is 0 so we increase the weights where the input is 1

48. Our very simple arch learner - 0.4 1 0.1 C 0.6 1 A B S 1 not arch 1 0.5 0.1 1 -0.5 now we look at the next example

49. Our very simple arch learner - 0.4 1 0.1 C 0.6 1 A B S 1 0 not arch 1 0.5 0.1 1 -0.5 sum = -0.4 + 0.1 + 0.6 + 0.1 - 0.5 = -0.1 which is not > 0.5 so output is 0

50. Our very simple arch learner - 0.4 1 0.1 C 0.6 1 A B S 1 0 not arch 1 0.5 0.1 1 -0.5 sum = -0.4 + 0.1 + 0.6 + 0.1 - 0.5 = -0.1 which is not > 0.5 so output is 0 that’s the right output for this input, so we don’t touch the weights