CogSci C131/Psych C123 Computational Models of Cognition

1. CogSci C131/Psych C123Computational Models of Cognition Tom Griffiths

2. CogSci C131/Psych C123Computational Models of Cognition Tom Griffiths

3. Computation Cognition

4. input input output output Cognitive science • The study of intelligent systems • Cognition as information processing

5. computation computation input input output output Computational modeling Look for principles that characterize both computation and cognition

6. Two goals • Cognition: • explain human cognition (and behavior) in terms of the underlying computation • Computation: • gain insight into how to solve some challenging computational problems

7. human cognition sets the standard Computational problems • Easy: • arithmetic, algebra, chess • Difficult: • learning and using language • sophisticated senses: vision, hearing • similarity and categorization • representing the structure of the world • scientific investigation

8. Three approaches Rules and symbols Networks, features, and spaces Probability and statistics

9. Three approaches Rules and symbols Networks, features, and spaces Probability and statistics

10. Logic All As are Bs All Bs are Cs All As are Cs Aristotle (384-322 BC)

11. Mechanical reasoning (1232-1315)

12. The mathematics of reason Thomas Hobbes (1588-1679) Rene Descartes (1596-1650) Gottfried Leibniz (1646-1716)

13. Modern logic PQ P Q George Boole (1816-1854) Friedrich Frege (1848-1925)

14. Computation Alan Turing (1912-1954)

15. Inference Rules Q The World Logic Facts P  Q P

16. Categorization

17. Categorization cat  small  furry  domestic  carnivore

18. Inference Rules Q The World Logic Facts P  Q P

19. Operations The World Early AI systems… Workspace Operations Facts Goals Actions Observations

20. Rules and symbols • Perhaps we can consider thought a set of rules, applied to symbols… • generating infinite possibilities with finite means • characterizing cognition as a “formal system” • This idea was applied to: • deductive reasoning (logic) • language (generative grammar) • problem solving and action (production systems)

21. Language as a formal system Noam Chomsky

22. L all sequences Language “a set (finite or infinite) of sentences, each finite in length and constructed out of a finite set of elements” This is a good sentence 1 Sentence bad this is 0 linguistic analysis aims to separate the grammatical sequences which are sentences of L from the ungrammatical sequences which are not

23. NP VP T N V NP man T N hit the the ball A context free grammar S  NP VP NP  T N VP  V NP T  the N  man, ball, … V  hit, took, … S

24. Rules and symbols • Perhaps we can consider thought a set of rules, applied to symbols… • generating infinite possibilities with finite means • characterizing cognition as a “formal system” • This idea was applied to: • deductive reasoning (logic) • language (generative grammar) • problem solving and action (production systems) • Big question: what are the rules of cognition?

25. human cognition sets the standard Computational problems • Easy: • arithmetic, algebra, chess • Difficult: • learning and using language • sophisticated senses: vision, hearing • similarity and categorization • representing the structure of the world • scientific investigation

26. “In solving a problem of this sort, the grand thing is to be able to reason backward. That is a very useful accomplishment, and a very easy one, but people do not practice it much.” Inductive problems • Drawing conclusions that are not fully justified by the available data • e.g. detective work • Much more challenging than deduction!

27. Challenges for symbolic approaches • Learning systems of rules and symbols is hard! • some people who think of human cognition in these terms end up arguing against learning…

28. The poverty of the stimulus • The rules and principles that constitute the mature system of knowledge of language are actually very complicated • There isn’t enough evidence to identify these principles in the data available to children Therefore • Acquisition of these rules and principles must be a consequence of the genetically determined structure of the language faculty

29. The poverty of the stimulus Learning language requires strong constraints on the set of possible languages These constraints are “Universal Grammar”

30. Challenges for symbolic approaches • Learning systems of rules and symbols is hard! • some people who think of human cognition in these terms end up arguing against learning… • Many human concepts have fuzzy boundaries • notions of similarity and typicality are hard to reconcile with binary rules • Solving inductive problems requires dealing with uncertainty and partial knowledge

31. Three approaches Rules and symbols Networks, features, and spaces Probability and statistics

32. Similarity What determines similarity?

33. Representations What kind of representations are used by the human mind?

34. Semantic networks Semantic spaces Representations How can we capture the meaning of words?

35. Categorization

36. error: +1 = cat, -1 = dog y cat dog y x2 x1 x2 x1 perceptual features Computing with spaces

37. Networks, features, and spaces • Artificial neural networks can represent any continuous function…

38. OR AND x2 x2 x1 x1 XOR x2 x1 Problems with simple networks y x2 x1 x1 x2 Some kinds of data are not linearly separable

39. output layer y z2 z1 x2 input layer x1 A solution: multiple layers y hidden layer z1 z2 x1 x2

40. Networks, features, and spaces • Artificial neural networks can represent any continuous function… • Simple algorithms for learning from data • fuzzy boundaries • effects of typicality

41. General-purpose learning mechanisms E (error) ( is learning rate) wij

42. influence of input output error The Delta Rule +1 = cat, -1 = dog y for any function g with derivativeg x1 x2 perceptual features

43. Networks, features, and spaces • Artificial neural networks can represent any continuous function… • Simple algorithms for learning from data • fuzzy boundaries • effects of typicality • A way to explain how people could learn things that look like rules and symbols…

44. output layer copy context units input input layer Simple recurrent networks x1 x2 hidden layer z1 z2 x1 x2 (Elman, 1990)

45. Hidden unit activations after 6 iterations of 27,500 words (Elman, 1990)

46. Networks, features, and spaces • Artificial neural networks can represent any continuous function… • Simple algorithms for learning from data • fuzzy boundaries • effects of typicality • A way to explain how people could learn things that look like rules and symbols… • Big question: how much of cognition can be explained by the input data?

47. Challenges for neural networks • Being able to learn anything can make it harder to learn specific things • this is the “bias-variance tradeoff”

48. Bias-variance tradeoff

49. Bias-variance tradeoff

50. Bias-variance tradeoff