Connectionism and LOTH

1. Connectionism Connectionism and LOTH & Language of Thought

2. Quizzes Some common problems: LOTH does not require mentalese. Face recognition is innate? Homuncular fallacy vs. homuncular functionalism. Folk psychology originated in Ancient Greece?

3. Grading 70 A 60-69 B 50-59 C 40-49 D Below 40 F

4. Remember: This quiz is only 25% of your final grade. 1st paper: 25% 2nd paper: 40% Tutorial performance: 10% So, if you did badly, don’t be too discouraged. But, if you did well, don’t be too cocky!

5. How is connectionism an alternative to LOTH? • LOT usually represented as implemented by “classical AI.” (Also known as GOFAI: “good, old-fashioned AI”.) • Semantic symbols and syntactic rules are easy to represent in classic AI architecture. • Connectionism does not require symbols, but representations can be symbolic.

6. Types of Connection Representations 1) Local representation. Meows Fur Pointed ears Whiskers Output: “it’s a cat” This node is a local representation of “cat”.

7. Local representations: • Individual nodes are symbols, and can be components of a language of thought. • Not typical of connectionist networks. • Not so biologically plausible -- “Grandmother neurons”

8. 2) Distributed representations. e.g. Cat Tiger Leopard Lion See also: www.mind.ilstu.edu/curriculum/nature_of_computers/computer_types.php

9. Distributed representations: • Connectionist networks are typically distributed representations. • Distributed representations are not necessarily symbolic. • Distributed representations are more robust to damage than local representations.

10. 3) No representation. More controversially, connectionist networks can have no representational properties. Note: • Output of connectionist network may be recognition of a concept, e.g. cat, mine, man, etc. but… • Output of connectionist network may be action, e.g. moving through space, reading aloud • Rather than representing content, networks can just act.

11. Comparison What goes on in your mind when you decide to drink a glass of water that is in front of you? LOTH: the action is the conclusion of a practical syllogism conducted through symbol manipulation Connectionism: the action is output of a neural net responding to a certain set of inputs

12. LOTH approach: I am thirsty. There is a cup of water in front of me. I believe that drinking the water will relieve my thirst. (There are is no reason not to drink the water) Conclusion: I drink the water. The conclusion is reached after manipulating the semantic symbols representing beliefs and desires in accordance with syntactic laws. Beliefs and desires give rise to action.

13. Connectionist approach: Inputs from body Inputs from environment Output: I drink the water. There are no symbols involved.

14. Connectionism makes eliminativism possible. Note: in the connectionist/eliminativist approach, the mind concocts the belief-desire explanation, “I drank the water because I was thirsty” to explain its behavior. But the desire (thirst) and beliefs (“the water is in front of me”, “the water is safe to drink”, “the water will relieve my thirst”) are not literally part of the process whereby the mind decides to drink. In other words, the mind only uses symbolic representation when translating/explaining its thoughts in language (talking to oneself or talking to others).

15. But how can “thirst” not play a role in deciding to drink? Isn’t it part of the input from the body? “Thirst” is a feeling. What plays the functional role of “thirst” may be a mechanism to detect that the body is low on water, or is somewhat overheated, but this may not be recognized by you as a desire, until you try to explain your own behavior. Note: imagine reaching unconsciously for a glass of water, and when someone asks, “why are you drinking that?”, you say, “I guess I was thirsty.” The explanation could be rather different than the cause.

16. Advantages of Connectionism 1) Biological plausibility Connectionist networks are deliberately analogous to neural processes in the brain Units ~ neurons Connections ~ synapses Activations ~ neural signals Neuron Connectionist unit

17. 2) Fast Parallel Processing • Neurons are slow. • Neurons change state very slowly compared with computer computations. Neurons can only process 100 changes a second, whereas computers can process a million. But the brain can solve many complex problems in less than a second, e.g. recognizing a face. • 100 Steps Rule • To mimic brain operations, computer programs should solve similar problems in less than 100 steps. • Connectionist programs are conducted through parallel processing, thus more can be done in 100 steps.

18. 3) Performance of connectionist networks resembles performance of human brains Connectionist networks are good at: • Pattern recognition: networks can learn through examples • Content-addressable memory: items can be retrieved based on their meanings or properties • Generalizations: networks can generalize connections between characteristics or properties

19. Connectionist networks exhibit: Graceful degradation When a connectionist network has some incorrect input -- “noisy input” -- or is partially damaged, it stills performs more poorly, but doesn’t completely break down.

20. 4) Connectionism provides a naturalistic mechanism for creating concepts. No need to posit inborn concepts. Concepts can precede language without being inborn. Fodor once claimed that mentalese was “the only game in town”. Connectionism is a new game!

21. Criticisms of connectionism The advantages of connectionism revisited: • Biological plausibility • 100-steps rule • Pattern recognition and concept formation: yes, but very slow

22. Biological plausibility Networks aren’t really like neurons. • No reverse connections (necessary for backward propagation) in the brain. • Neurons only fire or not: they cannot be both inhibitory and excitatory. • Connectionist units are too fast, neurons are quite slow.

23. Biological plausibility (cont.) • There are many different types of neurons in the brain, but connectionist units are meant to represent all neurons. • In addition, role of neurotransmitters and hormones in thinking is ignored in connectionist models. Note: most people admit that connectionist networks are still more biologically plausible than classical AI architectures. Different types of neurons

24. 2) The 100-steps rule Problem: what is a step? Is, recognizing a color one step? Or does it break down into numerous steps? The 100-steps rule only works if each unit of a connectionist network corresponds to one neuron. If one unit corresponds to several neurons working together, the 100-steps constraint may be greatly exceeded. Also, the 100-steps argument assumes only connectionist architectures are parallel processors, and all classical architectures are serial. But it is possible to build parallel classical architectures.

25. 3) Network learning is extremely slow Connectionist networks need a huge amount of explicit feedback to learn. The brain often can learn a new concept or pattern in one shot. One-shot learning is especially easy when information is gathered through language. Example: think of teaching an intelligent chimp vs. a five-year-old child, to push the red button for food.

26. Another weakness of Connectionism Systematicity and productivity: very difficult (impossible?) to implement in connectionist architecture. Connectionist responses: • Deny systematicity and productivity of the mind: Is human thinking really systematic/productive? Do animals think systematically/productively? • Maintain the ability of connectionist nets to generate systematicity and productivity

27. The Relationship between Connectionism and LOTH Three possibilities: • Connectionism implements LOTH • Connectionism replaces LOTH • Hybrid theory. Some mental processes are connectionist, some are conducted through LOT.

28. Connectionism implements LOT Connectionist nets can be regarded as a lower-level implementation of LOT. Neural nets can represent semantic symbols which are then manipulated in accordance with language-like laws (also implemented by neural nets). Criticism: if connectionist nets only implement LOT, many of the advantages of connectionism are lost.

29. 2) Connectionism replaces LOT. Consequence: all the advantages (e.g. systematicity and productivity) of LOT are lost. Can we do without them?

30. Hybrid theory. • Some mental processes are connectionist, some are conducted through LOT. • e.g. • Perception, pattern recognition and motor control handled by connectionist nets. • Reasoning and language handled by LOT (and implemented by connectionist nets).

31. Connectionism and Modularity Connectionist networks can do simple, small tasks. In more complicated tasks, they are overwhelmed by the complexity (because the connections increase exponentially). Mind must be organized into simple units, connected up in an efficient way. “Connectoplasm”: the mind an unorganized mess of connections. Not a viable idea. Mental modules: some connections preset, some learned. A way to contain the complexity.

32. Readings for next week Required: Thomas Nagel (1974), “What is it like to be a bat?”, The Philosophical Review, LXXXIII, 4 (October 1974), 435-50 at: members.aol.com/NeoNoetics/Nagel_Bat.html Block (2002), “Some Concepts of Consciousness”, in David Chalmers (Ed.). Philosophy of Mind: Classical and Contemporary Readings Oxford University Press at: www.nyu.edu/gsas/dept/philo/faculty/block/papers/Abridged%20BBS.htm Optional: Gallup, Jr., Povinelli (1998). Can Animals Empathize? Scientific American - Exploring Intelligence (a debate), available on reserve at Philosophy Department