CS 188: Artificial Intelligence Spring 2007

1. CS 188: Artificial IntelligenceSpring 2007 Lecture 27: Neural Computation 5/1/2007 Srini Narayanan– ICSI and UC Berkeley

2. Announcements • Reinforcement Learning Q/A • Wednesday 6 PM, 306 SODA • Helicopter control talk • Thursday (4 -5:30 PM) 310 SODA

3. Sci-Fi AI?

4. What is AI? The science of making machines that:

5. Thinking Like Humans? • The Cognitive Science approach: • 1960s ``cognitive revolution'': information-processing psychology replaced prevailing orthodoxy of behaviorism • Scientific theories of internal activities of the brain • What level of abstraction? “Knowledge'' or “circuits”? • Cognitive science: Predicting and testing behavior of human subjects (top-down) • Cognitive neuroscience: Direct identification from neurological data (bottom-up) • Both approaches now distinct from AI • Both share with AI the following characteristic: • The available theories do not explain (or engender) anything resembling human-level general intelligence} • Hence, all three fields share one principal direction! Images from Oxford fMRI center

6. Basic Ideas • Neural Computation • The brain as a computing device • Learning in the brain. • Brain-based Computing • Brain Machine Interfaces

7. BRAIN

8. Motor cortex Somatosensory cortex Sensory associative cortex Visual associative cortex Broca’s area Visual cortex Primary Auditory cortex Wernicke’s area

9. Imaging the Brain

10. Sensory Systems • Vision (nearly 30-50% ) • Audition (nearly 10%) • Somatic • Chemical • Taste • Olfaction

11. Motor Systems • Locomotion • Manipulation • Speech

12. NEURON

13. 1000 operations/sec 100,000,000,000 units 10,000 connections/ graded, stochastic embodied fault tolerant evolves, learns 1,000,000,000 ops/sec 1-100 processors ~ 4 connections binary, deterministic abstract crashes designed, programmed (usually) Brains ~ Computers

14. Computing in the Brain: Mirror Neuron in F5 (premotor cortex) Approx 2 s: Experimenter picks up food Approx 4 s: Monkey picks up food

15. Many human imaging studies showing activation of motor regions (primary and secondary) during action perception. Regions overlap with those engaged during action production. Distributed frontal/parietal activation during viewing of actions performed with mouth, hand, or foot. All activations compared to rest baseline. Mouth Hand Foot (chewing, biting) (grasping, pinching) (kicking, jumping) Buccino et al. 2001)

16. Foot Action Hand Action Mouth Action Observation of action activates premotor cortex in topographic manner, consistent with motor topography. a) no-object b) w/ object

17. MEG (magnetoencephalography) study comparing pianists and non-pianists. Pianists show activation in primary motor cortex when listening to piano. Activation is specific to fingers used to play the notes. Colored region: MEG signal for pianists minus non-pianists.

18. Significance of Mirror Neurons • Action, Perception, Imagination, and Understanding share a lot of the same brain circuits. • Question: • How are these circuits learnt?

19. Models of Learning • Hebbian ~ coincidence • Reinforcement ~ delayed reward • Supervised ~ correction (perceptron, mlp) • Unsupervised-similarity

20. Long-term Potentiation (LTP) Rapid and long-term increase in synaptic strength resulting from the pairing of presynaptic activity with postsynaptic depolarization

21. Synaptic Plasticity • Hebb’s Postulate: When an axon of cell A... excites cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells so that A's efficiency as one of the cells firing B is increased. • Slices of the hippocampus can be removed and its CA1 neurons studied in vitro with recording electrodes. Rapid, intense stimulation of presynaptic neurons evokes action potentials in the postsynaptic neuron. This is just what we would expect from the properties of synapses.

22. The Hebb rule is found with long term potentiation (LTP) in the hippocampus Schafer collateral pathway Pyramidal cells 1 sec. stimuli At 100 hz

23. Neural Correlates of RL Parkinson’s Disease  Motor control + initialtion? Intracranial self-stimulation; Drug addiction; Natural rewards  Reward pathway?  Learning? Also involved in: • Working memory • Novel situations • OCD • Schizophrenia • …

24. = Conditional stimulus = Unconditional stimulus Response = Unconditional response (reflex); conditional response (reflex) Conditioning Ivan Pavlov

25. Unpredicted reward (unlearned/no stimulus) Predicted reward (learned task) Omitted reward (probe trial) Dopamine Levels track RL signals (Montague et al. 1996)

26. Current Hypothesis Phasic dopamine encodes a reward prediction error • Evidence • Monkey single cell recordings • Human fMRI studies • Current Research • Better information processing model • Other reward/punishment circuits including Amygdala (for visual perception) • Overall circuit (PFC-Basal Ganglia interaction)

27. Neural Basis of Intelligence • How does a system of neurons with specific processes, connectivity, and functions support the ability to think, reason, and communicate? Take CS 182 in Spring 2008.

28. Basic Ideas • Neural Computation • The brain as a computing device • Learning in the brain. • Brain-based Computing • Determining cognitive states from imaging data. • Brain Machine Interfaces

29. Brain Machine Interfaces • Sensory Prosthesis • Brain Computer Interfaces from Brain signals

30. Sensory Prosthesis

31. Visual Prosthesis

32. Cochlear Implants

33. BCI using EEG

34. EEG Control of a robot in a labyrinth

35. Decoding Cognitive Signals