CS 182 Lecture 28: Neuroeconomics

1. CS 182Lecture 28: Neuroeconomics J.G. Makin April 27, 2006

2. Decisions, Uncertainty, and the BrainPaul Glimcher (2003); MIT Press • Thesis: neuroscience has been dominated by the reflex paradigm • Alternative: investigations rooted in economics, evolution, game theory, and probability

4. Reflex Theory • Model: Input-Association-Output • (think of trying to explain language this way) • Even ANNs? • Methodology: thoroughly constrain the environment • “Isn’t this how science is done?” • Obscures a system-level view • Has this really led researchers astray? • Why are there so many questions on this slide?

5. Glimcher 2003

6. Reflex Theory (con’t) Challenges to “naïve” reflex theory • T. Graham Brown and the Half-Center Oscillators [This is not the name of a band, as far as I know, though it should be] • Sherrington: stimulus for walking from enteroceptive or interoceptive sources only • Reafference and Efference Copy (Von Holst and Mittelstaedt) • [Glimcher actually has these confused]

8. Reflex Theory (con’t) Challenges to “naïve” reflex theory • T. Graham Brown and the Half-Center Oscillators [This is not the name of a band, as far as I know, though it should be] • Sherrington: stimulus for walking from enteroceptive or interoceptive sources only • Reafference and Efference Copy (Von Holst and Mittelstaedt) • [Glimcher actually has these confused]

10. An Alternative Behavior is structured • by goals (cf. shoulder reflex) • by optimization strategies in the face of uncertainty • Specification of the problem on the basis of function rather than implementation (Marr) • In particular, the problem is an optimization problem • Conclusion: Neuroscience needs probability theory, economics, evolutionary theory, and game theory

11. Reflex Theory (con’t) What reflex theory doesn’t address • the shoulder “reflex” (Paul Weiss) • foraging • mate selection • exploratory behaviors • Language & thought

13. An Alternative Behavior is structured • by goals (cf. shoulder reflex) • by optimization strategies in the face of uncertainty • Specification of the problem on the basis of function rather than implementation (Marr) • In particular, the problem is an optimization problem • Conclusion: Neuroscience needs probability theory, economics, evolutionary theory, and game theory

15. I: Optimization • Q: Optimization with respect to what? • A: Inclusive fitness but modularized. Evolution provides the goals, economics the optimization techniques • Do we have a prayer at specifying the optimum? • Phototransduction near the quantum limit • Hair cells can detect individual fluid molecule collisions • Convergent Evolution: Cichlid fish of Tanzania

17. II: Uncertainty: Epistemological • Reflex theory dominated by deterministic responses to input (from a highly constrained set) • Alternative: in general, we suffer from epistemological uncertainty, so we have to optimized in an indeterminate world

18. Uncertainty (con’t) • An empirical test of foraging economics: the prey model, Parus major • View foraging as an optimization problem: choose the probability p_i of attacking the prey i that maximizes the rate at which energy is gained • Solution: • “zero-one” rule • “independence from encounter inclusion rate” principle

20. Uncertainty (con’t) • Frequencies of large and small mealworms were varied • Small mealworms always had larger handling time • Prediction (from optimal sol’n): • Preference for large worms as their freq. increases, regardless of small worm freq. (by IEIR principle) • If the bird couldn’t get all the worms, it should give up entirely on the small ones (by the zero-one rule) • Result: yes and no (only 85% selective) • Maybe this is an optimal strategy after all…

21. Epistemological Uncertainty & the Brain:A Series of Studies • Input-association-output model: sensory-parietal-motor • Lateral intraparietal area (LIP) and monkey saccades: • Monkeys trained to perform task w/juice reward • Invariant to input stimulus (light or button or whatever) • Position-encoding • Conclusion: command signal (Mountcastle)

22. Epistemological Uncertainty & the Brain (con’t) • Lateral intraparietal area (LIP) and monkey saccades: • Fixation and saccade tasks w/eccentric light • Weak activation on fixation, but increasingly active over trials of saccade task • Conclusion: attentional enhancement (Goldberg)

24. Epistemological Uncertainty & the Brain (con’t) • Lateral intraparietal area (LIP) and monkey saccades: • Memory saccade task: target is extinguished but LIP neuron still fires—until the motor command is executed • Conclusion: motor intention (Gnadt & Anderson)

26. Epistemological Uncertainty & the Brain (con’t) Platt & Glimcher: encoding the probability of pay-off

27. Epistemological Uncertainty & the Brain (con’t) Probability experiment

28. Epistemological Uncertainty & the Brain (con’t) Value experiment

30. III: “Irreducible” Uncertainty & Game Theory • Static environment  Dynamic competition with other agents • Then the optimal approach is given by game- theoretic approaches • In these cases, the optimum often involves (purposefully) random behavior

31. Uncertainty & Game Theory (con’t) • Example 1: Chicken

33. Uncertainty & Game Theory (con’t) • Conclusion: Smith is best served by behaving non-deterministically, but with probability 0.647 of being a chicken. (Ditto for Jones.) • If Jones finds non-randomness in the distribution of Smith’s choices, he can predict above chance which option Smith will pick—and win. • Random behavior is the optimal solution, so: we shouldn’t expect behavior to look deterministic (contrast w/reflex theory).

34. Intermezzo: How Random Are We? • Paper, scissors, rocks • Dice, viscera divination, etc.: technological breakthrough (Jaynes) • Unconscious vs. conscious behaviors; natural selection vs. “rational actors” • Pigeons, babies, and adults: the matching rule and cognitive load (and reward)

35. Game Theory and Ethology • Duck foraging • Two feeders at opposite ends • 33 ducks • Rate of food depends on feeder, but the more ducks in an area the worse it is • Where to sit?

36. Game Theory & Ethology (con’t)

37. Game Theory & Ethology (con’t) • Person 1: 2-gram bread ball every 5 sec • Person 2: 2-gram bread ball every 10 sec

38. Game Theory & Human Behavior:Work or Shirk

39. Game Theory & Human Behavior:Work or Shirk (con’t) Insp = -50 Insp = -5

40. Game Theory & Human Behavior:Work or Shirk (con’t) • Experiment: subjects play against a computer program which looks for statistical regularities in its opponent’s plays and tries to exploit them • Subjects are only told that they can make money by playing • 150 trials, then the pay-off matrix switches (unannounced) • Guess how human beings played….

41. Game Theory & Human Behavior:Work or Shirk (con’t) • 150 trials, one pay-off matrix, vis-à-vis the Nash equilibrium?

42. Game Theory & Human Behavior:Work or Shirk (con’t)

43. Game Theory & Human Behavior:Work or Shirk (con’t) • Work-shirk-work-shirk yields 50% behavior. Shannon entropy of choices?

44. Game Theory & Human Behavior:Work or Shirk (con’t)

45. Game Theory & Human Behavior:Work or Shirk (con’t) • Switching between pay-off matrices?

46. Game Theory & Human Behavior:Work or Shirk (con’t)

47. Game Theory & the Brain • Repeat the game, this time with monkeys instead of humans • Simultaneously record from parietal area LIP • Prediction: if these neurons encode expected utility, then they will fire at constant rates over various movements and various rewards (contrast Platt & Glimcher 1999) • Now we have an experiment that yields non-deterministic behavior but about which predictions of lawful actions can nevertheless be made

48. Game Theory & the Brain (con’t)

49. Game Theory & the Brain (con’t) • Across trials: • Monkeys behave (near?) optimally: their behaviors track the Nash equilibrium • LIP neurons do not track the Nash equilibrium suggesting that they are, in fact, encoding (relative) expected utility • Play-by-play: • The relative expected value on any given play does vary slightly, given the randomness of play • Positive correlation b/n this fluctuating expected value and fluctuations in LIP neurons

50. Neuroeconomics & Language • Skinner’s Verbal Behavior • Programs that are more than input/output • Bayes Nets for utility as well as beliefs • Minimum description length: grammar • Minimum description length: evolution