1 / 16

Deconstructing the Law of Effect

Deconstructing the Law of Effect. C. R. Gallistel Rutgers Center for Cognitive Science. Matching Is. An innate strategy foraging strategy (or, possibly, an innate principle of choice = melioration)

fergal
Download Presentation

Deconstructing the Law of Effect

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Deconstructing the Law of Effect C. R. Gallistel Rutgers Center for Cognitive Science CSH, May 23, 2005

  2. Matching Is • An innate strategy foraging strategy (or, possibly, an innate principle of choice = melioration) • The apportionment of foraging time (and effort) in accord with the incomes obtained from the foraging locations: • T1/T2=I1/I2, where Ti = time invested in Location i & Ii = income obtained from Location i • Equivalently: T1/(T1+T2) = R1/(R1+R2) & T2/(T1+T2) = R2/(R1+R2) [“Herrnstein fractions”] • An evolutionarily stable strategy CSH, May 23, 2005

  3. Income vs Return • Income (Ii) = (reward from Loc i)/(unit time spent foraging, without regard to where time is invested, that is, summed over i) • Return (Ri) = (reward from Loc i)/(unit time spent foraging at a given Loc i), i.e., per unit time invested in a location • Matching equates returns;T1/T2=I1/I2 ---> I1/T1 = T2/I2 CSH, May 23, 2005

  4. Model 1: Law of Effect • Matching driven by inequality of returns • Subject compares returns and reapportions investments to favor location yielding greater return • If income is only loosely connected to investment, then this decreases return on favored side and increases it on disfavored side • Eventually, a dynamic equilibrium is attained (the equation of returns) • No change-detector necessary • Hill climbing = slow! CSH, May 23, 2005

  5. Model 2: Innate Strategy Pure Feed Forward • First proposed in a general way by Gene Heyman (“matching is unconditioned behavior”) • Leaving rate parameters the inverse of the proportions among the incomes • A separate process detects changes in income • Income estimator works on small samples taken from the period between the time when a change point is (retrospectively) estimated to have occurred and the time at which the change is detected CSH, May 23, 2005

  6. Old Rat Results CSH, May 23, 2005

  7. Appearance of Matching in the Experimentally Naïve Mouse • Mouse shuttles back and forth between two feeding hoppers • Hopper entries monitored by IR beams CSH, May 23, 2005

  8. Coupling between investment and income • Slightly coupled: traditional concurrent VIs with indefinite hold for harvesting • Income completely independent of investment (pellets accumulate at random rates; the mouse harvests the accumulation whenever it comes • Complete coupling: the schedule clock only for a hopper only runs when the mouse has its head in that hopper CSH, May 23, 2005

  9. Tracking Matching • Compute difference in “Herrnstein fractions” after each feeding • R1/(R1+R2)- R2/(R1+R2)=(R1- R2)/ (R1+R2)= Feeding Imbalance; possible values -1 & 1,depending on whether feeding was at 1 or 2 • T1/(T1+T2)-T2/(T1+T2)=(T1- T2)/(T1+T2)= Investment Preference; ranges -1 to 1, depending on how mouse spent its time between feedings • Cumulate differences: slope of cumulative record gives average values; when slopes of the 2 cumulative records are the same mouse is matching CSH, May 23, 2005

  10. Condition 1 • Relative income jointly determined by programmed ratio and sampling behavior • Naïve mouse samples rarely • Hence initial ratio of incomes may be 1:1 no matter what the programmed ratio • Mouse is always matching • Changes in relative income and changes in relative investment coincide (even though income only very loosely connected to investment) CSH, May 23, 2005

  11. All Mice in Cond 1 • They are, in general, as close to matching at the beginning of training as they are at the end of it CSH, May 23, 2005

  12. Condition 2 • Income imbalances now constant from beginning (as intended) • Mice match from the beginning CSH, May 23, 2005

  13. Condition 3 • When income is directly tied to investment, the only place in behavioral parameter space for matching is exclusive devotion to one or the other option • Thus, on our model, preference should be (almost) all-or-none, but unstable CSH, May 23, 2005

  14. A Closer Look • The session in which the major reversal occurred • Nothing in the returns justifies the switch! • The change in income does justify the switch • This change is caused by the switch, but, in our model, the animal doesn’t know that, because it takes no account of the impact of its behavior on what it observes CSH, May 23, 2005

  15. Terminal Economic Weirdness • Animals are oblivious to the impact of their own behavior on the rewards they receive (pace Skinner & the entire economics profession) CSH, May 23, 2005

  16. Matching & theHunter-Gatherer World • Hunting & gathering, unlike farming, do not bring food into existence, they merely exploit what is there • Intelligent sampling of a changeable environment is essential in hunter-gathering • Matching maximizes income when income is environment-limited rather than investment-limited • Matching samples the world intelligently CSH, May 23, 2005

More Related