160 likes | 266 Views
Deconstructing the Law of Effect. C. R. Gallistel Rutgers Center for Cognitive Science. Matching Is. An innate strategy foraging strategy (or, possibly, an innate principle of choice = melioration)
E N D
Deconstructing the Law of Effect C. R. Gallistel Rutgers Center for Cognitive Science CSH, May 23, 2005
Matching Is • An innate strategy foraging strategy (or, possibly, an innate principle of choice = melioration) • The apportionment of foraging time (and effort) in accord with the incomes obtained from the foraging locations: • T1/T2=I1/I2, where Ti = time invested in Location i & Ii = income obtained from Location i • Equivalently: T1/(T1+T2) = R1/(R1+R2) & T2/(T1+T2) = R2/(R1+R2) [“Herrnstein fractions”] • An evolutionarily stable strategy CSH, May 23, 2005
Income vs Return • Income (Ii) = (reward from Loc i)/(unit time spent foraging, without regard to where time is invested, that is, summed over i) • Return (Ri) = (reward from Loc i)/(unit time spent foraging at a given Loc i), i.e., per unit time invested in a location • Matching equates returns;T1/T2=I1/I2 ---> I1/T1 = T2/I2 CSH, May 23, 2005
Model 1: Law of Effect • Matching driven by inequality of returns • Subject compares returns and reapportions investments to favor location yielding greater return • If income is only loosely connected to investment, then this decreases return on favored side and increases it on disfavored side • Eventually, a dynamic equilibrium is attained (the equation of returns) • No change-detector necessary • Hill climbing = slow! CSH, May 23, 2005
Model 2: Innate Strategy Pure Feed Forward • First proposed in a general way by Gene Heyman (“matching is unconditioned behavior”) • Leaving rate parameters the inverse of the proportions among the incomes • A separate process detects changes in income • Income estimator works on small samples taken from the period between the time when a change point is (retrospectively) estimated to have occurred and the time at which the change is detected CSH, May 23, 2005
Old Rat Results CSH, May 23, 2005
Appearance of Matching in the Experimentally Naïve Mouse • Mouse shuttles back and forth between two feeding hoppers • Hopper entries monitored by IR beams CSH, May 23, 2005
Coupling between investment and income • Slightly coupled: traditional concurrent VIs with indefinite hold for harvesting • Income completely independent of investment (pellets accumulate at random rates; the mouse harvests the accumulation whenever it comes • Complete coupling: the schedule clock only for a hopper only runs when the mouse has its head in that hopper CSH, May 23, 2005
Tracking Matching • Compute difference in “Herrnstein fractions” after each feeding • R1/(R1+R2)- R2/(R1+R2)=(R1- R2)/ (R1+R2)= Feeding Imbalance; possible values -1 & 1,depending on whether feeding was at 1 or 2 • T1/(T1+T2)-T2/(T1+T2)=(T1- T2)/(T1+T2)= Investment Preference; ranges -1 to 1, depending on how mouse spent its time between feedings • Cumulate differences: slope of cumulative record gives average values; when slopes of the 2 cumulative records are the same mouse is matching CSH, May 23, 2005
Condition 1 • Relative income jointly determined by programmed ratio and sampling behavior • Naïve mouse samples rarely • Hence initial ratio of incomes may be 1:1 no matter what the programmed ratio • Mouse is always matching • Changes in relative income and changes in relative investment coincide (even though income only very loosely connected to investment) CSH, May 23, 2005
All Mice in Cond 1 • They are, in general, as close to matching at the beginning of training as they are at the end of it CSH, May 23, 2005
Condition 2 • Income imbalances now constant from beginning (as intended) • Mice match from the beginning CSH, May 23, 2005
Condition 3 • When income is directly tied to investment, the only place in behavioral parameter space for matching is exclusive devotion to one or the other option • Thus, on our model, preference should be (almost) all-or-none, but unstable CSH, May 23, 2005
A Closer Look • The session in which the major reversal occurred • Nothing in the returns justifies the switch! • The change in income does justify the switch • This change is caused by the switch, but, in our model, the animal doesn’t know that, because it takes no account of the impact of its behavior on what it observes CSH, May 23, 2005
Terminal Economic Weirdness • Animals are oblivious to the impact of their own behavior on the rewards they receive (pace Skinner & the entire economics profession) CSH, May 23, 2005
Matching & theHunter-Gatherer World • Hunting & gathering, unlike farming, do not bring food into existence, they merely exploit what is there • Intelligent sampling of a changeable environment is essential in hunter-gathering • Matching maximizes income when income is environment-limited rather than investment-limited • Matching samples the world intelligently CSH, May 23, 2005