Theories of Reinforcement

1. Theories of Reinforcement

2. Law of Effect • The law of effect is circular: • What is a reinforcer? An event that increases behavior. • What events increase behaviors? Reinforcers. • How can we break this circularity?

3. Need-Reduction Theory • Deprivation (e.g., food, water) lead to • Deficiencies (e.g., changes in tissue, endocrine system) which give rise to • A need to restore the status quo (homeostasis). • Reinforcers - stimuli that decrease some biological need state. Events that do not do so will not be reinforcers.

4. Problems with Need-Reduction Theory • Sexual behavior: Male rats learn to run to a female in heat with no copulation. • Rats pressing lever for electrical brain stimulation (Olds, 1958) • Rats will work for saccharine • It takes time for food to be ingested and the biological need to be reduced – • Very long delay of reinforcement.

5. Drive-Reduction Theory • Any strong stimulation (internal or external) is aversive and: • produces a negative drive state • reinforcement occurs when this drive state is reduced by some consequential event. • A drive is an intervening variable: • It is correlated with biological needs. • Satisfying a drive state is likely to satisfy a biological need later.

6. Drive-Reduction Theory • Food pellets are reinforcing • Satisfy a drive state. • Eventually satisfy the need state, as well. • Unlike needs, drives can satisfied instantly, and there is no delay of reinforcement problem.

7. Drive vs. Need • Oakley and Pfaffmann (1962) • Rats worked for saccharin (artificial sweetener, no nutritive value, therefore doesn’t satisfy a need). • Then they placed lesions in the central nervous system of rats (thalamus) to eliminate the sense of taste – eliminating drive, but need still present. • Response rates for saccharin (no nutritive value) and sucrose (sugar) both decreased. • Drive important

8. Drive vs. Need • Epstein and Teitelbaum (1962) • Rats pressed levers for injection of food into gastric tube. • They maintained perfect control of their body weights. • Food into stomach can therefore be a reinforcer, and can be used to train new behaviors. • This procedure too eliminates drive, food removes the need, so now need looks like it is important.

9. Problems with Drive-Reduction • Theory is unfalsifiable • Can have a conditioned drive state for anything • Monkeys working for visual stimuli, such as moving toy trains (Butler, 1953). • Or to see people in the lab. • Rats working for light (Premack & Collier, 1962). • But they prefer the dark… • Exploratory drive? Curiosity drive? Motherhood drive? • It is just too easy to think of new drives to explain everything that animals will work for...

10. Problems with Drive-Reduction • Prediction of reinforcers • If we had an adequate theory of reinforcement, we could predict what events were reinforcers • Especially problematic in applied work • everyone “needs” a different reinforcer. • Can predict some things • Make a person hungry, food will probably be a reinforcer • But which food? • Back to the process: • The details may be different but the underlying processes should be the same.

11. Classical Reinforcement Theory • Assumptions: • Some responses are reinforcible, and are always reinforcible. • Other responses are never reinforcible (consummatory responses). responses that can not be reinforced responses that can be reinforced instrumental responses consummatory responses

12. Classical Reinforcement Theory • Assumptions: • Reinforcers are regarded as events, as stimuli • Reinforcers are trans-situational • If a stimulus reinforces lever presses, it should also reinforce other responses. stimuli that are always reinforcers stimuli that are never reinforcers

13. Premack Theory • Premack suggested a way to predict a priori what events would be reinforcers. • Denied assumptions of classical reinforcement theory. • The reinforcement process: • Relation between responses • Not a relation between responses and consequential stimuli. • No clear boundaries between behaviors and reinforcers • Is it food that is the reinforcer, or eating? • Is it the toy, or playing? • Premack's principle of positive reinforcement: • If an instrumental response is followed by a contingent response that is more highly probable, the instrumental response will increase in frequency.

14. Premack Theory • Stated another way: • If a less probable response is followed by a more probable response, the less probable response will increase in frequency. • L: less probable response • H: more probable response • Reinforcement is L → H. (Probability of L increases) • But if H → L • H will decrease in frequency • Punishment

15. Premack Theory • Measure unconstrained baseline behavior and rank all activities in terms of their probability.   • Every behavior can reinforce behavior down the list and punish behavior further up. • The Premack principle implies the following: • 1. That all responses are potentially reinforcible - whereas classical theory said some were, and some were not. • 2. That all responses are potentially reinforcers for other, less probable, responses. • Premack's indifference principle • Irrelevant how the current behavior probabilities got to be what they are (e.g., through deprivation, learning, or whatever). All that matters is the current probabilities.

16. Testing Premack • Premack (1963) used four monkeys, and designed four manipulanda - • A door they could open (D) • A lever they could press (L) • A horizontal lever they could operate (H) • A plunger they could operate (P) • Only 1 of the 4 (Chicko) showed clear response-probability differentials • So, only Chicko can be used to test the theory. • Chicko’s order: H>D/L >P

17. Results for Chicko from Premack (1963) ItemIndependentPairedContingency P(H)7868243 P(D)7893214 L(H)270326342 P(L)7840246 D(H)382274467 L(D)270233245 H(P)543584382 D(P)382369298 H(D)543459424 H>D/L>P No reinforcement

18. Summarizing Premack's (1963) Results • Chicko ordered H>D/L>P • In contingency tests, • H reinforced D, L, and P • D and L did not reinforce H, but did reinforce P • P did not reinforce any other response. • Thus, D and L were both reinforcers for P • but not reinforcers (for H). • Contrary to classical reinforcement theory: • Events were either reinforcers, or not reinforcers • Violating the trans-situationality assumption

19. Reinforcing Consummatory Responses • Premack (1959) used children as subjects. • Could play pinball or eat chocolate. • 61% of the children preferred to play pinball • 39% preferred to eat chocolate • Divided each of these groups into two subgroups. • Eat-to-Play • Play-to-Eat

20. Premack (1959) • For the Pinballers: • Eat to Play increased Eat significantly • Play to Eat increased Play only a very small amount • For the Eaters: • Eat to Play very small increase in Eat • Play to Eat increased Play significantly • Supports Premack theory • Demonstrated that eating, a consummatory response according to classical reinforcement theory, could be reinforced

21. Premack (1962)-Assumptions • 1. Depriving a rat of water for 24 hours increases the probability of the rat drinking when water is available. • 2. Depriving a rat of running in an activity wheel for 24 hours increases the probability of running when the wheel is presented. • If running is more probable than drinking, then according to his theory running should reinforce drinking. • Reverse of the usual reinforcer relation: run to drink

22. Premack (1962) • Condition 1 • Rats had 24-hour access to an activity wheel, but were not allowed to drink. • In 1-hour test session with activity wheel and drinkometer, spent, on average, 240 s drinking and 50 s running. • Drinking was more probable than running. • Condition 2 • Rats had 24-hour access to water, but they were not allowed to run. • In a 1-hour test session, they drank for 28 s on average, and ran for 329 s. • Running was more probable than drinking.

23. Premack (1962) Results • Two contingency tests for both groups • Drink to run • Run to drink • Under Condition 1 (drinking > running), run-to-drink reinforced running, but drink-to-run did not reinforce drinking. • Under Condition 2 (running > drinking), drink-to-run reinforced drinking (drinking time went from 28 to 98 s), but run-to-drink did not reinforce running. • Increase in drinking in the drink-to-run contingency • Reinforcement of a typical consummatory response. • Supports Premack's indifference principle • Doesn't matter how the responses got to be at their current probability.

24. Punishment • If a high probability response is followed by a lower probability response, the high-probability response should decrease in frequency. • i.e. H → L • Problem: If the response has a low probability, how can we get the animal to emit it?

25. Weisman and Premack (unpublished) • Used motorized activity wheel: makes rats run • Condition 1: 24 hours water, no activity wheel. • Running reinforced Drinking in contingency tests.  • Condition 2: 24 hours activity wheel, no water. • Test: Drink was followed by a 5-s forced run. • The forced run decreased drinking substantially • Aged rats (who had a very low probability of running) completely stopped drinking in the test sessions when drinking was followed by the forced run. • Typical punishment effects.

26. Relations: Reinforcement and Punishment • H  L  H  L  H etc • Premack (1969): reinforcement and punishment two sides of the same coin, and cannot be separated. • Reinforce anything you’re punishing something else • Value isn't in the thing itself, it’s in the behavior that the thing controls. • Value of old bottle of water in your car? • Equally, food has no value, except if you can indulge in eating. • Symmetrical positive and negative laws of effect? • Or a single law concerned with how both response probabilities change when two responses occur in close temporal conjunction.

27. Premack • Premack has had a profound effect on the theory of reinforcement • Newer theories (such as response-deprivation theory and behavior-regulation theory) • Applications in real-life situations.

28. Applications of Premack • Homme, deBaca, Devine, Steinhorst and Rickert (1963) • 3-year old nursery-school children: want them sitting quietly in chairs • Verbal instructions ineffective • High probability behaviors: • Running around the room, screaming, pushing chairs, or quietly working on jigsaw puzzles • Low probability behaviors: • Sitting in their chairs • Make the high probability behaviors contingent on the low probability behaviors: • Contingency: Sit and attend  Bell + "Run and Scream!"

29. Homme, deBaca, Devine, Steinhorst and Rickert (1963) • Later: Tokens earned for low-probability behaviors which could be used to buy high-probability behaviors • Control was virtually perfect after just a few days.  • "In summary, even in this preliminary, unsystematic application, the Premack hypothesis proved to be an exceptionally practical principle for controlling the behavior of nursery school Ss."

30. Token Economies • Token economy programs • developed for the rehabilitation of long-term schizophrenic patients • The behavior of such patients can be brought under reinforcement control (Ayllon & Azrin, 1965; Atthowe & Krasner, 1968; Winkler, 1970). • But all report patients who failed to respond to the token regime. • 18% of Ayllon and Azrin's • 10% of Atthowe and Krasner's • Despite the use of multiple reinforcers, reinforcer sampling and reinforcement exposure (Ayllon & Azrin, 1968) designed to increase the utilization of back-up reinforcers, these patients did not work for the available rewards.

31. Mitchell & Stoffelmayr (1973) • "Application of the Premack Principle to the behavioral control of extremely inactive schizophrenics” • The ward with the most severely ill chronic patients was chosen. The ongoing treatment included industrial therapy, ward domestic work, and weekly group discussions. • Selected the 4 most inactive patients.. • Identified items that could be used as reinforcers: candies, cigarettes, fruit, and cookies. • For two patients cigarettes and fruit were used maintain working • Found no dispensible reinforcers for WL and PM • Used the Premack principle directly

32. Number of 30-s intervals in which an instance of work occurred in 30 minutes. Used WL and PM in tests of Premack theory

33. Mitchell & Stoffelmayr (1973) • Instructions + reinforcement sessions. • Response that occurred freely with very high frequency was sitting, which was used for both patients. • Shaping sessions: experimenter approached the patient and asked him to stand. • If the patient remained seated, the experimenter would tip the subject's chair forward until the patient stood up. • The patient then given a coil and if they removed some wire they got to sit down. • Repeat after 90 seconds, gradually increasing response requirement • By Session 14, WL was removing three coil wrappings before reinforcement was given, while PM achieved this by Session 17. • In the subsequent sessions, the patients were allowed to remain seated while working. Reinforcement became resting.

34. Inactive catatonic schizophrenic patients

35. "The present results suggest that even the most severely inactive patient will respond to a reinforcement regime. The strict application of Premack's principle then may have considerable therapeutic application for those patients, who in refusing to accept any tangible reward, do not respond to the token regime."

36. Changing Relations: Premack (1969) • The following responses were offered to rats: • Drink 4% sucrose • Drink 32% sucrose • Run • Responses tracked over a session

37. Tests: Drinking 32% sucrose reinforced running at the start of the session, but at the end of the session, the opportunity to run reinforced drinking 32% sucrose

38. Problems with Premack • Response probabilities can fluctuate within a session, therefore difficult to measure. • In an applied setting, we might offer the clients a choice: • “What do you want to do after this activity?” • Research has found that a lower-probability response can under certain circumstances reinforce a higher-probability response.

39. Response-Deprivation Hypothesis • According to the probability-differential view (the original version of Premack theory), a lower-probability response can never reinforcer a higher-probability response. • A few studies, however, have shown that this can happen if the animal is prevented from emitting the lower response activity at its baseline level. Any response, therefore, can be reinforcing (Timberlake & Allison, 1974). • And you don’t get the reinforcement effect if the contingent response can be emitted at the baseline level.

40. Summary • Need-reduction and drive-reduction theories could not provide satisfactory explanations of reinforcement. • Premack has changed how reinforcement is regarded: Instead of a stimulus/event, it is regarded as a response. • Response-deprivation hypothesis is one example of how Premack’s principle has been refined.