The Maintenance of Behavior: Intermittent Reinforcement, Choice, and Economics

1. CHAPTER 10 The Maintenance of Behavior: Intermittent Reinforcement, Choice, and Economics

2. MAINTENANCE Once a repertoire of operant behavior has been established, what sustains it? What keeps already-acquired operants occurring? Behavior theory explains maintenance of behavior in terms of same behavior-outcome relations used to explain its acquisition.

3. MAINTENANCE Consistent reinforcement is usually vital to acquisition of operant behavior. But, inconsistent or intermittent reinforcement can effectively sustain previously-learned behavior. Schedules of intermittent reinforcement are central to understanding maintenance of operant behavior.

4. MAINTENANCE A further issue in maintenance of operant behavior: human activity involves choice among many possible operants and reinforcers. How do organisms allocate behavior in more complex environments than those we have considered so far?

5. MAINTENANCE One way of looking at choice: people have limited resources of time, energy, and money. One must decide how limited resources should be allocated. From this perspective, study of choice overlaps with discipline of economics.

6. MAINTENANCE Today, we focus on three questions: What are effects on behavior of intermittent reinforcement? How do organisms choose among alternative operants and outcomes? How can concepts from economics help us understand choice behavior?

7. Schedules of Reinforcement A reinforcement schedule is a rule specifying how often and under what conditions a response will be reinforced. Consistent or Continuous: CRF Intermittent: Fixed Interval, Variable Interval, Fixed Ratio, and Variable Ratio--basic types of reinforcement schedules.

8. Fixed-Interval (FI) Schedules On FI schedule, one response after fixed time period produces reinforcer. If subject could tell time precisely, then it could wait until time was up and make a single response. Organisms do not behave this efficiently; FI scallop suggests that they do learn to discriminate length of interval.

9. Fixed-Interval (FI) Schedules

10. Variable-Interval (VI) Schedules On Variable-Interval schedules, reinforcement also depends on passage of time and one response; but, time between reinforcers varies from reinforcer to reinforcer. Value of VI schedule is average time between reinforcers. Response rate is constant in time.

11. Variable-Interval (VI) Schedules

12. Fixed-Ratio (FR) Schedules On Fixed-Ratio schedule, reinforcement depends on completion of certain number of responses; time is irrelevant. Pause/respond pattern results. Postreinforcement pause if followed by high rate until reinforcer is obtained. Ratio strain is seen with very large ratios: breaks in ratio run.

13. Fixed-Ratio (FR) Schedules

14. Variable-Ratio (VR) Schedules On a Variable-Ratio schedule, number of responses required for reinforcement varies from one reinforcer to next. Response rate is very high and constant.

15. Variable-Ratio (VR) Schedules

16. CONCURRENT SCHEDULES OF REINFORCEMENT Study of choice centers on how operant behavior is affected its reinforcement history and by reinforcement history of other operant behaviors. Research uses concurrent schedules: two or more schedules that operate simultaneously and independently, each for a different response.

17. CONCURRENT SCHEDULES OF REINFORCEMENT Research with concurrent schedules has revealed fundamental law relating multiple behaviors and reinforcers. Matching Law.

18. Matching Law RA/(RA + RB) = rA/(rA + rB) or RA /rA = RB /rB RA = Responses to A RB = Responses to B rA = Reinforcers to A rB = Reinforcers to B

19. Matching Law Provides accurate description of behavior of many organisms in many different choice situations. Different species Appetitive and aversive stimuli Frequency, magnitude, and delay of reinforcement

20. Matching Law in Operation An important application of Matching Law is to cases where one outcome is immediate and other is delayed. Here, organisms should exhibit maladaptive choices: Small, immediate reinforcer over large, delayed reinforcer. Precisely this result is observed.

21. Matching Law in Operation Choice of large, delayed reinforcer is predicted and observed when constant delay is added after both choices. Result shows role of commitment to adaptive choice behavior. Early commitment is essential for exercise of self-control and eschewal of impetuousness.

22. Matching and Maximizing How does behavior come to conform to Matching Law? Two views: Matching (melioration) Maximizing No clear winner. Mazur (1981) is one interesting test.

23. Matching and Maximizing In Mazur (1981), reinforcers were probabilistically assigned to two keys. Reinforcers were held until collected. Maximizing suggests equal responding to two keys at all nonzero ps of reinf. Matching was nevertheless found. Such matching actually cost pigeons food reinforcers.

24. Choice and Foraging Laboratory paradigms are often criticized because they do not capture essence of natural contingencies. More recent research has tried to bring natural contingencies of reinforcement into laboratory. This move has been prompted by optimal foraging theory.

25. Choice and Foraging Optimal foraging theory: feeding behavior is sensitive to relation between amount of energy expended in finding, securing, and consuming food, and amount of energy or nutrition of food. Patterns of foraging optimize relation between energy gain and energy expenditure.

26. Choice and Foraging One test by Bhatt and Wasserman (1987). Choice was afforded between Fixed Ratio and Progressive Ratio schedules. PR is analogous to depletion. PR to FR choice is analogous to travel. Different FRs are analogous to better or worse alternative food sites.

27. Choice and Foraging PR: 10, 30, 50, 70, 90 FR: 20 FR: 40 FR: 60 FR: 80

28. Choice and Foraging Results of Bhatt and Wasserman (1987): Pigeons persisted longer on PR schedule at longer FRs. Changeovers were close to optimal switch points.

29. OPERANT BEHAVIOR AND ECONOMICS Viewed through lens of Matching Law, questions about choice become questions about how organisms allocate limited resources. Issues relating to how people allocate their limited resources--both behavior and money--have traditionally resided in economics.

30. OPERANT BEHAVIOR AND ECONOMICS It is not surprising that behavior theorists have applied economic concepts to their own domain. Matching Law is viewed by some theorists as special case of general economic principles.

31. OPERANT BEHAVIOR AND ECONOMICS: BASICS Demand (amount purchased at given price): Elastic (luxuries) Inelastic (necessities) Commodities: Substitutes (more of one, less of other) Complements (more of one, more of other) Income (money or responses required to purchase commodities at given price)

32. OPERANT BEHAVIOR AND ECONOMICS: BASICS Economy (relationship between commodities and income: Closed (fixed income) Open (extra income): most operant studies

33. OPERANT BEHAVIOR AND ECONOMICS Evidence suggests behavior of animals in conditioning experiments conforms to what economic theory says people will do when confronted with similar choices. Implies that operant behavior involves a kind of economic decision making. Animals must decide how to allocate scarce behavioral resources; rules by which they do so are like those people use. Economic theory and behavior theory may jointly explain human and animal choice.