Chapter 9 – Extinction of Conditioned Behavior

1. Chapter 9 – Extinction of Conditioned Behavior • Outline • Effects of Extinction Procedures • Decreased responding • Increased variability in responding • Extinction of Original Learning • Spontaneous Recovery • Renewal of Original Excitatory Conditioning • Reinstatement of Conditioned excitation • Enhancing Extinction • Number and Spacing of Extinction Trials • Reducing Spontaneous Recovery • Reducing Renewal • Compounding Extinction Stimuli • What is learned in Extinction • Paradoxical Reward Effects • Mechanisms of the Partial-Reinforcement Extinction Effect

2. So far we have focused on acquisition effects. • What happens when a stimulus predicts the presence of some outcome • This chapter focuses on what happens when that outcome is later omitted • Pavlovian • Acquisition • CS-US • Extinction • CS alone • Instrumental • Acquisition • S+ • Barpress/keypeck RF • Extinction • Barpress/keypeck  no longer RF

3. Extinction is a hot area of research • Particularly relevant to therapy • Exposure therapies for phobias • Extinguish fear • Also for drug addictions • Extinguish cues for drug taking behavior • Effects of Extinction Procedures • Domjan’s key example • Key no longer works for door • What do you do? • Try several times • Try it in a variety of ways • Jiggle it • Eventually quit

4. Effects of Extinction • Decrease in responding • Increase in variability of responding • Empirical Evidence • Neuringer, Kornell, and Olufs (2001) • Right lever(R), left lever (L), key (K) • Three responses required • Group Var • Not allowed to repeat • Yoked control • No variability requirement but RF was yoked to Group Var

5. Train Extinction Var Variability control control Var Responding

6. Extinction also can cause a strong emotional response • Frustration (possible aggression) • Car won’t start • Vending machine doesn’t work • Pigeon’s out of food • Pigeon’s key pecking with restrained partner • Extinction = attack

7. Extinction and Original Learning • Does extinction erase original learning • Evidence says no • From several lines • Spontaneous Recovery • Renewal • Reinstatement • Retention of knowledge of the Reinforcer

8. Spontaneous Recovery • Phase 1 • acquisition • Train CS-US • Phase 2 • extinction • CS alone • Phase 3 • Time off • Phase 4 • Extinction • What does Spontaneous recovery tell us about extinction learning? • Original learning remains • A little time off? • Responding returns

9. Renewal • A shift in context can renew extinguished learning • bring back responding • Demonstrated by Bouton and King (1983) • used the conditioned suppression procedure. • Phase 1 • Train all rats to bar press • Phase 2 • Train conditioned emotional response to a CS (tone) • CS (tone)  US (shock)

10. Phase 3 (Extinction) • 3 Groups • Group Ext A) • Extinguish CER same context as phase 1 and 2 • Present CS alone in same context • Group Ext B) • Extinguish CER different context • Present CS alone in different context • Group NE • No extinction • Test • Conditioned suppression to tone in original context • Result • Ext A? • No Fear • Ext B? • Fear • NE? • Most Fear

12. Which group demonstrated Renewal? • What does this say about Extinction? • 1) Initial learning is not forgotten • 2) Extinction is at least somewhat context specific • Has implications for therapists attempting to extinguish unwanted behaviors • Extinction of phobia or drug taking behavior may be specific to the therapists office • Extinction in multiple contexts?

13. Reinstatement • Exposure to the US serves as a reminder • reinstates an extinguished response. • Train • CS (tone)  US (shock) • tone elicits fear. • Extinction • CS alone. • Reinstatement • US alone • Test • Fear tone? • Yes = reinstatement

14. Like Renewal, Reinstatement is context specific • US exposure works best if in the same context • Reinstatement is also an issue for therapists • Worry that extinguished fears/behaviors will return if exposed to certain reminder stimuli • Patient has intimacy issues because of abusive parents • Treated with therapy • Abusive encounter later in life • Reinstates intimacy issues?

15. Enhancing Extinction • Because Extinction can be so useful therapeutically,efforts have been made to enhance it • Number and Spacing of Extinction of Trials • More extinction is more effective • Makes sense • New learning after all • Massed trials are more effective than spaced trials • This effect seems to be temporary • Within session effect • Large spontaneous recovery

16. Reducing Spontaneous Recovery • Repeated spontaneous recovery sessions reduces the effect • If there are cues that are specific to extinction, those cues can be effective in reducing spontaneous recovery • Requires special extinction cues

17. Reducing Renewal • Providing extinction training in multiple contexts can reduce renewal effects • Extinction cues can reduce renewal • Same as for spontaneous recovery • patients asked to recall the context of extinction training (therapists office) showed reduced anxiety in novel locations • Prompted some therapists to specifically train “portable” extinction cues • Memorize a specific “relaxation phrase” or carry a “relaxation card”

18. One view of extinction is that it is due to an increase in frustration that interferes with normal responding • Paradoxical Reward Effects • support the acquired frustration view of extinction • Whenever expectancy of reward is greatest following training, extinction occurs fastest • The Paradox? • Better acquisition causes faster extinction • we might expect that the better learning would slow extinction • Not the case

19. Overtraining extinction effect • More training = faster extinction • animal is more sure that reward is forthcoming • causes increased frustration when the reward does not come.

20. Magnitude of reinforcement effect • Bigger reinforcers = faster extinction • Animals are far more frustrated when they miss out on big reinforcers • Leads to faster extinction • Would you behave the same? • If you expect a big bonus at work and don’t get it? • Might slow you down quite a bit • May not affect you much if you miss out on a small bonus

21. Partial reinforcement extinction effect (PREE) • Partial reinforcement slows extinction. • Much faster extinction following continuous reinforcement (CRF) • An animal that expects a reinforcer after every response quickly becomes frustrated.

22. PREE has received a lot of research attention • Has real world relevance • Gambling behavior • Widely dispersed pay offs make responding (gambling) very resistant to extinction • Keep playing despite a lot of loss • Parenting • Giving in after repeated (annoying) requests for candy/toys ensures that future annoying requests will persist.

23. Mechanisms of the Partial-Reinforcement Extinction Effect • Three hypotheses • Discrimination hypothesis • Frustration Theory • Sequential Theory

24. Discrimination hypothesis • Extinction is easier to detect following continuous reinforcement • When reinforcement only occurs every so often maybe you don’t notice extinction for a while. • Turns out this isn’t it. • Jenkins (1962) Theios (1962) • Phase 1 • Train animals • Group 1 • Partial RF • Group 2 • CRF • Phase 2 • Put all animals on CRF for a while • Test • Extinction • Still more responding in Partial RF group • Extinction should have been equally discriminable for both groups. • Seems animals learn something longer lasting • Perhaps it teaches them not to give up?

25. Frustration Theory (I consider this a molar theory) • Amsel • Persistence in extinction occurs because the animals have learned to make responses even when they expect nonreward (or are frustrated). • Breaking down frustration theory • Partial RF • Sometimes animals get rewarded when they don’t expect it • Early reward on a variable ratio schedule (after a few responses) = surprised • They also sometimes get rewarded when they are frustrated • Late rewards on a variable ratio schedule (after many responses) • Thus animals on partial RF schedules learn to respond even when they don’t expect reward and even if frustrated • Animals on CRF never learn this

26. In addition frustration can be viewed as a kind of drive in Amsel’s theory • It energizes behavior • Respond in order to reduce frustration • Daly (1969) • Trained rats to expect food in a goal box • Then stopped feeding them there • frustrated rats. • Allowed them to jump a hurdle to escape the goal box • rats learned to jump the hurdle • No other RF was available • Presumably jumped to escape frustration • Negative RF • Shows that frustration reduction can motivate behavior. • We previously discussed the differential outcomes procedure • Matching to sample • One choice was reinforced with no food • Animals still learn the correct response • Could be to avoid the frustration of repeating the trial

27. Keep in mind, that continued responding in the face of frustration is a human characteristic as well • We often value it • Songwriter continues to work in the face of multiple rejections • Sometimes it can be a negative • Gambling in the face of continued losses

28. Sequential Theory (I consider this a molecular theory) • Capaldi • Much of Capaldi’s work is done in a straight alleyway • To examine PREE sometimes there would be food at the end (R) and sometimes not (N) • Capaldi examines behavior across specific sequences of trials • RNNRRNR • The underlined trials represent rewarded trials that were preceded by nonrewarded trials • Thus the rat will have a recent memory of an N trial that is essentially reinforced. • Leading to persistence in extinction • The longer the strings of Ns that the animal has experienced that eventually lead to RF • The more resistance to extinction

29. Both theories are likely correct. • Frustration theory seems to do a better job of explaining PREE when trials are spaced out • Long ITIs • Sequential theory does better when trials are close to one another (easy to remember) • Short ITIs