Extinction of Conditioned Behavior

1. Effects of Extinction Extinction of Conditioned Behavior • the rate of responding decreases • response variability increases • experiment by Neuringer, Kornell, & Olufs (2001)

2. Extinction does not eliminate the original learning • Spontaneous Recovery • Renewal • Reinstatement • US-devaluation

3. What is learned during extinction? The non-reinforcement of a response in the presence of a specific stimulus produces an inhibitory S-R association that suppresses that response whenever S is present

4. Partial Reinforcement Extinction Effect Extinction is slower if a partial reinforcement schedule, rather than CRF, was in effect before extinction Possible explanations: • Discrimination hypothesis • Frustration hypothesis • Sequential memory hypothesis

5. Stimulus Control of Behavior • Stimulus control • Stimulus discrimination • Generalization Gradient

6. Stimulus and Response Factors in Stimulus Control • sensory capacity • relative ease of conditioning various stimuli • overshadowing • type of reinforcement • certain types of stimuli are more likely to gain • control over the instrumental response with positive • reinforcement than with negative reinforcement • experiment by Foree & LoLordo (1973) • type of instrumental response • experiment by Dobrzecka, Szwejkowska, & • Konorski (1966)

7. Foree & LoLordo (1973) 2 groups of pigeons trained to press a foot treadle in the presence of a LT compound stimulus For 1 group of pigeons, reinforcement was food For the other group of pigeons, reinforcement was the avoidance of shock

8. Foree & LoLordo (1973) Pigeons trained with food reinforcement responded much more when tested with the light alone than when tested with the tone alone Pigeons trained with shock-avoidance reinforcement responded much more when tested with the tone alone than when tested with the light alone Selective association?

9. Dobrzecka, Szwejkowska, & Konorski (1966)

10. Dobrzecka, Szwejkowska, & Konorski (1966) Two components to the auditory cues: - quality - location Dogs trained on the left/right discrimination responded mainly on the basis of the location of the auditory cues Dogs trained on the go/no go discrimination responded mainly on the basis of the quality of the auditory cues Selective association?

11. Stimulus elements versus configural cues So far, we’ve assumed that subjects treat stimulus elements as distinct and separate features of the environment (i.e., quality and location of sounds) However, subjects could treat a complex stimulus as an integral whole that is not divided into parts or elements Configural-cue approach Stimulus-element approach

12. Learning Factors in Stimulus Control Effects of Training on Generalization and Discrimination • Nondifferential Training : • - S+ always present. • Differential (or Discrimination) Training: - Presence/Absence Training: • * reinforced in the presence of S+, not in its absence. • - Intradimensional Training: • * reinforced in the presence of S+ and not reinforced in the presence of S-

13. Flat gradient Non-Differential More peaked gradient Less generalization; more discrimination Presence/Absence Most peaked gradient Intradimensional Least generalization; most discrimination

14. What is learned in discrimination training? • respond during the S+ • do not respond during the S- • or both (i.e., Spence)

15. Contextual Cues and Conditional Relations Control of behavior by discrete stimuli (i.e., L or T) - discrete cues are those which are present for a brief time and have a clear beginning and end Contextual cues are the visual, auditory, and olfactory cues of the room or place where the discrete stimuli are presented Contextual cues can provide an additional source of control of learned behavior

16. Control by Contextual Cues Conditioned Place Preference barrier Rats given drug and put in Black side Rats given saline and put in White side

17. On test: Barrier removed Rats placed in the center

18. On test: Rats spend more time in drug paired context

19. Contextual cues can control behavior if they serve as a signal for a US or reinforcer - experiment by Akins (1998) described on p. 243 Do contextual cues control behavior when they serve as background rather than explicit signal for reinforcer?

20. Experiment by Thomas, McKelvie, & Mah (1985) Trained pigeons on a line-orientation discrimination in context 1 Context 1: S+ = vertical line (90º) S- = horizontal line (0º) After the discrimination was learned, the context was changed and the discrimination training contingencies were reversed Context 2: S+ = horizontal line (0º) S- = vertical line (90º) After learning the second problem, pigeons given generalization tests in which lines of orientation between 0º and 90º were presented in the two contexts

21. Results:

22. Thomas et al (1985) showed that: • contextual cues can come to control instrumental behavior • contextual control can occur without one context being more strongly associated with reinforcement than the other • direct associations between context and reinforcer cannot explain the pattern of results • each context associated with a different S+/S- contingency • Conditional Relations

23. Control by Conditional Relations Binary relations: • S-R • S-O • R-O In some cases the nature of a binary relation is determined by a third event = modulator • S-(R-O) This is termed a conditional relation S+/S- discriminations are conditional relations

24. Conditional Control in Pavlovian Conditioning Rescorla, Durlach, & Grau (1985) Nonreinforced trials No Noise Key light – no food Reinforced trials Noise Key light - food

25. Conditional Control in Pavlovian Conditioning In instrumental conditioning, modulators (i.e., S+ and S-) are called discriminative stimuli In Pavlovian conditioning, modulators are called facilitators or occasion setters • A positive occasion setter signals that a CS-US contingency is in effect • A negative occasion setter signals that a CS-US contingency is not in effect

26. Positive Occasion Setting (Facilitation) Training If Occasion Setter is present: then CS 1 US If Occasion Setter is absent: then CS 1 no US Result Thus CS 1 CR only when Occasion Setter is present

27. Distinction between excitation and modulation 1. CS signals US; BUT occasion setter signals the relationship (if - then) between CS and US. 2. CS elicits CR; BUT occasion setter does not elicit a CR 3. CS does not facilitate responding to a new CS+; BUT an occasion setter can. 4. CS can support second order conditioning, BUT occasion setter does not. 5. CS presented alone results in extinction of CR; BUT occasion setter presented alone, no extinction of CS " CR

28. Negative Occasion Setting Training If Occasion Setter is present: then CS 1 no US If Occasion Setter is absent: then CS 1 US Result Thus CS 1 CR only when Occasion Setter is absent – is this familiar? Conditioned Inhibition may be opposite of facilitation, not excitation