1 / 59

The partial reinforcement extinction effect (PREE)

The partial reinforcement extinction effect (PREE). Frode Svartdal University of Tromsø. Extinction: Basics. Extinction is defined in terms of a reinforcement process Extinction contingencies The stimulus (S R or US) is discontinued The learning contingency is discontinued

kory
Download Presentation

The partial reinforcement extinction effect (PREE)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The partial reinforcement extinction effect (PREE) Frode Svartdal University of Tromsø

  2. Extinction: Basics • Extinction is defined in terms of a reinforcement process • Extinction contingencies • The stimulus (SR or US) is discontinued • The learning contingency is discontinued • Extinction process • The conditioned response is reduced (strength, frequency, etc.) • Relearning, … not forgetting

  3. Extinction: Basics Operant conditioning Catania, 1984)

  4. Extinction: Basics Classical conditioning

  5. Factors affecting the extinction rate • In general: Fast acquisition / high rate of responding  fast extinction • Amount of reward • High  fast extinction • Variability • Stimulus • Response • Reinforcement • Some forms of learning do not extinguish (easily) • Evaluative conditioning (e.g., Diaz, Ruiz, & Beyens, 2005) = high ext. persistence

  6. Factors affecting the extinction rate • Partial Reinforcement Extinction Effect • Partial (Intermittent) Reinforcement (PRF)  increased extinction response • Continuous Reinforcement (CRF)  reduced extinction persistence

  7. First demonstrations Operant conditioning; free operant; rats; Skinner (1938) Classical conditioning; blink response; students; Humphreys (1939) 50% 100%

  8. Free operant Ferster & Culbertson, 1975

  9. EXTINCTION Free operant • Compared to CRF: • PRF  • higher asymptotes • more persistent • responding under • extinction PRF CRF

  10. Rats, maze running speed under extinction (Weinstock, 1954) PRF (30%) CRF

  11. Classical conditioning (rats): PREE Extinction 25% PRF response rate LOWER than CRF response rate 50% 100% 15%

  12. Classical conditioning; eyelid; human subjects (Svartdal & Flaten, in prep.)

  13. Operant conditioning; humans; Svartdal, 2003, Exp. 4

  14. Conclusions (… preliminary) • PREE is a very robust outcome • Measures & species • Bar pressing, rats • Maze running, rats • Pecking, pigeons • Blink reflex, humans, rabbits • … • Contingency • Operant/instrumental • Discrete trial • Free operant • Classical

  15. But… • How general is the PREE? • Reversed PREE observed under some conditions • Generalized PREE observed under some conditions • Alternative methods of analysis • Nevin (1988): ”PREE is an artefact because of wrong method of analzing extinction performance” • Response unit issue • PREE or not dependig on how the response is defined (Mowrer & Jones, 1945!

  16. Reversed PREE What happens if the subject is exposed to a mixture of PRF and CRF contingencies?

  17. Reversed PREE Pavlik & Carlton, 1965: Rats; bar pressing, free operant • Gr. 1: Single contingency; CRF • Gr. 2: Single contingency; PRF • Gr. 3: Two signalled schedules alternated for the same subjects; CRF + PRF

  18. Reversed PREE Conventional PREE

  19. Reversed PREE Reversed PREE

  20. Reversed PREE Pavlik & Carlton (1965): • Single reinforcement schedules (CRF vs. PRF) in between-groups experiments PREE • Two schedules (CRF vs. PRF) for the same subjects  Reversed PREE Other research • Reversed PREE observed • Generalized PREE (overall increased persistence, but no difference between conditions) • Conventional PREE rarely if ever observed in within-subjects manipulations of CRF - PRF

  21. If applied to a situation with a very specific schecule for a specific behavior  PREE Example: Single mother – child is begging for toys only from mom If applied to various situations with mixed contingencies  Reversed PREE Generalized PREE Example: Mother and father – child begs for toys from both PREE as a generalization: Ecological validity

  22. PREE as a generalization • Relevance to ADHD • Complex schedules  RPREE or GPREE, not PREE • Factors associated with slower learning  slower extinction • Attentional problems, difficulties with concentration, memory, … • Would add to biological factors

  23. Response unit issue

  24. Free operant responding: What is the response unit? Mowrer & Jones,1945: What should be counted as the response unit - single responses or the unit of responses required for reinforcement? • Free-operant • Intermittent reinforcemet, e.g., FR4

  25. Response unit FR4 Reinforced responses

  26. PREE Total responses Reversed PREE Total responses / reinforcement ratio

  27. Nevin: PREE is an artefact

  28. PREE: Alternative analyses Nevin, 1988: Behavioral momentum • ”RPREE” is the rule – the response is stronger following CRF • in free-operant responding (but not in discrete-trial experiments) • following extended training • Extinction performance • Traditional measure: Number of responses • Nevin: Slope of the extinction curve

  29. SHORT LONG Nevin, 1988 Absolute number of responses PREE RPREE Relative to initial ext response level

  30. PREE vs. RPREE – important variables • Dependent measure • No. of responses vs. relative change • Type of situations • Free operant vs. discrete trial • Complexity of situation • One vs. more schedules (e.g., multiple schedule) • Design • Between groups vs. within subjects

  31. PREE typically observed

  32. PREE: My interests • Interaction PREE & Reversed PREE • Cognition (verbalization) related to behavioral PREE

  33. The experimental situation ”Computer responses” presented Left, right Subject responses recorded Left, right

  34. The experimental situation Task • Complete a four-response chain of responses started by the computer • E.g.: Computer: L R Subject: R L • Instructed task: Identify and apply the functional rule(s) • ”Obtain as many correct answers as you can.” • Rules (depending on experiment) • ”Repeat computer sequence” • ”Reverse computer sequence” • Feedback (visual, autitory) for correct answer; nothing happens if answer is incorrect

  35. The experimental situation Manipulations (between groups and/or within groups) Rule Reverse (typically used) Repeat Contingency CRF (100%) PRF (20-60%)

  36. The experimental situation • Reward rate manipulated • Between groups • Within subjects (multiple schedule) • Discrete trial situation; fixed number of trials • 180 acquisition trials • 40 extinction trials

  37. Conventinal PREE; operant responding; students; Svartdal, 2003, Exp. 4

  38. Reversed & conventional PREE; operant responding; students; Svartdal, 2000 Reversed PREE • Purpose: Explore the relationship between PREE and RPREE • PREE vs. RPREE: Contradiction or compatible effects? • Method • Independent groups: PRF and CRF • Within: CRF and PRF

  39. Svartdal, 2000 ctd. • Multiple schedule, alternating • Group 40/40 • Half trials (signalled): 40% • Half trials (signalled): 40% • Group 80/80 • Half trials (signalled): 80% • Half trials (signalled): 80% • Group 80/40 • Half trials (signalled): 80% • Half trials (signalled): 40% PRF ”CRF” ”CRF”+ PRF

  40. * No. of responses: RPREE * Relative change: No difference 80% PREE 40%

  41. Svartdal, 2000 ctd. • Relationship between schedule components • Simplest assumption: Modulation between component schedules: • 60% + context = 60% reference • 60% + context = 100% reduced persistence • 60% + context = 20% increaced persistence

  42. Performance of a 60% schedule depending on other schedule = 100%, 60%, or 20% Svartdal, 2000

  43. Svartdal, F. (2000). Persistence during extinction: Conventional and Reversed PREE under multiple schedules.Learning and Motivation, 31, 21-40.

  44. Cognition in PREE • Currently: Strong cognitive arguments to interpret conditioning in terms of cognition • Classical conditioning: Lovibond & Shanks, 2002 • Operant conditioning: Shanks & St John, 1994 • Implicit learning doubted: Shanks, 2005 • Extinction: Lovibond, 2004 • Basic argument: CONTINGENCY  CONSCIOUS APPREHENSION  BEHAVIORAL CHANGE CONTINGENCY  CONSCIOUS APPREHENSION  NO BEHAVIORAL CHANGE • Large number of studies supporting this assumption

  45. Cognition in PREE • So, since the behvioral PREE is very robust, a ”cognitive PREE” must be easy to measure • Basic prosedure: • Behavioral acquisition under 100% vs. 60% reinforcer rate • Measurement of verbalized PREE

  46. Cognition in PREE Prediction of persistence: ”How likely is it that you will continue responding if reward no longer appears?” Several experiments have demonstrated no sensitivity to learning history in predictions

  47. 3 extinction trials; immediate behavioral sensitivity No difference in predictions Svartdal & Silvera, in prep.

  48. Cognition in PREE Retrospective judgments: ”How many responses did you emit after reward no longer appeared?” Subjects are very accurate in descrbing their own behavior, including their own extinction persistence

  49. Cognition in PREE Svartdal, F. (2003). Extinction after partial reinforcement: Predicted vs. judged persistence.Scandinavian Journal of Psychology, 44, 55-64.

  50. Meta-cognitive PREE? • We all have long experience with various contingencies • Maybe a ”meta-cognition” evolves: • Uncertain outcomes  Persist • Certain outcomes  Quit

More Related