The Learned Dog

1. The Learned Dog • Class 4: Pavlovian Conditioning

2. Agenda • Quick review of Single-Event Learning • Pavlovian Conditioning • Chicken Camp, Part 1.

3. Quick Review of Single Event Learning

4. Single Event Learning • Learning that the occurrence of an event predicts the future occurrence of the same event... • The subsequent occurrence of the future event becomes increasingly less surprising. • Surprised when something changes (dishabituation) • 2 Processes: • Habituation: increasingly smaller response to future occurrences of the same event • Sensitization: larger response to future occurrences of any type of event. The animal becomes “jumpy”.

5. Pavlovian Conditioning

6. Examples of Pavlovian Conditioning Experiments...

7. Example of Pavlovian Conditioning • Unconditioned Stimulus (US) • Pre-existing trigger (innate or prev. learned) • Unconditioned Response (UR) • What animal does in response to US • Conditioned Stimulus (CS) • New trigger that becomes associated with pre-existing trigger • Conditioned Response (CR) • What animal does in response to CS Schwartz, B., E. A. Wasserman, et al. Psychology of Learning and Behavior. New York, NY, W.W. Norton & Co.

8. Yikes, all this terminology... • Remember SWR’s analogy of road signs and turns on a mountain road • The mountain road as the experimental context • Road sign -> CS • Turn -> US • Slowing down for turn -> UR • Slowing down because you saw the sign -> CR

9. Schwartz, B., E. A. Wasserman, et al. Psychology of Learning and Behavior. New York, NY, W.W. Norton & Co. Classic pavlovian conditioning experiment

10. Yikes all this terminology, another slice... • Think ‘innate’ or ‘pre-existing’ when you hear/read ‘unconditioned’ • Think ‘learned’ or ‘new’ when you hear/read ‘conditioned’ • What is the difference between ‘unconditioned response’ & ‘conditioned response’? Its pretty academic... • Typically the same motion/behavior, but one is done in response to the ‘unconditioned stimulus’ and one is done in response to the ‘conditioned stimulus’. • The drop of saliva that appears prior to appearance of the food, is a ‘conditioned response’, because it can only be in response to the CS.

11. Brief aside: so what is a stimuli anyway...

12. So what is a stimulus/stimuli anyway? • From answers.com: “Something causing or regarded as causing a response.” • Some feature of an animal’s perceptual environment. Things to think about... • What modality (visual, auditory, olfactory, taste, touch...) & intensity? • Species-specific biases! • How localized in space and time (lightning vs. hot and humid)? • How frequently does it occur? • If it doesn’t cause a response, did the animal perceive it? How do we know?

13. So what is a salient stimulus? • Salience means that for some reason, the stimulus ‘stands-out’. • Salience is very much in the mind of the perceiver and our intuition is often a very bad guide as to which stimuli will be salient to a given animal.

14. What are the stimuli in a traditional experimental set-up? • Context: Some set of the features that make up the experimental enclosure, i.e., the cage. • Unconditioned Stimulus: A known salient stimulus/trigger with a short duration, e.g., food appearing, a shock... that is associated with an identifiable response by the animal. • Conditioned Stimulus: A novel, but presumably salient stimulus/trigger, e.g., a light or tone that comes on for some period of time & then turns off. • Learning experiments typically vary when this novel stimulus/trigger comes on, and for how long, in relation to the known stimulus and then observe how animal’s response changes

15. Examples of Pavlovian Conditioning Experiments, cont...

16. Schwartz, B., E. A. Wasserman, et al. Psychology of Learning and Behavior. New York, NY, W.W. Norton & Co. Acquisition & Extinction curves: theory and practice • Nature loves ‘S’ curves

17. Schwartz, B., E. A. Wasserman, et al. Psychology of Learning and Behavior. New York, NY, W.W. Norton & Co. Fear conditioning as a way of studying Pavlovian conditioning • Looking for suppression of a learned behavior in response to a learned trigger that predicts bad things

18. Schwartz, B., E. A. Wasserman, et al. Psychology of Learning and Behavior. New York, NY, W.W. Norton & Co. Conditioned key-pecking as a way of studying P.C. • Light/tone is the learned trigger, food delivery is innate trigger

19. Examples of Pavlovian Conditioning Experiments: associative bias...

20. Taste Aversion • Pre-test: rats simultaneously experience taste, sound & light as they lick water bottle • Some are made sick via poison others experience shock • In test, rats that experienced • poison, suppressed licking in presence of taste only • shock, suppressed licking in presence of light & tone Schwartz, B., E. A. Wasserman, et al. Psychology of Learning and Behavior. New York, NY, W.W. Norton & Co.

21. Taste aversion experiments as an example of associative bias... • Example of associative bias that reflects species-specific adaptation. Pigeons are another example... • Light and tone presented together as learned triggers • For one group, food was the pre-existing trigger, for another shock • When responses to light or tone presented individually... • Group that was shocked responded to tone more (alarm calls) : bias to associate sound with danger. • Group that was given food responded to light more (visual feeders): bias to associate visual stimuli with food

22. Temporal patterns are a big source of associative bias...

23. Temporal Relationships • Standard Pairing • Onset of CS precedes onset of US • Duration of CS roughly the same as duration of US • CR starts soon after onset of CS Schwartz, B., E. A. Wasserman, et al. Psychology of Learning and Behavior. New York, NY, W.W. Norton & Co.

24. Temporal Relationships • Delay conditioning • Onset of CS precedes onset of US by a large amount, and may offset before onset of US • Duration of CS substantially longer than the duration of the US • Initially response starts after onset of CS but over time moves closer to onset of US Schwartz, B., E. A. Wasserman, et al. Psychology of Learning and Behavior. New York, NY, W.W. Norton & Co.

25. Temporal Relationships • Simultaneous Pairing • Onset and duration of CS is the same as that of US • Typically the animal does not learn to respond to the CS in this case Schwartz, B., E. A. Wasserman, et al. Psychology of Learning and Behavior. New York, NY, W.W. Norton & Co.

26. Temporal Relationships • Temporal Conditioning • The US is presented at regular intervals, but the CS is never presented. • Animals start responding based on the expected interval Schwartz, B., E. A. Wasserman, et al. Psychology of Learning and Behavior. New York, NY, W.W. Norton & Co.

27. Temporal Relationships • Backward Conditioning • The onset of the CS is after that of the US and its duration is greater • Animals do not respond to the CS, it is as if they learn that the CS means the US is over. Schwartz, B., E. A. Wasserman, et al. Psychology of Learning and Behavior. New York, NY, W.W. Norton & Co.

28. Temporal Relationships • Second Order Conditioning • CS1 initially paired with US • Once animal is responding to CS1, CS2 is paired with CS1 • Animal starts responding to CS2 • This is why ‘pre-existing’ may be a better word than ‘innate’ or ‘unconditioned’ Schwartz, B., E. A. Wasserman, et al. Psychology of Learning and Behavior. New York, NY, W.W. Norton & Co.

29. Making sense of what associations get made and which don’t...

30. In general, for an association to be learned, it must be proximate in space & time, and have some incremental predictive value, or differential contingency...

31. Predictive value, or differential contingency • Does knowing that event A occurred help you predict the probability of event B occurring in the future? If so, A has predictive value. • In math speak: P(B|A) which is read as “the probability of B given A”. • Suppose P(rain| no rain in forecast) = 0 and P(rain| rain in forecast) = 1 • Hearing the forecast definitely helps you predict the probability of rain • Now, suppose P(rain| no rain in forecast) = P(rain | rain in forecast) = .8. • Hearing the forecast, does not help you predict the probability of rain. The forecast has no predictive value.

32. Predictive value, or differential contingency, cont... • Does knowing that event A occurred help you predict the probability of event B occurring in the future? If so, A has predictive value. • Suppose P(rain| no rain in forecast) = 1 and P(rain| rain in forecast) = 0 • Hearing the forecast definitely helps you predict the probability of rain, although in this case the forecast is 100% wrong, but reliably wrong. • Suppose P(rain| rain in forecast) = .55 and P(no rain| rain in forecast) = .45 • It pays to listen to the forecast even though it is only marginally better than guessing

33. In general, for an association to be learned, it must be proximate in space & time and not only have some predictive value, but it must also provide incremental predictive value over and above any other cues the animal is using to predict the future...

34. Incremental predictive value • Suppose... • P(rain| charlie says it is going to rain) = .7 • Does it pay to listen to charlie? • P(rain| rain is forecast) = .6 : Yes, because in this case Charlie is better than the forecast at predicting rain. • P(rain| rain is forecast) = .9 : No, because in this case Charlie is worse than the forecast at predicting rain. • P(rain| rain is forecast) = .7 : No, because he isn’t telling you anything you don’t already know.

35. In other words, animals bring a slacker mentality to learning associations between triggers (stimuli...) • The rule: “only bother to learn an association if it easy to do so, and it has incremental predictive value with respect to answering two questions... • Will it let me better predict if some important trigger is going to occur, or conversely not occur? • Will it let me better predict when that event is going to occur, or not occur? • If you keep these two questions in mind, you can pretty much predict which associations animals are likely to learn and which they aren’t (ignoring perceptual biases, etc...)

36. CS1 Animal learns to respond to CS1 Step 1 US CS2 Animal never learns to respond to CS2. Step 2 CS1 US CS2 CS1 Test CR CR Blocking • CS2 doesn’t tell animal anything that CS1 doesn’t already tell it, so animal doesn’t bother learning CS2 even though it is an equally good predictor!!!

37. Animal learns that CS1 is generally predictive of US CS1 Step 1 US Animal learns to respond to CS2. because it localizes when US will occur CS2 Step 2 CS1 US But blocking doesn’t occur in this case • In this case, CS2 provides additional information about when the US will occur, so it pays to learn CS2

38. The CS as a ‘temporal localizer’ of the US. • The pigeon experiment: key illumination of 5 sec, 10 sec, 20 sec, and 30 sec with time between illumination of 30 sec. Food appears at some point during key illumination... • The greater the difference between key illumination & time between illumination, the faster the pigeons learn to peck the key • As SWR put it:”... the ability of a CS to localize the time of the US relative to other cues is what is critical to conditioning... it is not the absolute duration of a trial that is critical to conditioning; rather, it is the ratio of the trial duration to the duration of the inter-trial period.

39. CS1 Animal learns to respond to CS1 Step 1 US CS2 Animal learns that US never appears when CS2 &CS1 appear together Step 2 CS1 US CS2 CS1 Test CR CR Or in this case (summation)... • Animal acts as if it assigns blame totally to CS2 for US not appearing.

40. CS1 Animal learns to respond to CS1 Step 1 US CS2 Animal learns that the magnitude of US is greater in presence of CS1 & CS2 Step 2 CS1 US CS2 CS1 Test CR CR Or in this case ( another form of summation) • CS2 is learned because it provides additional information above and beyond CS1 about the magnitude of the US

41. CS2 CS1 & CS2 are initially presented as a compound stimulus CS1 Step 1 CR CS1 CR Or Step 2 CS2 CR Overshadowing • If CS2 & CS1 are always presented together, animal acts as if it assigns all of the predictive value to one or the other

42. Practical examples of overshadowing & blocking... • Sydney and “down” • Scuppers and “away”

43. Inhibition... • Most of what we have described previously with respect to excitory stimuli apply in the case where the presence of a stimuli reduces the likelihood of a response. • Studying inhibition requires creating a baseline level of activity and then introducing a putatively inhibitory stimulus and see if the level of activity goes down. • Inhibitory learning occurs when: P(US|CS) < P(US| (no CS)). This is just a form of summation learning. As SWR put it: “just as differential positive contingency is required for excitatory conditioning, a differential negative contingency is necessary for inhibitory conditioning.

44. The big picture... • Think road signs and turns on a mountain road • Road sign -> CS • Turn -> US • Slowing down for turn -> UR • Slowing down because you saw the sign -> CR

45. The big picture, cont. • Animals are constrained by species-specific biases as to what stimuli they will attend and as to the kinds of connections that are easy or hard for them to make. • Temporal patterns are a big source of associative bias because they reflect how the world works... • Animals learn just enough to get by as opposed to acing an IQ test: • Will learning this association (i.e., making this connection) tell me anything I don’t know already? • Pavlovian learning is the basis for cue learning and for much of your animal’s emotional response to its world.

46. The complication: • We can never observe the association directly, we can only observe some behavior that we assume is a consequence of the association. Or as SWR put it... • “We assume, in Pavlovian Conditioning, that CS-US pairings lead to the formation of an association, which in turn leads to the occurrence of CRs. In actual experiments, it is the occurrence of CRs that tells us that an association has been formed... We can never see associations directly; we can only see the behavior presumed to reflect them.” • Pavlovian conditioning is all about associations between stimuli (environmental triggers) but we can only infer the associations by looking at the response. Schwartz, B., E. A. Wasserman, et al. Psychology of Learning and Behavior. New York, NY, W.W. Norton & Co.

47. Why should you care? • Pavlovian Conditioning is at the heart of an animal’s learned emotional response to its world. • When you associate a cue with a behavior you are relying on Pavlovian conditioning. • So chapters 3 & 4 are useful to you in helping you to think about what kinds of associations will be easy for your dog to make and which will be hard.