There is a causal relationship between actions and outcomes

1. If behavior was dominated in the past by Hull’s S-R reinforcement paradigm, what paradigm is it dominated by today? • There is a causal relationship between actions and outcomes • There is some degree of control over actions according to the anticipation of, and desire for, the outcome • The circuits in the BG allow for control and expression of this paradigm (i.e. via the A-O and S-R systems later discussed) Clark L. Hull Edward Thorndike

2. Explain the two important contributors to behavior: the remembered value of the expected outcome, and the knowledge of the causal relationship between the action and the outcome. • “remembered value of the expected outcome” • We can learn the reward of an action and respond/act accordingly • “knowledge of the causal relationship between the action and the outcome” • We know that there is a relationship between how we act and what the outcome will be. • This allows us to place value on certain actions and the rewards we receive from them

3. Explain the two important contributors to behavior: the remembered value of the expected outcome, and the knowledge of the causal relationship between the action and the outcome. • “remembered value of the expected outcome” • Our sensitivity to manipulations of outcome value determines whether an action is habitual or goal-directed • “knowledge of the causal relationship between the action and the outcome” • This acknowledges the intentionality of behavior itself

4. Explain the experimental paradigms used to study these two psychological functions. • The value of the outcome is increased(inflated) or decreased(devalued). • Devaluation is far more common because it is easier to reduced the value of a reward. • If performance changes based on outcome value, then the behavior is controlled by the anticipation of the outcome (goal directed action)

5. Explain the experimental paradigms used to study these two psychological functions. • The Action-outcome (A-O) contingency is manipulated. (outcome depends on action) • Often done through contingency degradation • Introduces free rewards that are independent of any action • Instrumental contingency can be viewed as the probability of reward given a particular action relative to the probability of a reward given no action. • If the probabilities are the same, then the contingency is said to be completely degraded.

6. The A-O and S-R Systemsand Habit Formation • A-O : Action-Outcome • S-R: Stimulus-Response • Both engaged under different conditions • Amount of training or number of rewarded responses plays significant role in shift between systems-habit formation • Promoted by overtraining • Schedule of reinforcement also plays a role

7. 14. What are interval and ratio schedules? • Ratio Schedules: • Response results in a certain probability of a reward (more responses= more rewards) • Produce goal directed actions • Controlled by A-O contingency • A rat knows that a specific outcome will produce a reward, so the rat will direct it’s actions to reach that outcome • Interval Schedules: • A response is rewarded only after a certain time interval has passed • Produce stimulus response (S-R) habits

8. 14. How do they relate to research on BG function? • Consensus that A-O and S-R systems control instrumental behaviors • Engaged in different conditions • Amount of training determines the shift from A-O to S-R (habit formation) • Use of interval schedules in these studies explains why they failed to find evidence for A-O learning • Even when reinforcement was equated, interval schedules produced habits, and ratio schedules did not

9. 14. How do they relate to research on BG function? • So… • DMS, specifically the pDMS(posterior dorsomedial striatum) is involved in goal directed behavior (A-O learning) • DLS is involved in habit formation (S-R learning)