Decision Dynamics and Decision States in the Leaky Competing Accumulator Model

1. Decision Dynamics and Decision Statesin the Leaky Competing Accumulator Model Jay McClellandStanford University With Juan Gao, Marius Usher and others

2. A High-Stakes, Time-Critical Decision • A diffuse form is coming toward you rapidly: What should you do? • You could shoot at it, but it may be your friend • You can hold your fire, but it might shoot you! • You could wait to decide, but that might be risky too • How do we choose, and how well can we optimize our choices, under time pressure, with uncertain information?

3. A Classical Model of Decision Making:The Drift Diffusion Model of Choice Between Two Alternative Decisions • At each time step a small sample of noisy information is obtained; each sample adds to a cumulative relative evidence variable. • Mean of the noisy samples is +m for one alternative, –m for the other, with standard deviation s. • When a bound is reached, the corresponding choice is made. • Alternatively, in ‘time controlled’ or ‘interrogation’ tasks, respond when signal is given, based on value of the relative evidence variable.

4. The DDM is an optimal model, and it is consistent with neurophysiology • It achieves the fastest possible decision on average for a given level of accuracy • It can be tuned to optimize performance under different kinds of task conditions • Different prior probabilities • Different costs and payoffs • Variation in the time between trials… • The activity of neurons in a brain area associated with decision making seems to reflect the DD process

5. Neural Basis of Decision Making in Monkeys (Shadlen & Newsome; Roitman & Shadlen, 2002) RT task paradigm of R&T. Motion coherence anddirection is varied fromtrial to trial.

6. Neural Basis of Decision Making in Monkeys: Results Data are averaged over many different neurons that areassociated with intended eye movements to the locationof target.

7. Easy Prob. Correct Hard A Problem with the DDM • Accuracy should gradually improve toward ceiling levels as more time is allowed, even for very hard discriminations, but this is not what is observed in human data. • Two possible fixes: • Trial-to-trial variance in the direction of drift • Evidence accumulation may reach a bound and stop, even if more time is available

8. Usher and McClelland (2001)Leaky Competing Accumulator Model y2 y1 I1 I2 • Addresses the process of decidingbetween two alternatives basedon external input, with leakage, mutual inhibition, and noise: dy1/dt = I1-gy1–bf(y2)+x1 dy2/dt = I2-gy2–bf(y1)+x2 f(y) = [y]+ • Participant chooses the most active accumulator when the go cue occurs • This is equivalent to choosing response 1 iff y1-y2 > 0 • Let y = (y1-y2). While y1 and y2 are positive, the model reduces to: dy/dt = I-ly+x [I=I1-I2; l = g-b; x=x1-x2]

9. Wong & Wang (2006) ~Usher & McClelland (2001)

11. Time-accuracy curves for different |k-b| or |l| |k-b| = 0 |k-b| = .2 |k-b| = .4

12. Prob. Correct

13. Kiani, Hanks and Shadlen 2008 Random motion stimuli of different coherences. Stimulus duration follows an exponential distribution. ‘go’ cue can occur at stimulus offset; response must occur within 500 msec to earn reward.

14. The earlier the pulse, the more it matters(Kiani et al, 2008)

15. These results rule out leak dominance Still viable X

16. The Full Non-Linear LCAi Model y1 y2 Although the value of the differencevariable is not well-captured by thelinear approximation, the sign of thedifference is approximated very closely.

17. Three Studies Related to these Issues • Integration of reward and payoff information under time controlled conditions • Gao, Tortell & McClelland • Investigations of decision making with non-stationary stimulus information • Usher, Tsetsos & McClelland • Does the confidence of a final decision state vary continuously with the strength of the evidence? • Lachter, Corrado, Johnston & McClelland

18. Payoff Information and Decision Dynamics • How are reward asymmetries integrated into the decision making process? • What would be optimal, how close to optimal can decision makers come, and can deviation from optimality be explained by the LCAi model?

19. Timeline of the Experiment

20. Proportion of Choices toward Higher Reward

21. Sensitivity varies with time

22. Optimal vs. Actual Bias

23. Incorporating Reward Bias in the Competing Accumulator Model • First in the one-dimensionalmodel • Then in the full non-linear model

24. Three Hypotheses • Reward acts as an input from reward cue onset til the end of the integration period • Reward influences the state of the accumulators before the onset of the stimulus • Reward introduces an offset into the decision

25. Matches the pattern of the data!

26. Consistent Evidence from Physiology (Rorie et al, 2010) HL HH

27. Fits Based on Linear Model

28. Fitted Parameters How optimal is each S’s Yr given the other parameters?

29. Short Long Average

30. Fits based on full LCAi

31. Relationship between response speed and choice accuracy

32. Different levels of activation of correct and incorrect responses in Inhibition-dominant LCA Final time slice errors correct

33. High-Threshold LCAi

34. Preliminary Simulation Results

35. Three Studies Related to these Issues • Integration of reward and payoff information under time controlled conditions • Gao, Tortell & McClelland • Investigations of decision making with non-stationary stimulus information • Usher, Tsetsos & McClelland • Does the confidence of a final decision state vary continuously with the strength of the evidence? • Lachter, Corrado, Johnston & McClelland

36. Decision making with non-stationary stimulus informationUsher, Tsetsos & McClelland (in prep) • Participants viewed 6-10 sec displays of four flickering dots • Brightness varied around a mean, and the means alternated between phases of random durations. • Participant had to choose which dot was brightest overall • In correlation condition, there is no correct answer

37. Example trials from the three *’d conditions

38. Three Models Race: Best – Avg. Diffusion: LCA: In all models, choice goes to most active alternative at end of trial;In Race and B-AD, we consider the possibility that a bound isreached before the end of the trial. If so, choose the alternative thatreaches the bound first.

39. Model predictions for the effect of consistency LCA: High L,I Low L,I

40. Simulations of Two Correlated TrialsTop: A/B start high Bottom: C starts high

41. Preference for the dissimilar alt. in the 3 modelsDissimilar favored firstDissimilar favored secondAverageTop: low noiseBottom: higher noise

42. Group data and best fits for each of the models Consistent and Inconsistent Conditions Correlated Condition Race model looses; to capture the consistency effect even approximately, it over-predicts a primacy effect in the correlated condition LCA and Diffusion do about equally well, but neither is a perfect fit to the data LCA fit is slightly better, even accounting for the additional parameters

43. Performance in the correlated condition for individual participants and with varying parameter values in each of the models A perfect integrator should choose the first alternative 65% of the time (+), since it tends to receive slightly more overall evidence. A few participants look a bit like perfect integrators. The indifference to order exhibited by several participants is striking. For Race and Diffusion, parameter values are chosen at random, but are restricted to values consistent with the range of participants’ consistency effects. [Grid search is underway.] Race is restricted to the extreme upper left (as we saw previously). Diffusion is also restricted above and to the left of optimal. LCA fit includes 3 levels of noise, variation in I/L ratio. LCA has more flexibility, can come close to most of the participants data, but not 1-2 participants in the upper right. Model predicts greater accuracy in predominant trials for those in upper right vs those at or below (.5, .5). prediction is confirmed (.83 vs. .73)

44. LCA simulations Ruled out region based on consistency Dot size corresponds to I/L Leak fixed at ~.15 Low Noise High Noise Very High Noise

45. Discussion • What are the sources of individual differences and are they stable or malleable? • Bounded Race and Best-Avg. Diffusion can’t fit most of the individual participants data without further modification • What modifications might allow one or both to work? • Are participants really using an LCA-like process or is something very different going on? • What about the two participants in the upper right corner? • LCA is complex and has considerable freedom to fit particular data patterns • Is all of this necessary? • Are there ways of ensuring that we aren’t just overfitting while still keeping the flexibility where needed?

46. Three Studies Related to these Issues • Integration of reward and payoff information under time controlled conditions • Gao, Tortell & McClelland • Investigations of decision making with non-stationary stimulus information • Usher, Tsetsos & McClelland • Does the confidence of a final decision state vary continuously with the strength of the evidence? • Lachter, Corrado, Johnston & McClelland

47. Continuous Report of ConfidenceLachter, Corrado, Johnston & McClelland (in progress) Observers had up to 10 sec to position joystick, then click to indicate response

49. Results and Descriptive Model of Data from 1 Participant