Dynamic Decision Making in Complex Task Environments: Principles and Neural Mechanisms

1. Dynamic Decision Making in Complex Task Environments:Principles and Neural Mechanisms Progress and Future Directions November 17, 2009

2. A High-Stakes, Time-Critical Decision • A diffuse form is coming toward you rapidly: What should you do? • You could shoot at it, but it may be your friend • You can hold your fire, but it might shoot you! • You could wait to decide, but that might be risky too • How can we optimize our choices, and the timing of our choices, in time-critical, uncertain situations?

3. FY07 MURI BAA06-028 Topic 15 Building Bridges between Neuroscience, Cognition, and Human Decision Making Objective: The general goal is to form a complete and thorough understanding of basic human decision processes … by building a lattice of theoretical models with bridges that span across fields …. The main effort of this work is intended to be in the direction of new integrative theoretical developments … using mathematical and/or computation modeling … accompanied and supported by rigorous empirical model tests and empirical model comparisons …. . From BAA 06-028, Topic 15

4. Our MURI Grant • Builds on past neurophysiological, behavioral and theoretical investigations of the dynamics of decision making in humans and non-human primates toward the development of an integrated theory. • Extends the empirical effort by employing fMRI, EEG, and MEG convergently to understand the distributed brain systems involved in decision making. • Bridges to investigations concerned with decision making processes in real-life situations (e.g. those faced by air-traffic controllers and pilots).

5. Who We Are • PIs: • McClelland, Newsome (Stanford) • Holmes, Cohen (Princeton) • Urban (Carnegie Mellon) • Johnston (NASA Ames) • And many other professional scientists, post-docs, and graduate students in applied mathematics, neuroscience, cognitive science, and engineering

6. A Classical Model of Decision Making:The Drift Diffusion Model of Choice Between Two Alternative Decisions • At each time step a small sample of noisy information is obtained; each sample adds to a cumulative relative evidence variable. • Mean of the noisy samples is +m for one alternative, –m for the other, with standard deviation s. • When a bound is reached, the corresponding choice is made. • Alternatively, in ‘time controlled’ or ‘interrogation’ tasks, respond when signal is given, based on value of the relative evidence variable.

7. The DDM is an optimal model, and it is consistent with neurophysiology • It achieves the fastest possible decision on average for a given level of accuracy • It can be tuned to optimize performance under different kinds of task conditions • Different prior probabilities • Different costs and payoffs • Variation in the time between trials… • The activity of neurons in a brain area associated with decision making seems to reflect the DD process

8. Neural Basis of Decision Making in Monkeys (Shadlen & Newsome; Roitman & Shadlen, 2002) RT task paradigm of R&T. Motion coherence anddirection is varied fromtrial to trial.

9. Neural Basis of Decision Making in Monkeys: Results Data are averaged over many different neurons that areassociated with intended eye movements to the locationof target.

10. Two Problems with the DDM Easy • Accuracy should gradually improve toward ceiling levels as more time is allowed, even for very hard discriminations, but this is not what is observed in human data. • The model predicts correct and incorrect RT’s will have the same distribution, but incorrect RT’s are generally slower than correct RT’s. Prob. Correct Hard Errors Correct Responses RT Hard -> Easy

11. Usher and McClelland (2001)Leaky Competing Accumulator Model • Inspired by known neural mechanisms • Addresses the process of decidingbetween two alternatives basedon external input (r1 + r2 = 1) with leakage, mutual inhibition, and noise: dx1/dt = r1-k(x1)–bf(x2)+x1 dx2/dt = r2-k(x2)–bf(x1)+x2 f(x) = [x]+

12. Wong & Wang (2006) ~Usher & McClelland (2001)

13. Neglect the non-linearity in the LCAM: dx1/dt = r1-k(x1)–b(x2)+x1 dx2/dt = r2-k(x2)–b(x1)+x2 Then subtract and let x = x1-x2, I = r1 - r2 ; x = x1 - x2 To obtain: dx/dt = I -k(x)+b(x)+x This allows precise mathematical analysis, while approximating the outcomes found in the full non-linear version of the LCA… And it reduces to the DDM if k-b = 0 So can see a direct link between neural mechanisms and optimal decision making under uncertainty One-dimensional reduced version of the LCAM (U&M, 2001; Bogacz et al, 2006)

14. Roles of k and b Bifurcation Produces A Decision- Like Outcome x1 – x2 represents the difference in activation of the twoaccumulators for the same value of r1 – r2. Time proceedsfrom stimulus onset. Distribution of values of x1-x2 is shownat three different time points for three combinations of k and b

15. Time-accuracy curves for different |k-b| |k-b| = 0 |k-b| = .2 |k-b| = .4

16. Can we Distinguish Alternative Explanations? • LCAM fits time-accuracy data well, but there are other possible reasons for bounded accuracy • Trial-to-trial variation in the direction of drift • Bounded integration prior to the termination of the stimulus • Distinguishing these alternatives as best we can is one of the goals of our project

17. Specific Aims • Aim 1: Extend the theory of the dynamics of decision making to address integration of stimulus and payoff information in real-time decision making task situations, integrating behavioral, neuroscientific, and theoretical investigations. • Aims 2 and 3: Extend this multi-pronged approach to more complex tasks and task environments • Continuous time and space • Uncertain timing of stimulus onset • Real-life situations including distraction, multi-tasking, and payoff uncertainty

18. Some of the Questions we Will be Addressing in Our Work • To what extent can humans and other participants achieve optimality in decision making, in a range of different decision contexts?  What can we learn from deviations from optimality? • Within the space of dynamic decision making models, can we find evidence that distinguishes among alternatives consistent with existing data? • Can we identify the brain mechanisms that underlie the decision process and determine how these mechanisms achieve optimal performance? • To what extent are the dynamics of decision making fixed characteristics of the decision mechanism, and to what extent are the tunable to task demands?  If so, how is this tuning achieved?

19. 0930 Introduction and scientific background   Jay McClelland 1000 Integrating Payoff and stimulus information            Bill Newsome 1030 Modeling the dynamics of choice behavior         Phil Holmes 1100 Break 1115 Integration of payoff and stimulus information in humans      Jay McClelland 1145 Using fMRI to identify decision structures in humans           Jon Cohen 1215 Discussion 1230 Lunch 1330 EEG and MEG studies of decision dynamics             Patrick Simen1400 Effects of stimulus perturbations and switches on decision dynamics             Juan Gao 1430 Developing continuous measures of decision state         Joel Lachter & James Johnston1430 Break 1515 Multi-single unit recording studies in primates          Bill Newsome 1545 Other future directions and general discussion      Jay McClelland The Schedule