Dynamic Decision Making in Complex Task Environments: Principles and Neural Mechanisms

1. Dynamic Decision Making in Complex Task Environments:Principles and Neural Mechanisms Annual Workshop Introduction August, 2008

2. FY07 MURI BAA06-028 Topic 15 Building Bridges between Neuroscience, Cognition, and Human Decision Making Objective: The general goal is to form a complete and thorough understanding of basic human decision processes … by building a lattice of theoretical models with bridges that span across fields …. The main effort of this work is intended to be in the direction of new integrative theoretical developments … using mathematical and/or computation modeling … accompanied and supported by rigorous empirical models tests and empirical model comparisons …. . From BAA 06-028, Topic 15 Slide from Zhang 2007

3. Our MURI Grant • Builds on past neurophysiological and theoretical investigations of the dynamics of decision making in humans and non-human primates. • Extends the empirical effort by employing fMRI, EEG, and MEG convergently to understand the distributed brain systems involved in decision making. • Extends both the theory and experimental investigations to successively more complex decision making environments as the project continues. • Bridges to investigations concerned with decision making processes in real-life situations (e.g. those faced by air-traffic controllers and pilots).

4. Aims of the Grant • Aim 1: Investigate dynamics of decision making in classical tasks via • Theory and Modeling • Primate Neurophysiology • Human Cognitive Neuroscience • Fundamental tenets of the research: • Decision making occurs through a real-time dynamic process that depends upon neural activity distributed across a wide range of participating brain areas, each shaping the decision making process in its own way. • An effort to understand decision making as an optimization problem is useful because • They allow us to understand how closely behavioral and neural processes can approximate optimality • They allow us to understand how simple neural mechanisms can lead to optimal performance.

5. Aim 2: Extending the theory to decision making in continuous time and space • Detection of targets in noisy backgrounds when time of onset and possible location of targets is uncertain. • Optimality analysis, role of leaky integration, threshold tuning, and adjustment of integration rate in achieving or approximating optimality. • Locating targets in a continuous space. • How is optimization achieved and regulated in response to different demands for speed and precision? • How does the neural representation of a continuous value (e.g. location in space) evolve over time during processing?

6. Aim 3: Extensions to Real-World Situations • Distraction, vigilance, and divided attention. • Extension of neurocognitive models to address such phenomena. • Examination of the neural basis of the Central Bottleneck: • Competition among neural populations representing stimuli/responses associated with different tasks?

7. Goals for this workshop • Review progress on Aim 1 • Primate behavior and neurophysiology • Experiment • Optimality analysis • Relationship between neural activity and behavior • Human experiments and model tests • Further cognitive neuroscience investigations • Brainstorm on wrapping up Aim 1, and moving forward to Aims 2 and 3.

8. Wald (1947) “Sequential Probability Ratio Test (SPRT)” Multiple hypotheses setting Change-point detection setting • Armitage (1950): N(N-1)/2 pair-wise likelihood ratio processes • Baum and Veeravalli (1994): Bayesian analysis on posterior probability of N hypotheses; • Dragalin et al, (1999, 2000): asymptotic optimality of MSPRT • Page (1954): CUSUM procedure • Shiryayev (1963): Bayesian scheme with geometric prior • Roberts (1966): modifying Shiryayev to non-Bayesian version Slide from Zhang 2007 “Sequential Methods” in Statistics

9. The Drift Diffusion Model • Continuous version of the SPRT • At each time step a small random step is taken. • Mean direction of steps is +m for one direction, –m for the other. • When criterion is reached, respond. • Alternatively, in ‘time controlled’ tasks, respond when signal is given.

10. Two Problems with the DDM Easy • Accuracy should gradually improve toward ceiling levels, even for very hard discriminations, but this is not what is observed in human data. • The model predicts correct and incorrect RT’s will have the same distribution, but incorrect RT’s are generally slower than correct RT’s. Prob. Correct Hard Errors Correct Responses RT Hard -> Easy

11. Usher and McClelland (2001)Leaky Competing Accumulator Model • Addresses the process of decidingbetween two alternatives basedon external input (r1 + r2 = 1) with leakage, self-excitation, mutual inhibition, and noise: dy1/dt = r1-l(y1)+af(y1)–bf(y2)+x1 dy2/dt = r2-l(y2)+af(y2)–bf(y1)+x2

12. Wong & Wang (2006) ~Usher & McClelland (2001)

13. Contributions from Princeton • Holmes et al: • Mathematical analysis of dynamical models of decision making. • Investigations of optimality and deviations from optimality. • Relations between models and levels of description • Cohen et al: • Neural basis of executive function and cognitive control. • Functional brain imaging and neurally grounded models in many areas of cognitive neuroscience.

14. Comparative Model Analysis (Bogacz et al, 2006)

15. Physiology of Decision and Value • Neural basis of decision making based on uncertain sensory information, recording from individual neurons in primates. • How do neurons represent (and update our representation of) the value of a choice alternative?

16. Other Participants • Urban lab: • Biophysical processes that allow neurons to oscillate and synchronize their activity • Roles of oscillation and synchrony in information processing in neural circuits • Urban-McClelland collaboration: • Use of MEG to investigate functional synchronization of neural populations across brain areas. • Will extend to decision making in concert with ongoing EEG investigations. • Johnston / Lachter: • Processing limitations affecting throughput, accuracy, and timely responding in human operators. • Attentional limitations and the central bottleneck revealed in dual task situations. • MURI work: investigating decision dynamics using continuous response measures.