Decision Making Theories in Neuroscience

Decision Making Theories in Neuroscience Alexander Vostroknutov October 2008

Choice in the brain From Sugrue, Corrado and Newsome Nature Neuroscience, 2005, Vol 6, May 2005 • Weak motion – chance performance; strong motion – optimal performance • “Decision making” area should aggregate noisy signal and suggest the decision

Monkey brain • LIP area – part of visuo-motor pathway • Its activation is covaried with choice AND modulated by movement strength during motion • not purely sensory (mistake trials); • not purely decision oriented (modulated by strength of movement) • LIP is where “deliberation” takes place From Sugrue, Corrado and Newsome Nature Neuroscience, 2005, Vol 6, May 2005

Three processes of choice From Bogacz, 2007,TRENDS in Cog. Sci., Vol 11(3) • Neurons in Visual cortex provide evidence for alternatives (noisy) • Intergation takes place (in LIP), removes noise • The choice is made once certain criterion is reached (confidence level)

Optimal decision making • This procedure can be formulated as a statistical problem • Statistical test to optimize decision making • It can be tested whether the brain implements optimal test (evolution) • Links optimal tests with neurobiology (basal ganglia) • and behavior (speed-accuracy tradeoff)

Optimality criterion • Sequential Probability Ratio Test (Wald) • A procedure to distinguish two distributions H0: p=p0 and H1: p= p1 given a sequence of observations {yn} • Sum log-likelihood ratios of incoming data and stop once threshold is reached: Sn = Sn-1 + log(p0(yn)/p1(yn)) • Given fixed accuracy, SPRT requires the least expected number of observations • Animals would be interested in implementing SPRT: minimizes reaction time

Input A A - B I > 5: choose A I < -5: choose B Input B Integrator (I) Diffusion model (2 alternatives) • Is there simple way to implement SPRT? • Integrator accumulates evidence about the difference of inputs In = In-1 + An - Bn • Once threshold is reached (|In| > 5), choose A or B

Diffusion Model is optimal • Continuous limit of SPRT can be described by Wiener process with drift (Bogacz et al, 2006) dy = (mA-mB)dt + cdW • Choose once threshold is reached(assumed: A and B are normal, same variance) • mA is mean of alternative A • This is exactly Diffusion Model! • Thus DM implements SPRT • Given fixed accuracy, DM has the best reaction time(important for animals) • Simple to implement in neural networks(requires only addition and subtraction)

Connection to the brain • How can we test whether something like diffusion model is implemented in the brain? • We have evidence (LIP) of the presence of intergators • We need evidence for the presence of “criterion satisfying” region • Good candidate: basal ganglia • They resolve competition between cortical and sub-cortical systems that want expression • Inhibit all actions; the “winning system” is allowed to express itself through disinhibition

Diffusion Model (n alternatives) Input A1 A1 – ln[exp(A2)+exp(A3)] • DMn implements optimal MULTI SPRT • Uses exponentiation • Neurons which exponentiate are rare • Good evidence for Diffusion Model I1 choose whenever any of these is higher than threshold Input A2 A2 – ln[exp(A1)+exp(A3)] I2 Input A3 A3 – ln[exp(A1)+exp(A2)] I3

Evidence • Bogacz, 2007 reports studies that demonstrate that neurons in subthalamic nucleus (STN) perform exponentiation • STN targets output nuclei of basal ganglia, that “decide” on which system to allow to act

More evidence

Diffusion Model and Economics • Difficult to perceive the difference between n and n+1 grains of sugar • Non-transitivity of indifference • Beyond the scope of classical preferences model • DM suggests a simple and natural way to model this

price A B C quality Diffusion Model and Economics A B A, B available: • Violation of Weak Axiom of Revealed Preference(recent evidence: Kroll, Vogt, 08) • Again, DM with 3 alternatives gives simple explanation • Prospect Theory, Regret do not account for this • Can save the “existence” of underlying preferences • Additional prediction of DM: smaller reaction time in second case 80% 20% A B A, B, C available: 50% 50%

Diffusion Model and Economics S1 = $1 R1 = ($5, 0.1; $1, 0.89; $0, 0.01) • Allais paradox: violation of Expected Utility maximization • In choice between S1 and R1: information about S1 is accumulated much faster than about R1: high chance of hitting S1 threshold • In choice between S2 and R2: information accumulates at comparable speeds, R2 is almost like S2, only with $5 instead of $1, high chance to hit R2 threshold first • Additional prediction of DM: reaction time in S1-R1 choice is shorter than in S2-R2 • No need to get rid of Expected Utility EU maximizer prefers S’s or R’s Evidence: S1 > R1 and R2 > S2 S2 = ($1, 0.11; $0, 0.89) R2 = ($5, 0.1; $0, 0. 9)

Conclusion • It seems like there is evidence that Diffusion Model is implemented in the brain • Sensory inputs are integrated in the respective pre-motor regions (LIP) • Basal ganglia check which option should be chosen by comparing competing “integrators” to the threshold • Important for economists. DM explains with ease many different phenomena

Decision Making Theories in Neuroscience