Dr. Bruce J. West Chief Scientist Mathematical & Information Science Directorate

1. UNCLASSIFIED / FOR OFFICIAL USE ONLY Decision Making and Stochastic Delay at Workshop on Social Computing, Behavior Modeling and Prediction 1 April 2008 Dr. Bruce J. West Chief Scientist Mathematical & Information Science Directorate Army Research Office Bruce.j.west@us.army.mil 919-549-4257 UNCLASSIFIED / FOR OFFICIAL USE ONLY

2. Decision Making and Delay Outline of talk • Discounted Utility Model & intertemporal choice • Anomalies from discounted utility theory • irrationality • hyperbolic discounting • Objective and subjective time • entropy and the direction of time • time as a stochastic variable • Individuality and paternalism • some experiments • fit of theory to data • Conclusions

3. Discounted Utility Model (DUM) Decision Making and Delay • Discount factor δ compresses many mechanisms • mortality, uncertainty, time compression,… • Accepted as both normative (how things should be) and • descriptive (how things are)…..but was initially arbitrary • Samuelson (1937). • Exponential form implies time consistency (rationality)

4. Anomalies from DUM Decision Making and Delay • Time inconsistency • empirical discount factor is not constant • over time • across type of intertemporal choices • Delay effect (hyperbolic discounting) • Interval effect (non-stationarity) • Sign effect (gains vs. loses) • Magnitude effect (small vs. large) • Direction effect • Sequence effects (ordered set vs. single)

5. Model comparison Decision Making and Delay • Exponential delay model • monotonic decrease in value with objective time • constant rate results in time consistency • rationality • Hyperbolic delay model • decreasing rate results in time inconsistency • irrationality (preference reversal) hyperbolic exponential exponential hyperbolic hyperbolic exponential

6. Objective vs. subjective time Decision Making and Delay • Hyperbolic models • Objective time • clockwork universe • entropy and the direction of time • Subjective time • unidirectional • probability and statistics • Motivate decision-making using ensemble distributions • subjective time • stochastic delays

7. Delay and uncertainty Decision Making and Delay • Decision-making models of intertemporal choice can be extended to incorporate probabilistic choice where p is the probability of reward at time t and F is an unspecified function. discretecontinuous • No reward before delay time t • Delay-time probability density

8. Decision Making and Delay Stochastic rate • Deterministic discount rate is replaced with a conditional probability per unit time • The ratio of the delay time distribution function to the survival probability density, integrates to • The utility function in terms of subjective time is therefore

9. Example rate Decision Making and Delay • Rate of reward production suggested by hyperbolic distribution • Probability of no reward before time t is so that the utility function is inverse power law • T measures response time and α measures irrationality

10. Experimental data Decision Making and Delay • Students (20) asked to make decisions about hypothetical money to be received immediately or at a later time, concerning the subjects themselves or another person not known to them. Takahashi, Physica A (2007). self other T=1.85 α=0.11 T=31 α=0.28

11. Implications from experiments Decision Making and Delay • The response times could describe paternalistic policy making government officials, where irrationality is enhanced. • Irrationality is nowhere more significant than in the military where choices may determine whether others live or die.

12. Decision Making and Delay • Nonlinear dynamic equation solved on the interval (0,1). • define a delay-time distribution density • assume a uniform distribution of initial conditions to obtain • delay-time distribution density is non-Poisson, renewal and non-ergodic

13. Measured discount rates Decision Making and Delay • Higher discount rates compared with controls* ( smaller T and α in stochastic intertemporal model) • smoking • excessive alcohol consumption • illicit drug use (cocaine, crack-cocaine and heroin) • pathological gambling • age • cognitive ability (negative correlation with intellectual achievement) • Consistent with neuroeconomic hypothesis that prefrontal cortex is essential for patient (forward looking) decision making. * Chabris, Laibson & Schuldt, The New Palgrave Dictionary of Economics (2007).

14. Brain Activity Decision Making and Delay β network”: midbrain dopamine network; reward processing (ventral striatum V.Str. and medial prefrontal cortex mPFC.) δ network: cognition; dorsolateral prefrontal cortex (diPFC) and right posterior parletal cortex (R.Par.) Sanfey, Loewenstein, McClure and Cohen, TRENDS in Cognitive Science (2006).

15. Decision Making and Delay More brain activity • Two discounting slopes • < one year • > one year • Different parts of the brain light • up under fMRI • short-term • long-term Wittmann & Paulus (2007)

16. Decision Making and Delay Conclusions and Speculations • decision-making is not always rational • irrationality in intertemporal choice models take a hyperbolic form • inverse power laws or hyperbolic utility functions can be generated by stochastic delay times • different parts of the brain control decisions associated with long and short delay times • the complexity of the brain produces the subjective nature of biological time