Value computation and value modulation: a dual-process theory of self-control

1. Theoretical REsearch in Neuroeconomic Decision-making(www.neuroeconomictheory.org) Value computation and value modulation: a dual-process theory of self-control Isabelle BrocasUSC and CEPR May 2013 Juan D. CarrilloUSC and CEPR

2. What is “Neuroeconomic Theory”? Neuroeconomic theory (NT) uses evidence from neuroscience, neurobiology and neuroeconomics to revisit theories of decision-making Neuroscience evidence includes: Functionality of different brain systems Interactions between systems Physiological constraints in choice processes NT uses evidence to build constrained optimization models using standard microeconomic theory toolkit. The brain is, so it should be modeled as, a multi-system organization

3. What is “Neuroeconomic Theory”? Advantages • Discipline “Too much” freedom to model bounded rationality (in memory, foresight, information processing, etc.). NT helps decide which constraints are crucial and how to incorporate them. • Micro-foundations Discounting, risk-aversion, ambiguity aversion, etc. are traditionally primitives of the models. Some can be derived from the interplay of brain systems. • Testability Implications at multiple levels: behavioral (choice) andneurophysiological (brain activation).

4. This paper • How do we decide whether to consume a yummy good?

5. This paper • Consuming a food item is not only about taste: • Health: is that candy good for my teeth? • Scarcity: is it my last candy or the first candy in a big pack? • Environment: candy at home vs. candy at the daycare • Attention: candy when I am bored vs. candy when I am playing

6. This paper • Our research questions: • Why different environments trigger different behaviors? • (salad at lunch and burger at dinner) • Why do we observe both self-restraint and self-indulgent behavior? • (water at home and spirits at a party) • Our perspective: • Neural correlates of choice inform us about the underlying mechanisms that produce behavior • Observations can be used to build models mapping underlying mechanisms (primitives) and behavior (result) • Models explain (not assume) heterogeneity in behavior

7. This paper • In this paper: • Rely on Hare, Camerer & Rangel (Science, 2009) [HCR]: description of a neurophysiological mechanism in the tasty unhealthy food domain • Build an economic brain-based model based on HCR

8. This paper HCR: consumption of good with 2 attributes: taste θ and healthiness h θ h • Dual-process approach: • vmPFCcomputes goal value: considers all the attributes from which it receives information. • dlPFCmodulates value: transmits information about high-order attribute (exerts self-control) or not (no self-control) • vmPFC activity reflects θ and h when self-control is exerted • vmPFC reflects only θ when no self-control is exerted • dlPFC activity correlates with self-control behavior vmPFC dlPFC Self-control

9. The model The model • Experiment consists in offering items with taste θdrawn from uniform [0,1]and health rating h drawn from log-concave p.d.f. f(h)whichrepresents the “environment” (party vs. home, vegan vs. steakhouse). • Realization of taste θ known by vmPFC. • Realization of (high-order) health h known by dlPFC. • dlPFC can communicate health but modulation requires costly attention c(cost of modulation) • Goal value is θ – αh, where α (0,1) is the relative importance of health. • vmPFC integrates all information available to compute decision value: • - If modulation, decision value is θ – αh • - If no modulation, does vmPFC make inferences? • - NaivevmPFC: represents only θ • - SophisticatedvmPFC: represents θ – αE[h|nomodulation]

10. The model • We assume dlPFC is sophisticated and we model the problem as a sequential game • Stage 1: dlPFC receives information about h and decides whether to send it to vmPFC (at a cost c), anticipating vmPFC’s degree of sophistication • Stage 2: vmPFC computes goal value based on: • - Hard information available (θ and maybe h) • - Inferences made (depend on degree of sophistication) • Stage 3: vmPFCtriggers consumption if goal value is positive (outside option normalized to 0) • Obviously, this is an “as if” model

11. Benchmark: naive vmPFC • As usual, we proceed by backward induction • Consumption choice (by vmPFC): • If modulation: • If no modulation: • Modulation choice (by dlPFC): • Value of Modulation: • Value of no Modulation: • Choice:

12. Benchmark: naive vmPFC Let Hdenote the set of values of health such that dlPFC does not exert modulation iffh  H Theorem 1. Modulation at the top. H = [0, ] Bias towards over-consumption Modulation equivalent to costly self-control Corollary 1.Choices can be affected with distractors and multiple tasks: d/dc > 0, d2/dc2 < 0. Related to social psychology (Baumeister et al.) which argues that multi-tasking impairs self-control.

13. Sophisticated vmPFC • Naïve interpretation is somewhat ad-hoc and one of many non-rational ways not to account for information (why assume h=0?). • Suppose now “rational expectations”. • Consumption choice (by vmPFC): • If modulation: • If no modulation: • Modulation choice (by dlPFC): • Value of modulation: • Value of no modulation: • Choice:

14. Sophisticated vmPFC For any given c, αand f(.) the equilibrium is unique. Theorem 2. Modulation at the top and/or bottom.

15. Sophisticated vmPFC Modulation at the top: if no modulation, vmPFC infers health rating is good and consumption takes place (unless θis low). Modulation used to instruct to be cautious when h is bad (costly self-control). This strategy is efficient when h is likely to be good (cost spent rarely), e.g. at a friend’s party. Modulation at the bottom: if no modulation, vmPFC infers health rating is bad and consumption does not take place (unless θis high). Modulation used to authorize consumption when his good. This strategy is efficient when h is likely to be bad (cost spent rarely), e.g. in a bar.

16. Sophisticated vmPFC θ θ Consumption Consumption α E[h|h>h*] No Consumption α E[h|h<h**] No Consumption h h h** h* No M M M No M Environment 1: Mildly unhealthy goods Environment 2: Strongly unhealthy goods

17. Cues and self-control • In new behavioral theories, “cues” change preferences (Laibson, QJE 2001) or trigger mistakes (Bernheim-Rangel, AER 2004). • Here, environment determines f(h), the distribution from which his drawn, which affects modulation (and ability to not consume): a spirit at a bar may trigger different response than the same drink at a friend’s home. • Modulation is costly  used when informativeness content is greatest. f2(h)/f1(h)increasing in h: modulation for low h in f2 and high h in f1. Corollary 2. The likelihood of modulation and consumption depends on h but also on the distribution f(h) from which it is drawn. Modulation occurs for the unexpected realizations of health.

18. Self-regulation and self-control θ θ Consumption Consumption α E[h|h>h*] No Consumption α E[h|h<h**] No Consumption h h h** h* No M M M No M Environment 1: Mildly unhealthy goods Environment 2: Strongly unhealthy goods Tasteless healthy item under-consumed Tasty unhealthy item over-consumed

19. Self-regulation and self-control Suppose cvaries from decision to decision. For simplicity, η ~ U[0,1]. Systems learns ηbefore value modulation and value computation. Corollary 3. Subjects over-consume highly unhealthy goods (self-indulgence) and under-consume mildly unhealthy goods (self-restraint). θ Consumption c = 0 c > 0 No Consumption h

20. Impatience and self-control Suppose health effect h is known (and = 1) but not the date t when it occurs. Goal value: θ – δtwhereδis the (exponential) discount rate. A straightforward reinterpretation of the model implies the following: When health effect is expected to be in the short term, modulation occurs for late dates. In the absence of modulation, vmPFC expects health effects to occur soon and consumption does not take place unless the food is very tasty. Very tasty food with extremely soon effects are over-consumed. When health effect is expected to be in the long term, modulation occurs for early dates. In the absence of modulation, vmPFC expects health effects to occur late and consumption takes place unless the food is not tasty enough. “Tasteless”food with very long-term effects are under-consumed.

21. Impatience and self-control Goal value: θ – δtwhereδis the (exponential) discount rate. Corollary 4.Costly modulation endogenously generates decreasing impatience. θ c = 0 Consumption c > 0 No Consumption 1 T t

22. Impatience and self-control The same self-indulgence and self-restraint principle implies: • Over-consumption if negative effects are in the near future • Under-consumption if negative effects are in the distant future • Exponential discounting with c > 0 is observationally equivalent to hyperbolic discounting with c = 0 • Beh. Econ: hyperbolic discounting induces self-control problems.Our theory: costly modulation generates behavior consistent with decreasing impatience (or hyperbolic discounting)

23. Addiction and self-control Standard extension to habit formation. Denote uxy the utility of consuming y  {0,1} in period 2 after having consumed x  {0,1} in period 1. u01 = θ – αh u11= pθ– αh u00= 0 u10= -qθwith p < 1 and p+q > 1 We can do the exact same analysis as before Corollary 5. Signal modulation is less prevalent for addicted subjects than non-addicted ones. Higher marginal value of consumption lower incentives to pay cost of signal modulation higher incentive to remain ignorant of health effects This is not irrational self-delusion but optimal cost-benefit trade-off.

24. Conclusions Premises of the general approach • The brain is a multi-system organization. • Bounded rationality model based not on inspiration but on physiological constraints of the brain  derive behaviors from brain limitations Results of this paper Unified framework to explain several anomalies related to self-control: • Choices can be manipulated with distractors • Environmental cues endogenously affect the choices • Costly modulation generates both self-indulgence and self-restraint • Costly modulation generates choices consistent with decreasing impatience • Addicted subjects exercise less self-control • Predictions are testable (neural & behavioral predictions)