Experimental Psychology Society

1. Lancaster Conference, April 2013 Symposium on Mental Time Travel organised by Tony Dickinson Experimental Psychology Society Intertemporal choice, hyperbolic discounting, and mental time travel: A comparative and evolutionary discussion Stephen E. G. Lea, University of Exeter

2. The Economic Problem:The stylised facts about inter-temporal choice Humans are terrible at making choices between outcomes that appear at different times in the future Everyday examples: • Extra sleep now vs. getting to work punctually in an hour’s time • Having a washing machine now vs. being debt-free next year • Luxury expenditure now vs. enough income in retirement • A cigarette now vs. living 20 years longer This is by far the biggest deviation from rational choice in the whole of economic behaviour S. E. G. Lea

3. Experimental data support this, under many different names: • Failure to delay gratification • Lack of self control • Weakness of will • Impulsiveness • Myopia • Short time-horizon • Inability to consider future consequences • [excessive] Delay discounting • Failure of self-regulation • ...etc S. E. G. Lea

4. In what ways are we bad at intertemporal choice? • Impatience (Fisher, 1930): We choose the outcome that will arrive sooner even when it is much less valuable than the more delayed one, beyond any point that could be justified by a rational analysis • Inconsistency (Ainslie, 1974): We change our preferences between immediate and delayed outcomes as they (both) come closer in time – though this cannot happen if delay is discounted exponentially, as rationally it should be S. E. G. Lea

5. In terms of discount rates: • We have to set a “subjective discount rate” at an absurdly high level (Friedman: 33%) to account for the (econometric) facts of human saving (and even higher for experimental data) • No single discount rate will account for all the inter-temporal choices that an individual makes even in one situation, let alone across situations S. E. G. Lea

6. The psychological solution • Suppose that the value of a future outcome declines according a hyperbolic rather than an exponential function of the time until it will be realised • This automatically generates a discount rate that is higher for outcomes further in the future • Hyperbolic discounting is now widely accepted by (behavioural) economists. It is arguably the most important concept to have been imported into economics from psychology S. E. G. Lea

7. What is the psychological evidence for hyperbolic discounting? • Herrnstein’s matching law, formulated to describe the behaviour of pigeons working on concurrent variable interval schedules of reinforcement asserts that: R1/R2 = Rf1/Rf2 • In this free-operant situation, Ri is a rate of responding (pecks/unit time) and Rfi is a rate of reinforcement (food deliveries/unit time) • In discrete trial choice, there is exactly one reinforcement, after a delay Di ; and the natural analogue of reinforcement rate is then simply 1/Di– leading to the prediction of hyperbolic discounting S. E. G. Lea

8. Commitment • Ainslie and Rachlin used this analysis to design experiments on pigeons, in which the birds showed both impatience and preference reversal, and choices were qualitatively consistent with the discrete-trial version of the matching law • Mazur shows that the following form is more precise: Vi = Ai/(1+KDi) Where Vi is “subjective value”, Ai is reward size, Di is delay, and K is a fitting constant • Vi values can be used to predict choice across a wide range of animal choice experiments S. E. G. Lea

9. Evolutionary argument No. 1 • Matching is a consequence of basic processes of operant conditioning / associative learning, common to all vertebrates, and at least some invertebrates, e.g. bees, molluscs • It predicts “near enough” optimal behaviour in standard foraging situations, e.g. the optimal diet problem, faced by organisms of virtually all taxa • We therefore predict that the laws of associative learning, in general, and matching in particular, will be evolutionarily conservative • Humans should be no exception to any of this S. E. G. Lea

10. If you aren’t getting worried, you haven’t been paying attention… • We are explaining life-time, one-off, deliberative, human intertemporal choices by means of phenomena demonstrated in short-term, highly repetitive, contingency-governed, choices made by rats and pigeons. • A key step in predicting the phenomena is the use of the matching law, which in general does not hold for human deliberative choices (Horne and Lowe) • The discount rates required to account for commitment choices in the laboratory would be grotesque if extrapolated to the life-time scale – even by the standards of human intertemporal irrationality S. E. G. Lea

11. …and how reliable is hyperbolic discounting anyway? • Not all direct experimental tests of discount functions show a hyperbolic form in all participants • Not all market data are consistent with hyperbolic rather than exponential discounting • Not all demonstrations of preference reversal show reversal in the direction predicted by hyperbolic discounting (Read et al 2012) • Intertemporal choices can be radically affected by framing of the question (Read et al 2013)… • …or by the comparison set made available (Vlaev et al 2007) S. E. G. Lea

12. Paradox As psychologists, what we have to explain is not why humans are so bad at intertemporal choice, but why we are so (relatively) good at it… and not why hyperbolic discounting occurs, but why it sometimes doesn’t S. E. G. Lea

13. How, other than from the matching law, might we get hyperbolic discounting (or something like it)? Two basic kinds of possibility • Humans have some mechanism, either specific instincts or a specific cognitive capacity, arising from our evolutionary past, that deals with the far future (i.e. more than a few seconds/minutes ahead) • Decisions about the far future are outside our evolutionary experience, so we deal with them as best we can using general-purpose cognitive mechanisms S. E. G. Lea

14. Adaptations for long-term choices • Examples: • birds gaining weight before migration • hibernators gaining weight in the autumn • hoarders (both scatter hoarders e.g. some corvids, some squirrels, and larder hoarders e.g. hamsters) spending much time and energy creating caches in the food-rich season • These behaviours are subserved by specific physiological or instinctual mechanisms and/or specific cognitive capacities (e.g. seasonal change in size of stomach or hippocampus) Red knot. Photograph by Jan van de Kam from PLoS, used with permission S. E. G. Lea

15. A common selective pressure: Seasonal variation • Such mechanisms have evolved because they enable animals to exploit particular niches, and in all these examples, it is seasonal variation in food supply • It’s conceivable that • During human evolution we had to cope with such seasonal variations • The mechanism for doing so is capable of being extended beyond its natural one-year limitation S. E. G. Lea

16. Life history strategies • However many animals also make “choices” between patterns of behaviour that affect the entire course of their lives, not just survival in the coming season • In evolutionary theory these patterns are referred to as life-history strategies • A very common field for such choices is the timing of reproduction (the r/K decision axis) • Although these differences in strategy are clearly facultative, to what extent can we regard them as being chosen? It’s plausible that they are subserved by evolved instincts S. E. G. Lea

17. Human reproductive choice • Humans are long-lived, reproduce slowly, and occupy many different ecological niches: these all predict multiple life-history strategies • In existing hunter-gatherer societies, birth rate is well below the maximum achievable, implying some kind of trade-off to achieve maximum fitness (e.g. Kaplan, 1996) • In all modern societies, birth rate has crashed (the demographic transition), implying that the trade-off is labile S. E. G. Lea

18. Reproductive choice and economic choices • Modern humans vary in speed of reproduction strategies • These variations are correlated with social class, economic status and other indicators of environmental harshness and unpredictability (e.g. Quinlan, 2008; Ellis et al., 2009; Nettle, 2010) • Increased use of credit and reduced saving are associated with the fast-reproduction (r) strategy • However direct experimental tests or a correlation between life history strategies and high discount rates do not reliably support predictions (e.g. Wilson & Daly, 2006) S. E. G. Lea

19. What about general cognitive mechanisms? • In delayed gratification tasks, whether the contingencies are hypothetical or real, people need to imagine the future • This is closely linked to recollecting the past – episodic memory • Suddendorf & Corballis (1997, 2007) call these two capacities “Mental time travel” and claim that they constitute a uniquely human cognitive capacity (autonoetic consciousness: Tulving, 1985) • The forward looking version of it is referred to as anticipation or episodic future thought S. E. G. Lea

20. Temporal discounting and mental time travel • Stewart, Chater & Brown (2003) propose Decision by Sampling(DbS) theory • According to DbS, we compare a proposed delay with a sample of delays drawn from memory • Because long delays are experienced less often than short delays, the distribution in the sample follows a power law (cf. Zipf’s law, etc) • This yields (sub)hyperbolic discounting even with random sampling from memory, without any underlying psychoeconomic scale S. E. G. Lea

21. Evolutionary argument No. 2 • For whatever reason (there are many hypotheses), humans, and perhaps a few other species, have evolved the capacity for reflective thought. • This gives us a consciousness of the self – and thereby makes mental time travel over long periods possible • This allows us to compare an imagined future with present reality; this might be the selective pressure that caused the capacity to evolve • But whatever pressures produced this capacity, they did not involve modern, finance-based, choices, so there is no reason to expect these to be made optimally or accurately S. E. G. Lea

22. An inevitable complication: the need for self-control • Associative learning still works in humans, even if we have other cognitive processes available in addition • So we have two mechanisms available for making many kinds of decisions – and they will often lead in opposite directions • When we use reflective thought to override the response that would be made on the basis of the laws of conditioning, we call that self-control or self-regulation (though it would be better to call it control by the self) • This “mental tool” is used in many situations – including inter-temporal choice • There is much evidence that it is cognitively and even physiologically expensive to use S. E. G. Lea

23. But (lack of) self-control is not the core problem • Myopia and preference reversals occur even when self-control has not broken down. When operant conditioning completely takes over, the degree of myopia is even worse • There are plausible evolutionary accounts of mental time travel, and its use to make value comparisons over months or years • Hyperbolic discounting can be predicted by a general cognitive mechanisms, Decision by Sampling theory • No mechanism has been proposed that would lead to the “economically rational” solution, exponential discounting S. E. G. Lea

24. So how do humans discount? To assess the value of far distant rewards, we have to rely on: • Self control, to avoid the hyper-myopia predicted by the matching law • Episodic future thought, to imagine the future situation • And to assess its value • Either Decision by Sampling (or something like it) • Or rational calculation, if we have the theoretical knowledge and computational ability , not to mention the required faith in economists S. E. G. Lea

25. This means we are likely to observe: • Substantial individual differences • Substantial intra-individual inconsistencies • Varying forms of the discount function • Little correspondence between actual and rational discounting, either in terms of discount functions or discount rates QED S. E. G. Lea

26. Thanks to… • DetlefFetchenhauer, Erik Hölzl and their students at Köln, and Stephanie Preston and all at the Michingan Meeting on the Interdisciplinary Science of Consumption, for discussion of earlier versions of this paper • Paul Webley, Lesley Newson, AvrilMewse, Wendy Wrapson, Gordon Brown, Peter Fischer and David Crelley for discussion of essential ideas S. E. G. Lea

27. References • Ainslie, G. (1974). Impulse control in pigeons. Journal of the Experimental Analysis of Behavior, 21, 485-489. • Ashby, F. G., Alfonso-Reese, L. A., Turken, A. U., & Waldron, E. M. (1998). A neuropsychological theory of multiple systems in category learning. Psychological Review, 105, 442-481. • Barron, G., & Erev, I (2003). Small feedback-based decisions and their limited correspondence to description-based decisions. Journal of Behavioral Decision Making, 16, 215-233. • Baumeister, R. F., Bratslavsky, E., Muraven, M., & Tice, D. M. (1998). Ego depletion: Is the active self a limited resource? Journal of Personality and Social Psychology, 74, 1252-1265. • Clayton, N. S., Bussey, T. J., & Dickinson, A. (2003). Can animals recall the past and plan for the future? Nature Reviews Neuroscience, 4, 685-691. • Conger, R., & Killeen, P. (1974). Use of concurrent operants in small group research - demonstration. Pacific Sociological Review, 17, 399-416. • Fisher, I. (1930). The theory of interest. New York: Macmillan. • Frank, M. J., Cohen, M. X., & Sanfey, A. G. (2009). Multiple systems in decision making: A neurocomputational perspective. Current Directions in Psychological Science, 18, 73-77. • Friedman, M. (1953). The methodology of positive economics. In M. Friedman (Eds.), Essays in Positive Economics, pp. 3-43. Chicago: University of Chicago Press. • Gigerenzer, G., & Todd, P. M. (1999). Fast and frugal heuristics: The adaptive toolbox. In G. Gigerenzer, P. M. Todd, and the ABC Research Group (Eds.), Simple heuristics that make us smart. New York: Oxford University Press. • Herrnstein, R. J. (1961). Relative and absolute strengths of response as a function of frequency of reinforcement. Journal of the Experimental Analysis of Behavior, 4, 267-272. • Horne, P. J., & Lowe, C. F. (1993). Determinants of human performance on concurrent schedules. Journal of the Experimental Analysis of Behavior, 59, 29-60. • Kaplan, H (1996). A theory of fertility and parental investment in traditional and modern human societies. Yearbook of Physical Anthropology, 39, 91-135. • Katona, G. (1975). Psychological economics. New York: Elsevier. • Lea, S. E. G (1979). Foraging and reinforcement schedules in the pigeon: Optimal and non-optimal aspects of choice. Animal Behaviour,27, 875-886. • Lea, S. E. G. (2001). Anticipation and memory as criteria for special welfare consideration. Animal Welfare, 10, S195-S208. • Nettle, D. (2010). Dying young and living fast: variation in life history across English neighborhoods. Behavioral Ecology, 21, 387-395. • Mazur, J. E. (1984). Tests of an equivalence rule for fixed and variable reinforcer delays. Journal of Experimental Psychology: Animal Behavior Processes, 10, 426-436. • Piersma, T., Dietz, M. W., Dekinga, A., Nebel, S., van Gils, J. A., Battley, P. F. & Spaans, B. (1999). Reversible size-changes in stomachs of shorebirds: when, to what extent, and why? ActaOrnithologia, 34, 175-181. • Quinlan, R. J. (2008). Human Pair-Bonds: Evolutionary Functions, Ecological Variation, and Adaptive Development. Evolutionary Anthropology, 17, 227-238. • Rachlin, H., & Green, L. (1972). Commitment, choice, and self-control. Journal of the Experimental Analysis of Behavior, 17, 15-22. • Roberts, W. A. (2002). Are animals stuck in time? Psychological Bulletin, 128, 473-489. • Schneider, W., & Shiffrin, R. M. (1977). Controlled and automatic human information-processing: 1: Detection, search, and attention. Psychological Review, 84, 1-66. • Suddendorf, T., & Corballis, M. C. (1997). Mental time travel and the evolution of the human mind. Genetic Social and General Psychology Monographs, 123, 133-167. • Suddendorf, T., & Corballis, M. C. (2007). The evolution of foresight: What is mental time travel, and is it unique to humans? Behavioral and Brain Sciences, 30, 299-351. • Thaler, R. H., & Shefrin, H. M. (1981). An economic theory of self-control. Journal of Political Economy, 89, 392-406. • Wilson, M., & Daly, M (2006). Are juvenile offenders extreme future discounters? Psychological Science, 17, 989-994. S. E. G. Lea