Chris Starmer

1. Chris Starmer The Experimental Economist TSU Short course in Experimental and Behavioural economics, 5-9 November 2012

2. Aims for this Session • To explore • What are experiments? • What are they for? • What can they tell us about the economic world? • Will draw on two ‘demonstration’ experiments • But first…….

3. Folk wisdom • Part of the folk law of C20th economics: • Economics a non-experimental discipline

4. Sceptic #1 "unfortunately, we can seldom test particular predictions in the social sciences by experiments explicitly designed to eliminate what are judged to be the most important disturbing influences." M. Friedman (1953)"The Methodology of Positive Economics".

5. Sceptic #2 - Richard Lipsey “It is rarely, if ever, possible to conduct controlled experiments with the economy. Thus economics must be a non-laboratory science.” (Introduction to Positive Economics, 1979)

6. Enthusiast “as currently practised, economics is ideally suited for experimental investigation” John Hey (1991), Experiments in Economics.

7. Recent Trends • Experimentation practised world-wide e.g. • USA: Caltech, Harvard, Princeton, UCLA...... • Europe: Germany, Netherlands, France ,Italy, UK • Textbooks (1990s) • Journal of Economic Literature • Classification (late 1980s) • Econometrica Guidelines • Palfrey and Porter, Econometrica, 1991. • Experimental Economics (Journal - 1998)

9. What is an Experiment? Let’s Explore!

10. Demo Experiment 1 A (very) Simple Auction Market • Run two auctions • Each auction – I sell a £1 Coin to the highest bidder (£1=2.5GEL) • Different auction rules • I tell students….One auction is for real • You only find out which one, after the two auctions have ended

11. Selling money – Auction 1 • Going to sell a £1 coin • “English auction” • The coin goes to the highest bidder What do you think happens? Do people bid? How much do I sell the coin for?

12. Selling money - Auction #2 • As before - the coin goes to the highest bidder • And they pay their bid • Additional Rule: • the second-highest bidder also has to pay the amount of his/her last bid– but they get nothing in return. What do you think happens? Do people bid? How much do I sell the coin for?

13. Why have I shown you this?

14. Illustrates some typical features of experiments

15. Feature 1 • Experimenter creates decisions for people to make • Decisions are usually REAL • There is MONEY on the table • Participants can gain (and sometimes lose) as consequence of their decisions

16. Feature 2 • Experimenter often interested in what happens when you change some aspect of the decision • In this case, I changed the auction rule (who pays) • The two auctions are ‘TREATMENTS’ with different ‘CONDITIONS’

17. Feature 3 • A good experiment has a well-defined purpose • Treatment comparison may be intended to : • Test a Theory • Explore a Hunch • Experimental design • should normally be directed to a well -formulated research question

18. Feature 4 • Demo experiments show it is not necessarily hard to run an experiment • Some are very simple to implement • Some are of course more complicated • It is harder to run GOOD experiments • Carefully designed to shed light on well-defined and worthwhile research questions

19. Experiment 2: Mind Reading! Instructions • Imagine - You are paired with one other person • Answer the questions on the next slide with the aim of trying to give the same answer as the person you are paired with. • You should do this on your own and don’t try to communicate in any way. • You score a point for every answer that matches the one given by the person you are paired with.

20. Here are the questions: 1.Name any flower: 2. Name any car manufacturer: 3. Write down any positive number: 4. Write down any colour: 5. Write down one of the words: ‘Heads’ or ‘Tails’ Write your answers down PRIVATELY in ENGLISH on a piece of paper

21. Session 2 - Part II

22. So what is an experiment? • An environment created and/or manipulated by an experimenter • For purpose of • observing some variable (X) of interest In econ experiment, usually some kind of (economically relevant) human decisions • while controlling possible influences on X • Note controlling a variable might mean a variety of things including: • Holding it constant, varying it systematically

23. General aim to explore relationships between variables of economic interest Multiple influences economic variables, typically affected by multiple influences in “the wild” y = f(x1, x2………xn) Data generated in the wild might investigate statistically (e.g. econometrics) Alternatively may investigate relationships experimentally i.e., try to ISOLATE effects of particular variables Some Basic Design Principles

24. Suppose we believe: y = f(x1, x2,………, xn) Wish to explore relation between y and x1 So, construct an environment in which... 1) Observe y 2) Vary the level of x1 3) ‘Controlling’ effects of other variables e.g. Holding constant: x2……..xn Hence ISOLATE effect of x1 on y Control

25. Some purposes of experiments • Testing predictions of theory • Looking for ‘surprising’ regularities • Designing and Testing institutions…..

26. The Wind Tunnel Analogy

27. The Biggest Auction Ever K. Binmore and P. Klemperer, 2002. "The Biggest Auction Ever: The Sale of the British 3G Telecom Licences," Economic Journal, 112, 2002. • Advised UK Govt on design of auctions • Sale of 3G spectrum licences • Tricky theoretical problem • Auctions ran in spring 2000 • Raised £22.5 Billion • 2.5% GNP

28. Success and Failure • Sometimes experimental results confirm predictions of economic theory • This itself can be a surprise if the predictions seem surprising or rest on strong assumptions • See day 5 on Markets • Sometimes results challenge economic theory: • see tomorrow on “preference reversal”

29. Programmes of Research • Surprising experimental results typically generate further research • Experiments to test robustness of experimental findings • E.g. did subjects understand the instructions? • New theories or conjectures to explain surprising results

30. Back to mind reading experiment What question(s) was that experiment designed to investigate?

31. Here are the questions: 1.Name any flower: 2. Name any car manufacturer: 3. Write down any positive number: 4. Write down any colour: 5. Write down one of the words: ‘Heads’ or ‘Tails’ Write your answers down PRIVATELY in ENGLISH on a piece of paper

32. Col Heads Tails Heads Prize No Prize Row No Prize Prize Tails Fully rational agents can’t ‘solve’ this game ordinary people seem pretty good at it T. Schelling - 86% of folks choose heads

33. Investigating Coordination • Coordination games are common in world • Experiment, as described, investigates: • Can people coordinate better than chance/rationality predicts? • Looking for a regularity in behaviour • If there, current theory can’t explain it • Perhaps more theory and/or experiment can help us understand it better?

34. Mehta, Starmer and Sugden, American Econ. Review (1994) Design • 20 similar tasks • Subjects randomised between two conditions. • Coordinators (“motivated”) • motivated to try to coordinate and paid according to success • Pickers (“unmotivated”) • ‘rewarded’ so long as all questions answered – no coordination motive

35. Col Heads Tails Heads Prize No Prize Row No Prize Prize Tails • Fully rational agents can’t ‘solve’ this game • Yet ordinary people seem pretty good at it • T. Schelling (1960) Strategy of Conflict, 86% of folks choose heads

36. What explains coordination success of ordinary people • MSS consider three possible hypotheses • Primary Salience • Secondary Salience • Schelling Salience • MSS Experiments designed to discriminate between them

37. Primary Salience • Some labels ‘stick out’ more than others • they come to mind, more easily, for what ever reason • Call these the ‘Salient’ labels • Primary Salience Hypothesis • some people have a tendency to choose alternatives with salient labels • maybe a non-rational propensity

38. Secondary Salience • In a two player game, I act according to secondary salience if: • I try to guess what has primary salience for you • I assume you choose that • I optimise relative to that prediction

39. Schelling Salience • Primary and secondary assume ‘seed’ of non-rational behaviour • Schelling suggests a theory of rational coordination • view coordination as a ‘team’ activity • try to identify a rule which, if followed by everyone, would produce a good outcome for the group • if you can find such a rule, follow it!

40. Predictions • Primary • no difference between groups • Secondary • S’s motivated to coordinate - answers more concentrated on options most popular among unmotivated groups • Schelling • modal choice may vary between groups

41. Unmotivated Rose 35.2 Daffodil 13.6 Daisy 10.2 Tulip 9.1 Coord ind: 0.184 #resp types: 26 Motivated Rose 67.2 Daisy 13.3 Daffodil 6.7 Coord ind: 0.447 #resp types: 11 Results - Task 1 Flower

42. Unmotivated Ford 53.4 Rover 5.7 Coord ind: 0.290 #resp types: 22 Motivated Ford 81.1 Coord ind: 0.659 #resp types: 11 Task 2 - Car Manufacturer

43. Unmotivated 7 11.4 2 10.2 10 5.7 {1 4.5} Coord ind: 0.052 #resp types: 28 Motivated 1 40.0 7 14.4 10 13.3 2 11.1 Coord ind: 0.206 #resp types: 17 Task 3 - Positive Number

44. Conclusions • More than primary salience!

45. Some Observations • Motivated by specific hypotheses • Subjects allocated to groups • groups face decisions • different ‘conditions’ (decision environments) • incentives key to setup • hypotheses predict different behaviour in the two groups • Randomisation of subjects to groups • not just differences between individuals • Mixed success in answering research questions • good on primary vs other • not so good on Secondary vs Schelling • So, as is typical, more to do.........

46. Should I Believe Results of Experiments?

47. Should I Believe Results of Experiments? • One standard query • Most experiments observe simple and abstract decisions of students in unusual decision environments making decisions for small sums of money – do such data tell us much about how the economy works? • A question of external validity

48. Possible responses on external validity 1. Theory testing: Theories of economic phenomena are much simpler than the phenomena themselves If I want to test a theory in an experiment, it can be an advantage to make the experiment simple like the theory

49. Theory, Experiment and World Economic Phenomena “in the wild” LAB EXPERIMENTS Theory Increasing complexity

50. Possible responses on external validity 2. If you are worried that an experiment leaves out something important to the real world phenomena………. Make the experiment more like the target phenomenon