1 / 25

An Experimental Test of Information & Decision Markets

This study examines the effectiveness of information and decision markets in aggregating accurate predictive information. It explores different market designs and mechanisms to measure their performance in thin trading situations. The results suggest that conditional contracts and combinatoric call markets or combinatoric sequentially shared scoring rules significantly improve market performance compared to standard markets.

bburgoyne
Download Presentation

An Experimental Test of Information & Decision Markets

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 25 minute presentation! An Experimental Test of Information & Decision Markets Robin Hanson, Takashi Ishikida and John Ledyard Caltech 2/4/2005

  2. Information Markets • Standard Information Markets seem to work. • Small but complete set of securities • Many informationally small and unbiased traders. • Theory and evidence from experiments and applications are all positive. • But all assume, require, use, ….. • Straight-forward behavior • Price taking, honest revelation, etc. • Complete set of state dependent contracts • Common knowledge of all priors, … • Even then we see “failures to fully aggregate information” • Incomplete Bayesian updating • Incompletely revealing Rational Expectations Equilibrium DIMACS

  3. Decision vs Prediction • A policy maker does not just want to know the probability that “terrorist attacks in the US will increase in 2005.” • They want to know the probability that “terrorist attacks in the US will increase in 2005”if “US troops remain in Iraq for 2005.” • With N events (attacks, troop size, …) and S outcomes for each (increase from 10-20%, decrease, …), a complete set of state dependent contracts requires N^S - 1 contracts. S = 2, N = 8 => 255 contracts DIMACS

  4. Remember PAM? Goal: Collect accurate predictive information on political and economic stability in the middle east. DIMACS

  5. Remember PAM? • Every nation*quarter: • Political stability • Military activity • Economic growth • US $ aid • US military activity • & all combinations • & ……… • 8 nations, 5 indices, • 4 quarters DIMACS

  6. Remember PAM? • Every nation*quarter: • Political stability • Military activity • Economic growth • US $ aid • US military activity • & all combinations • & ……… • 8 nations, 5 indices, • 4 quarters (N = 180) • Even if we only use up-down questions, • completeness requires 2^180 =1.5*(10^54) contracts. DIMACS

  7. Using Conditional Contracts • The good news • There may more overall trading. • Traders may know more about and be more willing to trade on the relatively more precise event {terrorism up | troops up} as opposed to the less precise {terrorism up}. • The bad news • There may be thinner trading per security. • Too many markets to pay attention to. • Thinner trading => bad price discovery and incomplete arbitrage => prices do not aggregate information. DIMACS

  8. Decision Markets • Markets for Decision Analysis will be thin. • Large and possibly incomplete set of securities • Few informationally large and biased traders • Theory is unlikely to be a good predictor of behavior. • Current applications and experiments may not be applicable to the thinner situation. • How can we know what will actually work? DIMACS

  9. Experimental Test Beds • Create an environment that”captures” as much of the problem as possible (the econ wind tunnel) • Three traders, three events with 2 outcome each (8 states) • Common prior with asymmetric information • 10 draws from one urn of 6 equally likely => (1,0,1), (1,1,0),….. • Each trader sees only two entries of each draw: (1,0,x), (1,1,x),… • Run different mechanisms and market designs • Measure performance • How close are final prices to the fully informed posterior? DIMACS

  10. Theory Benchmarks - 3 events priors uniform posteriors DIMACS

  11. Individual Scoring Rule priors uniform posteriors DIMACS

  12. Standard Markets priors uniform posteriors DIMACS

  13. Design Matters • Asking is not enough. • “Let there be markets” is not enough. • Conjecture: An IM will work better in thin situations, if we use (to “thicken” trading) • Conditional contracts and • a Combinatoric (package bid) Call Market • Includes “no arbitrage” pricing but is intermittent • Does not directly address “monopolistic agents” DIMACS

  14. Combinatoric Call Market uniform posteriors priors DIMACS

  15. Design Matters • We are not yet at complete aggregation. • Conjecture: An IM will work even better in thin situations, if we use (to “thicken” trading) • Conditional contracts and • A Combinatoric Sequentially Shared (Market) Scoring Rule • Is continuous and directly addresses report manipulation • But it involves a subsidy to traders. DIMACS

  16. Shared Scoring Rule - w/CC posteriors uniform priors DIMACS

  17. Tentative Conclusions • Standard markets and surveys do not work will in thin situations. • Using conditional contracts and assuming some self - selection, either combinatoric call markets or combinatoric sequentially shared scoring rules significantly improve performance over standard markets. DIMACS

  18. Open Questions • There are many others we did not test • Pari-mutuel mechanisms • Economides, Lange, and Longitude (some combinatorics) • Pennock - Dynamic Pari-mutel Market • Plott - Auction then Pari-mutuel DIMACS

  19. Open Questions • There are many others we did not test • Pari-mutuel mechanisms • Economides, Lange, and Longitude (some combinatorics) • Pennock - Dynamic Pari-mutel Market • Plott - Auction then Pari-mutuel • Others • HP - • ………. DIMACS

  20. Some Open Questions • There are many other mechanisms we did not test. • Pari-mutuel mechanisms • Economides, Lange, and Longitude (some combinatorics) • Pennock - Dynamic Pari-mutel Market • Plott - Auction then Pari-mutuel • Others • HP - • ………. • There are many other environments we did not test in. • Information monopolist • External incentives to manipulate internally • And for PAM -- do these results survive in an ultra-thin world? DIMACS

  21. A Force 12 Storm • Create an environment that really stress-tests the mechanisms • Six traders, 8 events w/ two outcomes each (256 states) • Common prior with asymmetric information • 10 draws from one urn of 8! equally likely: • (1,0,1,0,1,1,0,0), (1,1,0,0,0,0,0,0),….. • Each trader sees only 4 different entries: • (1,0,x,0,x,1,x,x), (1,0,x,0,x,1,x,x), … DIMACS

  22. posteriors priors uniform DIMACS

  23. Summary of Testing • Thin: 3 traders, 3 events • 7 independent prices from 3 people in 12 minutes • Markets < Individual Scoring Rule < Call < SSSR • SSSR ~ Call given that the group beats the prior • With selection, SSSR and Call Market do best. • Ultra-Thin: 6 traders, 8 events • 255 independent prices from 6 people in 12 min. • Markets ~ Individual Scoring Rule ~ Call < SSSR • SSSR beats the priors at the top (60%) • Nothing else even beats the priors • SSSR is only one with any aggregation DIMACS

  24. Final Thoughts • Information Markets are possible and desirable. • Can improve our ability to identify and deal with uncertainty. • Many policy applications will be in thin situations. • Traditional market designs do not work in thin situations. • Information monopolies, adverse decisions, partial updating • The SSSR (w/conditionals) definitely sharpens the signal/noise ratio in thin and ultra-thin markets over traditional markets. • Can we do better? Undoubtedly. DIMACS

  25. DIMACS

More Related