1 / 33

Internet Economics כלכלת האינטרנט

Internet Economics כלכלת האינטרנט. C lass 9 – predictions and scoring rules. Today. Prediction markets and scoring rules. Some guidelines for the 2 nd semester. Wisdom of the crowds. The Internet is used to aggregate information from the general population.

ely
Download Presentation

Internet Economics כלכלת האינטרנט

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Internet Economicsכלכלת האינטרנט Class 9 – predictions and scoring rules

  2. Today • Prediction markets and scoring rules. • Some guidelines for the 2nd semester

  3. Wisdom of the crowds • The Internet is used to aggregate information from the general population. • Creation of “information markets”. • Main use: calculating probabilities of one-time events • Winners in elections. • Outcomes of sports events. • Political changes. • Financial events (future exchange rates, future stock values) • In large companies: elicit information from employees. • Probability that our new smartphone will gain more than 10% of the market.

  4. Information markets intrade

  5. Contracts • We need to sign a contract with the market participants. • Contract should be based on verifiable data. • Sometimes it is tricky…. • Israel will strike Iran by December 31st2011. • Robert De-Niro will win the 2011 academy awards for best actor in main role.

  6. Winner-takes-all contract • A bond is issued: • If event A occurs, it pays $100. • Otherwise, pays 0. • For example: Saddam security Paid out $100 if Saddam is no longer in power by June 2003. • Traded on website tradesports.com • In the hour after President Bush's war "declaration", the Saddam-gone-by-March bet shot up nearly 10 percentage points (odds went from about 60 to 69 percent) • Price on Jan 1, 2003: $55 • Price on March 1, 2003: $70

  7. Trading Contracts • How are contracts traded? • In most prediction markets, the mechanism used is the continuous double auction. • This is the mechanism used on the NYSE.

  8. Winner-takes-all Contracts • US 2008 elections – price history (from Iowa Electronic Market)

  9. Scoring rules • We sometime aim to more delicate contracts • For example, think about the following simple scenario:we do not want to aggregate the wisdom of the crowds, but the wisdom of one individual.

  10. Example: Weatherman • Tomorrow can be either sunny or rainy. • We would like to know the probability of a sunny day. • A weather forecaster knows, but he can lie. • Can we incentivize him to tell us the truth? 75% for a sunny day tomorrow

  11. Contracts • We need to sign a contract with the weatherman. • Contract with the weatherman can depend on: • The probability declared by the forecaster. • The realization of the outcome (sunny/rainy) • Contract cannot depend on the true probability • It is unobservable, even in retrospect.

  12. Proper scoring rules • We would like that given the contract, a utility-maximizing forecaster will report the true probability. • Via a bonus system. • Contracts where the forecaster strictly prefers to report the truth are called proper scoring rules. • It is easy to construct scoring rules that are not proper.

  13. Model • Consider a binary outcome: z=0 or z=1 • Example: z=1 means rain, z=0means sun. • p is the true probability of z=1. • Pr(z=1)=p, Pr(z=0)=1-p • q is the report of the forecaster. • The contract: • After realizing the outcome z, the forecaster paysR(z,q). • Are there proper scoring rules in this model?

  14. First try: pay your prediction • Consider the following contract: • R(z,q) = q • Pay the probability that you gave to the realized outcome. • i.e., when z=0 forecaster is paid R(0,q) = 1-q when z=1 forecaster is paid R(1,q) = q • Expected surplus for forecaster: • p*q + (1-p)*(1-q) = (1-p)+(2p-1)q • Forecaster will optimize q. • Report 1 if p>1/2, report 0 otherwise. • Not a proper scoring rule.

  15. Example: quadratic scoring rules • Consider the following contract: • R(z,q) = 1 - (z-q)2 • i.e., when z=0 forecaster is paid R(0,q) = 1-q2 when z=1 forecaster is paid R(1,q) = 1-(1-q)2 =q(2-q) • Expected surplus for forecaster: • p*R(1,q) + (1-p)*R(0,q) =pq(2-q)+(1-p)(1-q2) • Forecaster will optimize q. • FOC: • p(2-2q)+(1-p)(-2q) = 2p-2pq-2q+2pq = 0  q=p • A proper scoring rule (“quadratic scoring rule”)

  16. Example: logarithmic scoring rule • Consider the following contract: • R(z,q) = zlogq + (1-z)log(1-q) • i.e., when z=0 forecaster is paid R(0,q) = log(1-q) when z=1 forecaster is paid R(1,q) = logq • Expected surplus for forecaster: • p*R(1,q)+(1-p)*R(0,q) = p*log(q) + (1-p)* log(1-q) • FOC: • p(1/q)-(1-p)(1/(1-q))=0  q=p • A proper scoring rule (“logarithmic scoring rule”)

  17. Finding proper scoring rule A general result: • n possible events (1,…,n) • Probability estimate r=r1,…,rn • Scoring rule for each event si(ri) (i=1,…,n) • Expected score function: Theorem:(McCarthy ‘56, Hendrickson and Buehlera'71 )Every homogeneous and convex expected score function defines a proper scoring rule. • Example: quadratic scoring rule H(q)=q2(2-q)+(1-q)(1-q2)

  18. Summary • Often one would like to estimate probabilities of future events • Aggregate information from the crowds • Used in Information/prediction markets • Strong ties with financial securities trade • Options, etc. • Some beautiful economic theory behind it.

  19. Today • Prediction markets and scoring rules. • Some guidelines for the 2nd semester

  20. Grade • 30% presentation • 58% seminar paper • 12% participation • You will get 12 points if you attend at least 80% of the classes in both the 1st and the 2nd semester. • If you miss the 80% in one of the semesters, you get 0 points. • You will not get a grade in the course if you attend less than 50% of the classes in at least one of the semesters. • 12% problem set (“magen”)

  21. Course duties: presentation • In pairs. • Let the better speaker talk, but two speakers possible. • Practice • Remember: presentations in real-life are much faster than the practice talks. • 25 minute talk. (~3 in each meeting) • Schedule will be posted in the beginning of the semester break. • You should be ready to present in the first week of the semester. • Make it interesting to the class! • Speak to class, not to instructor. • Language: Slides in English (preferred, not mandatory), Speak Hebrew.

  22. Course duties: presentation • Goal: present the paper to the class. • Present what is written in the paper, no need for criticism or your interpretation. • Give the background, and make the main results clear(do not present all results). • If suitable, spend at least 5-10 minutes on technical details (proof idea, analysis methods etc.) • Survey related work very briefly, only if needed. (sometimes it is good to read some of these…) • Don’t follow the structure of the paper exactly. • Explain in your own words.

  23. Course duties: presentation • Presentation grading will be based on: • Clarity. • Understanding. • Identifying and understanding the main results. • Quality of presentation. • Conveying complicated ideas in a simple way. • Presenting your own understanding (not just cut and paste).“Make everything as simple as possible, but not simpler. ” Albert Einstein 

  24. Some presentation tips. • Use large fonts • in general, rarely use fonts below 24. • Don’t have dense slides – split them. • A figure is worth a thousand words:If x is an individually rational revenue maximizing auction then there exists a Pure Nash equilibrium in the game. • Of course: Be politically correct • No offensive language, images, examples etc. X is individually rational revenue maximizing auction Pure Nash equilibrium exists

  25. Some presentation tips. • Use colors. • Use images. • Use animation • But with care, do not overuse it please • Make it interesting, but remember that this is an academic duty. Content is the main thing. • Most effort should go on understanding the paper you read.

  26. Some presentation tips. • Basic presentation structure: • Motivation (usually, real life problem, informal) + Background • Informal results (if possible) • Model • Results • Technical details • Summary • Of course, you should fit the structure to your paper.

  27. Course duties: Problem set • Optional (Magen12%) • Strict deadline:February 1st • By your own. (not in pairs.) • Available tomorrow on the website.

  28. Course duties: Seminar paper • Should be on the same topic as your presentation • With the same partner. • Up to 10 pages, font 12, 1.5 spacing. (strict!) • Submission time: May 30th.

  29. Course duties: Seminar paper In the paper: Part 1: Summarize the paper. Background, main results, main technical details . • Some proofs (outline if needed), methods of data analysis. (Tables with data should not be part of the 10 pages you submit, can be in an appendix.) • Go to depth regarding the main result/s. Part 2:A new research direction, • Phrase exact questions and challenges, one (small) step toward to solution. • Idea may be theoretical, empirical or other. You need to define the framework of the research (model, goals, questions, difficulties, data resources, suggested theorems)

  30. Course duties: Seminar paper Originality: • Cite all materials that you use. • Statements that are not from the paper you read should be given with a bibliographic reference. • Especially, let me know if you intend writing any project for another course on a similar subject. • Do not copy text, even from the paper you read. • Explain in your own language. Give new examples. Give the intuition behind the proofs.

  31. All seminar topics are chosenand approved(January 1st) Timeline Problem set due date (Feb. 1st) Students presentations start(February16thth) All seminar papers are submitted(May30th) Presentation outline + hard scheduling constraints sent to Avi. (January 12th) Presentation schedule posted (January 20th) presentations Summervacation starts Winter vacation starts Winter vacation ends

  32. Good luck. Don’t hesitate to come to me for questions or advise.

More Related