The Roles of Uncertainty and Randomness in Online Advertising

1. The Roles of Uncertainty and Randomness in Online Advertising Raga Gopalakrishnan 2nd Year Graduate Student (Computer Science), Caltech Eric Bax Ragavendran Gopalakrishnan Product Manager (Marketplace Design), Yahoo!

2. Display Advertising AD-SLOT

3. Simple Model for Display Advertising webpage feedback AD-SLOT ad calls AD SELECTION ALGORITHM ads w/ bids implement resultant matching (selected ad for each ad call)

4. Objective  Make Money! ad slot m ad calls • May not be the right thing to do, for two reasons: • Reason 1: Not Incentive Compatible • Reason 2: Coming up… k2 kn k1 ad 1 ad 2 ad n . . . Bid Value b2 bn b1 . . . s2 sn Response Rate s1 ?

5. The Caveat • The response rate is not known, it has to be estimated. • The actual revenue differs from the estimated expected revenue due to two factors: • Uncertainty (error in estimating response rates si) • Randomness (fluctuations around the response rate: )

6. How bad can Uncertainty be? ad slot billion ad calls per day AD 1 AD 2 Bid Value $1 per response $1 per response 0.0007 w/ prob ½ 0.0013 w/ prob ½ Estimated Response Rate 0.001 w/ prob 1 $1 million $1 million Estimated Expected Revenue Standard Deviation of Revenue $1000 (0.1%) $0.3 million (30%)

7. How can we combat it? How much time do we have? • Again, these solutions are not automatically incentive compatible. Long-Term Short-Term LEARNING RISK SPREADING ? MAIN FOCUS Future Work

8. Model for Variance of Revenue ad slot m ad calls k2 kn k1 ad 1 ad 2 ad n . . . Bid Value b2 bn b1 . . . Response Rate S1 S2 Sn . . . . . . X2(S2) Xi(Si) Xn(Sn) Revenue X1(S1) . . . X2i(Si) Xmi(Si) X1i(Si)

9. Model for Variance (contd.) • The variance of the revenue can be derived as: • Independent Returns Case: UNCERTAINTY RANDOMNESS

10. Factors affecting Variance ad 1 k ad calls S Mean = p Std. Dev. = d*p Fraction of Variance Due to Uncertainty is X(S) is Bernoulli w/ parameter S

11. Uncertainty or Randomness?

12. Bottom Line Uncertainty can be really bad Real World – Uncertainty dominates SOLUTION Long-Term Short-Term LEARNING RISK SPREADING

13. Effect of Learning preal : Real response rate (unknown) ad 1 ad 1 u responses v learning ad calls k ‘real’ ad calls Fraction of Variance due to Uncertainty is Estimate preal as p = u/v

14. Bottom Line Uncertainty can be really bad Real World – Uncertainty dominates SOLUTION Long-Term Short-Term LEARNING RISK SPREADING

15. Effect of Risk Sharing AD 1 AD 2 billion ad calls per day Bid Value $1 per response $1 per response 0.0007 w/ prob ½ 0.0013 w/ prob ½ Estimated Response Rate 0.001 w/ prob 1 $1 million $1 million Estimated Expected Revenue $1000 (0.1%) $0.3 million (30%) Standard Deviation of Revenue Variance of Revenue 0 + 1000000 90000000000 + 1000000 New Strategy: Use each of a billion ads iid to AD 2 on each ad call Variance of revenue = 90 + 1000000

16. Formalize Risk-Sharing • The goal of sharing risk and bringing the variance down motivates the following optimization problem:

17. Simulations generate response rates 10 “CPC” ADS Bid $1 Normal Distribution m = 0.001, s = 0.0001 • Start with an assumed prior (uniform, approximate or exact) • All 20 ads are given 100000 learning ad calls each, responses are counted, corresponding posteriors are obtained using Bayes’ Rule • Method 1 (Portfolio): Compute the optimal portfolio and allocate ad calls accordingly • Method 2 (Single Winner): Allocate all ad calls to the ad with the highest estimated expected revenue • Compare Results generate response rates 10 “CPA” ADS Bid $10 Normal Distribution m = 0.0001, s = 0.00001

18. Estimated Expected Revenue

19. Uniform Prior – Actual Expected Revenue

20. Uniform Prior – Efficiency

21. Uniform Prior – Allocation by share of ad calls

22. Uniform Prior – Allocation by actual expected revenue

23. Exact Prior – Actual Estimated Revenue

24. Exact Prior – Allocation by share of ad calls

25. Exact Prior – Allocation by actual expected revenue

26. Approximate Prior – Actual Expected Revenue

27. Approximate Prior – Allocation by share of ad calls

28. Approximate Prior – Allocation by actual expected revenue

29. A Word of Caution – Covariance • Randomness is usually uncorrelated over different ad calls. • More often than not, uncertainty is correlated over multiple ads, as their response rates could be estimated through a common learning algorithm. • Covariance can be estimated from empirical data, using models that are specific to the contributing factors (e.g., specific learning methods used).

30. Summary • Actual Revenue differs from Estimated Expected Revenue for two reasons – uncertainty and randomness. • Uncertainty can be very bad, and dominates randomness in most cases. • Learning helps reduce uncertainty in the long run, but in the short run, portfolio optimization (risk distribution) is one way to combat uncertainty. • Simulations show that actual revenue can improve as an important side effect of reducing uncertainty.

31. Further Directions… • Can we tie up the long term and short term solutions? • Example: Consider the explore-exploit family of learning methods. • After every explore step, we have better estimates of response rates, but they may still be bad. So the exploit phase could be replaced with the portfolio optimization step! • Side Effect: Additional exploration in the “exploit” phase. • Is this an optimal way of mixing the two? • Financial Markets – does it make sense for risk-neutral investors to employ portfolio optimization? • Incentive Compatibility – can we deal with it?

32. Thank You • Questions?