A Hard Knock Life: Why Analyst Accuracy Falls Short

1. A Hard Knock Life: Why Analyst Accuracy Falls Short QWAFAFEW May 25, 2010 Carson Boneck CFA, David Pope CFA

2. Abstract It is a commonly held belief among investors that there is persistence in security analyst's EPS forecast accuracy.  Studies by Clement and Brown examine persistence directly and show that, at least in the pre Reg-FD period, there is a measurable amount of analyst accuracy persistence.  We confirm this work and extend it to the Post-FD period, to the international markets, and then focus on the market profitability associated with persistence. Perhaps to many practitioners' (non quants) surprise, we find that persistence is not all that it is believed to be.  We show that while persistence is statistically measurable, it has very limited usefulness in predicting the future accurate analysts.  Our work shows that using past accurate analysts forecasts leads to somewhat better results than the naive mean, but falls well short of alternative methods.  Our work explores the metrics that make forecasts accurate rather than the individuals that make the forecasts themselves.

3. Agenda • Validate Clement / Brown Work • Update for Post Reg FD • Isolate importance of Past Skill • Past skill as indicator of future accuracy • Compare alternative metrics of forecasting: • Rock Stars, Time Weighted, Naïve Mean, BEST • Q&A

4. How Good are the “Experts”, really?

5. We want to BELIEVE – “Lead Analyst” Syndrome • The reality is forecasting is hard • Part of an Analyst’s role is to get noticed or get “shelf space” • To be the most accurate, a forecast by definition has to be an outlier or bold forecast • Analysts typically form their reputation by making one BIG call rather than being consistently right – Is Abby Joseph Cohen Accurate? • Surprisingly clients we spoke with had never actually looked at Analysts’ accuracy themselves • People like to believe in persistence of skill, or performance when it counts, often citing sports case studies • Hot Hand in Basketball - Gilovich, Vallone & Tversky (1985) • Clutch Hitting in Baseball – Cramer (1977)

6. Perfect Foresight is Very Profitable

7. Clement / Brown – Accuracy Persistence

8. Clement / Brown – Accuracy Persistence (Top Decile)

9. Isolate Age – Regress on Accuracy

10. Isolate Accuracy – Regress on Age

11. Skill Looks to Mean Revert (1990 – 2009)

12. Post FD – Story is the Same (2001-2009)

13. In the (not so) Long-Run All Analysts Are Average

14. High T-Stat, Low R^2 – Explaining a lot of nothing • Quantitative Slight of Hand • Neither Clement or Brown ever show R^2’s on past accuracy, only multiple regressions • Analogy: Think about medicine as an example. Maybe they can find out that coffee drinking has a statistically significant and even important effect on the probability of heart disease, but still not be able to predict whether or not you will suffer from heart disease, regardless of your coffee-drinking habits. - Leamer

15. How Do You Define Accuracy • Two Definitions of Accuracy • Demeaned Accuracy - Allows easy comparisons across stocks, time - [Abs(Estimate – Actual) – Abs(Average Error) ] / Abs( Avg Error) • Accuracy relative to Actual - Probably the first thing most investors think about when considering accuracy - Abs(Estimate – Actual) / Abs(Actual)

16. Rockstars Mean Revert (Post FD)Avg Error RockStar = 0, TimeWeighted = -.16

17. “RockStar” Analysts are BOLD, but not Accurate

18. The Reality of Analysts’ (Lack of) Accuracy

19. Even “Rockstars” don’t beat Naïve Mean Consensus

20. What, then, makes a Forecast more Accurate? • We know Analysts are not Accurate; they have more error than a naïve consensus • Literature proves us a guide into other areas • Clement, Lee Importance: 1000 Importance: 100 Importance: 10 Tenure Broker Size Estimate Age

21. Alternative – Focus on Factors that Make Forecast Accurate

22. Similar Results Internationally

23. Beware…. • Forecast Accuracy is not the same as market impact • Take an average analyst at a top brokerage • Compare to a skilled unknown analyst at a brokerage we never heard of….. • Analyst typically establish their reputation on ONE big call • II Analysts (Institutional Investor) Poll

24. Bibliography Bagnoli, Levine, Watts, 2003,”Analyst Estimation Revision Clusters and Corporate Events, Parts I and II”, Working Paper Beaver, Cornell, et al, 2008,” The Impact of Analysts’ Forecast Errors and Forecast Revisions on Stock Prices”, Journal of Business Finance & Accounting, 35(5) & (6), 709–740 Brown, Lawrence, 2001, “How Important is Past Analyst Forecast Accuracy?”, Financial Analysts Journal, Vol 57, pp 44-49. Chan, Jegadeesh, Lakonishok, 1996, “Momentum Strategies “, Journal of Finance Clement, M. B., 1999. “Analyst forecast accuracy: Do ability, resources, and portfolio complexity matter?”, Journal of Accounting and Economics, 27 (3): 285-303. Clement, Tse, 2003, “Do Investors Respond to Analysts' Forecast Revisions as If Forecast Accuracy Is All That Matters?”, The Accounting Review, Vol. 78 (1), 227-249 Doyle, Lundholm, Soliman, 2006, “The Extreme Future Stock Returns Following I/B/E/S Earnings Surprise”, Journal of Accounting Research, 44 (5) Elgers, P., M. Lo, and R. Pfeiffer,J r, 2001, “Delayed security price adjustment to financial analysts' forecasts of annual earnings”, The Accounting Review, 76 (4): 613-632. Gleason, C., and C. Lee, 2003, “Analyst forecast revisions and market price discovery”, The Accounting Review, 78 (1): 193-225. Mikhail, Walther, Lewis, 1999, “Does Forecast Accuracy Matter to Security Analysts?”,The Accounting Review, Vol. 74( 2), pp. 185-200 Sinha, Brown, Das, 1997, “A Re-Examination of Financial Analysts’ Differential Earnings Forecast Accuracy”, Contemporary Accounting Research; 14, 1;

25. Interesting Stats – US: FY1

26. Interesting Stats – Intl: FY1