How to Evaluate An Investment Manager Jessica N. Portis, CFA Senior Vice President

1. How to Evaluate An Investment Manager Jessica N. Portis, CFASenior Vice President SummitStrategies Group 8182 Maryland Avenue, 6th FloorSt. Louis, Missouri 63105

2. Manager Selection Liability Management Funding Rate Volatility Government Regulation Rate of Return and Funding Issues Liquidity Inflation Risk Asset Allocation Market Volatility Risk Management * Greenwich Associates, Market Trends 2013, USII-13

3. PUBLIC FUND MANAGER HIRING ACTIVITY* * Greenwich Associates, Market Trends 2013, USII-13. “Have Hired” refers to mandates or assignments US institutions have hired for – additional or replacement – in the past 12 months. “Expect to Hire” refers to anticipated hiring for mandates or assignments in the next 12 months.

4. Clients find it easy to hire managers Thrill of the Chase Attention from Managers Intellectually Challenging Exercise “Doing” Something Good Source of Information for Personal Accounts It’s Fun to Go Shopping

5. Clients find it difficult to terminate managers YOU’RE FIRED! • Admitting a mistake. • Personal relationships established. • Probably less today than 10 years ago. • The crowd mentality. • Politics • Local • Minority • Timing • Don’t want to sell at the bottom. • It’s costly to switch managers. • Manager’s reason for poor performance or personnel changes appears reasonable.

6. Active management makes for an interesting media topic Why 'active' investment is a losing bet • 3/23/2012 – Moneywatch by Larry Swedroe Medieval Medicine And Active Fund Management • 7/25/2013 – Personal Finance by Rick Ferri Active Managers Stink? Blame These All-Too-Common Fund Flaws • 10/22/2013 – Focus on Funds by Brendan Conway The Problem for Active Managers • 4/29/2014 – Opinion by Pauline Skypala Fama’s Nobel Work Shows Active Managers Fated to Lose • 10/15/2013 – Charles Stein

7. Alpha does exist; you just have to know where to find it Relatively consistent and high excess return Very consistent and high excess return Very consistent and positive excess return Closet index strategies

8. Multi dimensional approach to due diligence process

9. Client constraints = search criteria • Detail Client Constraints Candidate Pool • Desirable Manager Attributes: • Experience • Firm tenure • Investment team “pedigree” • Product history • Historical Performance • Relative to peers • Relative to benchmark • Level of Assets Under Management • Minimum size • Number of clients • Client diversity • Preference for local, emerging or minority manager

10. define expectations for the mandate and manager

11. Why setting expectations is important • Helps frame your goals and objectives. • Focus your efforts on meaningful/impactful strategies. • Improves your odds of being “satisfied” with the manager. • Narrows down the list of managers you have to perform in-depth due diligence. • Makes your due diligence process more efficient. • Tedious process that involves the review of both quantitative and qualitative aspects of firm.

12. EFFICIENCY OF ASSET CLASS – ESTABLISHING REALISTIC ALPHA EXPECTATIONS Emerging Markets US Large Cap Value Median Annualized Excess Return: 1.75% Median Annualized Excess Return: 0.18% *Rolling 1-yr Quarterly 1994-201. Source Evestment Alliance, Large Value - 733 Strategies/ Emerging Markets - 497 Strategies (active & inactive)

13. UNDERSTANDING WHERE YOU ARE IN THE ECONOMIC CYCLE AND THE ALPHA POTENTIAL FOR SUB STRATEGIES • Defining Expectations for the Mandate Large Cap Value Manager Sub Strategy Performance

14. IMPACT OF TRACKING ERROR ON RESULTS: WHAT ARE YOU COMFORTABLE WITH? Risk Tolerance: Relationship Between Excess Return and Tracking Error. Relationship between excess return and tracking error. 27,889 rolling 3-year periods for 723 US large cap value strategies 1994-2014. Source: Evestment Alliance. Observation: Historically, strategies utilizing a high-conviction investment styles performed best within the US Large Cap Value market segment. Takeaway: Align your alpha expectations with your risk tolerance.

15. PERFORMANCE EVALUATION DILEMMA • Good managers sometimes appear “bad”. • The gut reaction is to terminate poor-performing managers. • What makes a poor manager? • How to evaluate track record?

16. HOW TO EVALUATE TRACK RECORDS • Avoid simply relying on recent performance to evaluate a manager – it can be dangerous. • Incorporate elements of risk and return. • Standard Deviation • Information Ratio • Tracking Error • Batting Average • Up/Down Market Capture • Rolling period evaluation is much more telling than a single point in time. • Consistency of results. • Helps isolate luck versus skill. • Not constrained by point in time comparisons. • Assess the effect of macro economic conditions on results.

17. A CASE STUDY Annualized Results Ending 9/30/2007 Trailed 8 of last 12 quarters with no positive alpha Trailed 13 of last 20 quarters with no positive alpha One-Year Rolling/Quarterly Percentile Ranking Question: Should this manager be terminated? Answer: It Depends! Performance only captures one dimension. Need to qualitatively understand the manager to better assess future prospects for success.

18. Product Assessment Personnel Issues MANAGER AVOIDANCE ≥ MANAGER SELECTION Philosophy & Process Firm Specific Issues “I’d rather be roughly right than precisely wrong.” - John Meynard Keynes Due Diligence Is Focused on Risk Management.

19. qualitative assessment: firm Ownership Leadership “I don’t think culture is something you can describe.” – Bill Gates Product Integration Compensation Litigation Business Mgmt. Operational Infrastructure FIRM

20. qualitative assessment: product Strategy History Client Type Diversity AUM “I don’t want a lot of good investments; I want a few outstanding ones.” – Philip Fisher Culture Star vs. Team Management Capacity PRODUCT ASSESSMENT

21. qualitative assessment: personnel Team Stability Team Resources Add’ l Team Pedigree Team “Coming together is a beginning. Keeping together is progress. Working together is success.” – Henry Ford Size & Depth Team Experience PERSONNEL

22. qualitative assessment: philosophy and process Philosophy Investment Process Performance Drivers Portfolio Characteristics “If you don’t have a competitive advantage, don’t compete.” -Jack Welch Trading Decision Process PHILOSOPHY & PROCESS Risk Culture

23. TRIGGERS FOR A FORMAL REVIEW OF A MANAGER Firm – Material ownership change or regulatory concern. Product – Change in assets under management, style drift, or dilution/orphan of product. Personnel – Departure of key investment professional or change in team structure and decision process. Philosophy and Process – Change in investment philosophy, shift in market environment not conducive to style, or relaxation of risk controls. Performance – Three-year excess return outside of original investment thesis or deterioration in quantitative criteria – information ratio, tracking error, standard deviation, batting average, up-/down- capture ratio, etc. Strategic Fit in Client Portfolio – Does the strategy still make sense in terms of your needs?

24. Revisiting the case study

25. Case study: the complete picture Five-Year Rolling/Quarterly Percentile Ranking A high conviction manager underperforms badly: do you stay or go? • Was the firm sold? No. • Had the AUM changed dramatically? No. • Did the team experience turnover? No. • Did the investment process change? No. • Were the markets rewarding their approach? No! Quality and valuation were out of favor for several years. Growth of $1.00 Universe: International Value Equity (Separate Accounts & Commingled Funds)

26. conclusion Quantitative Review + Qualitative Review + Ongoing Monitoring = Success • Clearly define the role the manager and strategy play in the overall portfolio. • Understand both the opportunities and threats associated with a manager prior to investing. • Define performance expectations for each manager. • What does the historical pattern of returns for a manager look like? • Understand which strategies work best in the asset class and when. • When issues arise, investigate thoroughly and compare to documented expectations. • Act quickly when determining that a termination needs to occur.

27. QUESTIONS AND ANSWERS Disclaimer: Summit Strategies Group (Summit) has prepared this report for the exclusive use by the client for which it was prepared. The information herein was obtained from various sources, such as the client’s custodian(s) accounting statements, commercially available databases, and other economic and financial market data sources. While Summit believes these sources to be reliable, Summit does not guarantee nor shall be liable for the market values, returns or other information contained in this report. The market commentary, portfolio holdings and characteristics are as of the date shown and are subject to change. Past performance is not an indication of future performance. No graph, chart, or formula can, in and of itself, be used to determine which securities or investments to buy or sell. Any forward-looking projection contained herein is based on assumptions that Summit believes may be reasonable, but are subject to a wide range of risks, uncertainties and the possibility of loss. Accordingly, there is no assurance that any estimated performance figures will occur in the amounts and during the periods indicated, or at all. Actual results and performance will differ from those expressed or implied by such forward-looking projections. Any information contained in this report is for information purposes only and should not be construed to be an offer to buy or sell any securities, investment consulting or investment management services.