Benchmarking money manager performance: Issues & evidence

1. Benchmarking money manager performance: Issues & evidence Louis K. C. Chan University of Illinois Urbana-Champaign March 2006

2. Objectives • The evaluation and attribution of investment performance is crucial for investment research and practice • Money manager performance • Results of investment strategies & trading rules • Effects of managerial decisions on shareholder wealth • Academic and practitioner research has produced a large array of methods for evaluating and attributing investment performance

3. Objectives • Question: are conclusions sensitive to the choice of evaluation and attribution methods? why? • We compare the results from various methods applied to common samples • Set of active institutional money managers • Passive indexes

4. Evaluating method performance • Many widely-used methods draw on evidence from asset pricing studies that size, value/growth describe much of the variation in returns (notably Fama and French (1992), Fama and French (1993)) • We concentrate on benchmarking methods based on size, value/growth • Characteristic-matched control portfolios • Time-series factor model regressions • Effective asset mix regressions • Cross-sectional regressions on characteristics • 1998 – 2000 market boom as stress test of benchmarking methods

5. Evaluating manager performance • Much previous work on evaluating performance of mutual and closed-end funds (e.g. Jensen (1968), Elton et al. (1993), Malkiel (1995), Gruber (1995), Carhart (1997), Daniel et al. (1997), Kothari and Warner (2001), etc.) • Managers of pension plan equity assets are just as important, but much less previous research (see LSV 1992, Coggin et al. 1993)

6. A first look: characteristic-matched portfolios vs. 3 factor model

8. Benchmark details • Benchmarks vary according to • Characteristics or loadings • Measuring size, value/growth style • Treating size, value/growth effects separately • Portfolio weighting scheme • Frequency of benchmark reconstitution

9. Benchmark details • Characteristics versus loadings • Predict benchmark return using portfolio’s attributes (size, book-to-market …) or predict benchmark return using portfolio’s loadings on factors • Some evidence that attributes predict returns better than loadings (Daniel and Titman 1997) • Data on holdings not generally accessible

10. Building performance benchmarks • Measuring size, value/growth style • Size: market capitalization (float?) • Value/growth orientation usually measured by book-to-market ratio (book value of equity divided by market value of equity) • Book value of equity does not record value of intangible assets; includes goodwill from acquisitions

11. Building performance benchmarks • Treating size, value/growth effects separately • E.g. independent 2-way sorts by size, BM • In one-way sorts by book-to-market equity large stocks typically are classified as growth • Under an independent size/BM sort procedure large-cap managers, regardless of large value/large growth style, will tend to be compared against a growth benchmark

12. Building performance benchmarks • Weighting scheme for stocks in benchmark • Equal-weighting • Value-weighting • Benchmark reconstitution frequency • Over time benchmark becomes more heterogeneous and may no longer correspond to managed portfolio’s features

13. Data • Holdings and returns every quarter for 199 portfolios offered by money managers to clients, 1989Q1 - 2001Q4 • Domestic U.S. equity portfolios only • Different styles (large/mid/small, value/blend/growth) • Some selection bias

14. Results outline • Performance relative to benchmarks based on characteristics • Overall active manager sample • Classified by investment style • Diagnostics • Performance relative to benchmarks based on loadings • Overall active manager sample • Classified by investment style • Diagnostics

15. Performance measures • Abnormal return = portfolio’s return minus return on benchmark portfolio • Tracking error volatility = standard deviation of quarterly difference between portfolio’s return and benchmark’s return

19. Benchmark performance

20. Benchmark performance

21. Benchmark comparisons

24. Performance based on regression benchmarks • Three factor model excess return is ( rpt – rft ) – benchmark return • benchmark return is from fitted regression β(rmt – rft ) + s SMBt + h HMLt

25. Regression-based benchmark details • Exposures estimated • over full period (including the quarter when we measure performance) • or leaving out the quarter when we measure performance • Measuring size, value/growth factors • High versus low book-to-market • Other indicators of value/growth orientation

26. Building regression-based benchmarks • 3 factor model accounts for size, value/growth separately • E.g. benchmark return for small value manager = return for market exposure plus return for smallness plus return for value • Benchmark credits manager for smallness even though small stocks’ performance is because small growth does better than small value

27. Regression-based benchmarks • Alternative: compare manager to a selection of passive benchmarks (effective asset mix regressions) rpt = α + w1*LGt + w2*LVt + w3*MCGt + w4*MCVt + w5*SGt + w6*SVt + υpt w1, … ,w6 portfolio weights (between 0 and 1, add up to 1)

28. Building regression-based benchmarks • Another widely-used alternative: each stock’s predicted return is from a cross-sectional regression using stock characteristics, industry dummy variables rit = α + β1*X1i + β2*X2i + …

32. Regression-based benchmark comparisons

33. Regression-based benchmark comparisons

35. Conclusions • Benchmarking methods that appear similar on the surface can lead to very different conclusions about investment performance • Popular methods (characteristic-matched reference portfolios, 3 factor time series regression models, cross-sectional regression) have disappointing ability to track managed active portfolios and passive benchmarks

36. Conclusions • Methods based on within-size classifications, use multiple measures of value-growth orientation, improve ability to track managed and passive portfolios • Given the fragility in reliably separating skill from style, detailed decomposition and attribution of performance should be treated with caution