Portfolio Performance Measurement › Limitations and Practical Considerations

Understanding the limitations of performance metrics is as important as computing them correctly. Each metric makes implicit assumptions that may not hold in practice.

The Sharpe ratio assumes returns are normally distributed, which conflicts with the stylized facts we discussed earlier in the book. Financial returns exhibit fat tails, skewness, and time-varying volatility, all of which the Sharpe ratio ignores. A strategy that earns steady small gains but occasionally suffers catastrophic losses (like selling volatility) can have an attractive Sharpe ratio despite its unappealing risk profile.

The Sharpe ratio is also sensitive to the measurement period and frequency. The same strategy can have different Sharpe ratios when measured daily versus monthly because annualization choices and autocorrelation in returns affect how returns aggregate. Strategies with positive autocorrelation (momentum) will have higher Sharpe ratios at longer frequencies, while mean-reverting strategies show the opposite pattern.

The choice of risk-free rate also matters more than practitioners often acknowledge. During periods of near-zero interest rates, this seems trivial, but historically and in certain currencies, the risk-free rate significantly affects computed Sharpe ratios.

The information ratio inherits the Sharpe ratio's assumption issues and adds benchmark selection risk. Choosing an inappropriate benchmark can make a mediocre manager look skilled or a skilled manager look mediocre. A small-cap value manager benchmarked against the S&P 500 will generate tracking error simply from style differences, not from active decisions.

The information ratio also assumes that alpha and tracking error are stable over time, which is rarely true. Managers may take more active risk in certain market conditions or when they have higher conviction. Averaging across these varying regimes can produce misleading summary statistics.

Maximum drawdown is inherently sample-dependent. It measures the worst observed outcome, but future drawdowns could be worse. The longer the track record, the larger the expected maximum drawdown, simply because more opportunities for drawdowns have occurred. Comparing maximum drawdowns across managers with different track record lengths is problematic.

Maximum drawdown is also highly path-dependent and sensitive to the specific sequence of returns. Two managers with identical return distributions but different timing will have different maximum drawdowns. This can make the metric feel arbitrary when a manager's maximum drawdown occurred due to a one-time market event rather than systemic strategy weaknesses.

When evaluating historical performance, survivorship bias can inflate apparent results. Failed funds are not included in historical databases, making the average surviving fund look better than a randomly selected fund at inception would have been. Similarly, look-ahead bias occurs when performance measurement uses information that was not available at the time decisions were made. Using in-sample optimized weights to compute backtested Sharpe ratios produces overly optimistic results.

All computed metrics are estimates subject to sampling error. A Sharpe ratio of 0.8 based on three years of monthly data has substantial uncertainty around it. The standard error of the Sharpe ratio estimator is approximately:

$$\text{SE}(\widehat{\text{SR}}) \approx \sqrt{\frac{1 + \frac{\text{SR}^2}{2}}{n}}$$

where:

• $\widehat{\text{SR}}$: estimated Sharpe ratio
• $\text{SR}$: true Sharpe ratio
• $n$: number of observations

The term inside the square root accounts for uncertainty in estimating both the mean (the $1$) and the variance (the $\frac{\text{SR}^2}{2}$). For monthly data over three years ($n = 36$) with a true Sharpe ratio of 0.8, the standard error is roughly 0.18, meaning a 95% confidence interval spans from about 0.44 to 1.16. This wide range highlights how difficult it is to distinguish skill from luck with limited data.

The same caution applies to the information ratio. A positive IR based on limited data may simply reflect sampling variation rather than manager skill. Statistical tests for the significance of alpha, which we'll cover in the Performance Attribution chapter, help address this uncertainty.