Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

Backtesting & Simulation: Frameworks for Strategy Validation › Performance Metrics

Michael BrenndoerferJanuary 7, 202610 min read

Quantitative FinanceAll chapters

Part 14 of 15Strategy Development and Backtesting

Chapter 73Backtesting & Simulation: Frameworks for Strategy Validation › Performance Metrics

Reading section 4 of 9 · Performance Metrics · 10 minSection 4 of 9 · 10 min

Section 4 of 9 · 10 min read

Backtesting & Simulation: Frameworks for Strategy Validation › Performance MetricsOne section · Sections in this chapter4 / 9 ↓

Sections in this chapter9

Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Once you've run a bias-free backtest, you need metrics to evaluate the results and understand the risk-return characteristics of your strategy. As we discussed in Part IV on portfolio performance measurement, returns alone are insufficient for making informed decisions. We need risk-adjusted measures that account for the volatility and drawdowns incurred to achieve those returns. A strategy that earns 20% annual returns with 10% volatility is quite different from one that earns 20% with 40% volatility, even though the raw returns are identical.

Return MetricsLink Copied

The foundation of performance measurement is the return series, which captures how the portfolio's value changed over time. From the simulated portfolio values, we compute various return statistics that describe the strategy's profitability and growth characteristics.

In[15]:

Code

def calculate_return_metrics(portfolio_values):
    """
    Calculate basic return metrics from a portfolio value series.
    """
    # Convert to pandas if needed
    if isinstance(portfolio_values, np.ndarray):
        portfolio_values = pd.Series(portfolio_values)

    # Simple returns
    returns = portfolio_values.pct_change().dropna()

    # Total return
    total_return = portfolio_values.iloc[-1] / portfolio_values.iloc[0] - 1

    # Annualized return (assuming 252 trading days)
    n_years = len(returns) / 252
    annualized_return = (1 + total_return) ** (1 / n_years) - 1

    # Annualized volatility
    annualized_vol = returns.std() * np.sqrt(252)

    return {
        "total_return": total_return,
        "annualized_return": annualized_return,
        "annualized_volatility": annualized_vol,
        "daily_returns": returns,
    }

The Sharpe Ratio and Its VariantsLink Copied

The Sharpe ratio, covered extensively in Part IV, remains the dominant risk-adjusted performance measure in quantitative finance. It normalizes returns against risk, allowing for fair comparison between strategies with different volatility profiles. The fundamental insight behind the Sharpe ratio is that returns should be evaluated relative to the risk taken to achieve them. A high-return, high-risk strategy may be less attractive than a moderate-return, low-risk strategy when viewed through this lens.

The Sharpe ratio expresses excess return per unit of volatility, and gives a standardized measure of risk-adjusted performance:

\text{Sharpe Ratio} = \frac{R_p - R_f}{\sigma_p}

To understand this formula, let's examine each component. The term $R_p$ represents the annualized portfolio return, measuring how much the strategy earned over the evaluation period expressed as an annual rate. The term $R_f$ denotes the risk-free interest rate, typically proxied by Treasury bill yields, which represents the return you could have earned without taking any market risk. The difference $R_p - R_f$ is the excess return generated by the strategy over the risk-free alternative, capturing the additional compensation you received for bearing the strategy's risks. Finally, $\sigma_p$ is the annualized standard deviation of portfolio returns, commonly called volatility, which measures the typical magnitude of return fluctuations around the mean.

The Sharpe ratio thus answers a practical question: how much excess return did the strategy generate for each unit of risk it assumed? A Sharpe ratio of 1.0 means the strategy earned 1% of excess return for every 1% of volatility. Higher values indicate more efficient risk-reward tradeoffs, while negative values indicate the strategy failed to beat the risk-free rate.

In[16]:

Code

def calculate_sharpe_ratio(returns, risk_free_rate=0.02, periods_per_year=252):
    """
    Calculate annualized Sharpe ratio.

    Parameters:
    -----------
    returns : array-like
        Period returns (daily, monthly, etc.)
    risk_free_rate : float
        Annual risk-free rate
    periods_per_year : int
        Number of periods per year (252 for daily, 12 for monthly)
    """
    excess_returns = returns - risk_free_rate / periods_per_year

    if excess_returns.std() == 0:
        return 0

    sharpe = excess_returns.mean() / excess_returns.std()
    annualized_sharpe = sharpe * np.sqrt(periods_per_year)

    return annualized_sharpe

The Sharpe ratio has important limitations that you should keep in mind when using it to evaluate strategies. It penalizes upside and downside volatility equally, treating a strategy that experiences large gains the same as one that experiences large losses. Additionally, it assumes returns are normally distributed, which may not hold for strategies with fat tails or asymmetric return profiles. Several alternative metrics address these limitations by focusing on specific aspects of risk that the Sharpe ratio overlooks.

Alternative metrics address specific limitations of the Sharpe ratio:

Sortino Ratio: Uses downside deviation (volatility of negative returns) instead of total volatility. This rewards strategies that achieve volatility through upside gains rather than losses.
Calmar Ratio: Divides annualized return by maximum drawdown. This highlights the worst loss experience, which is critical for assessing psychological and capital feasibility.
Information Ratio: Measures excess return relative to a benchmark divided by tracking error, capturing the consistency of outperformance.

In[17]:

Code

def calculate_sortino_ratio(returns, risk_free_rate=0.02, periods_per_year=252):
    """
    Sortino ratio: like Sharpe but using downside deviation only.
    """
    excess_returns = returns - risk_free_rate / periods_per_year

    # Downside returns only
    downside_returns = excess_returns[excess_returns < 0]

    if len(downside_returns) == 0 or downside_returns.std() == 0:
        return np.inf  # No downside

    downside_deviation = downside_returns.std()
    sortino = excess_returns.mean() / downside_deviation

    return sortino * np.sqrt(periods_per_year)

Maximum DrawdownLink Copied

Maximum drawdown measures the largest peak-to-trough decline in portfolio value. It represents the worst loss you would have experienced if you had the misfortune of buying at the top and selling at the bottom during the evaluation period. This metric captures a dimension of risk that volatility-based measures may miss: the severity of the worst period of losses.

To calculate drawdown at any point in time, we first need to establish the high-water mark, which is the highest portfolio value achieved up to that moment. The drawdown is then the percentage decline from that peak:

\text{Drawdown}_t = \frac{V_t - \max_{s \leq t}(V_s)}{\max_{s \leq t}(V_s)}

In this formula, $V_t$ represents the portfolio value at time $t$ , while $\max_{s \leq t}(V_s)$ denotes the high-water mark, which is the highest portfolio value observed up to time $t$ . The numerator $V_t - \max_{s \leq t}(V_s)$ gives the drawdown magnitude, measuring the current loss relative to the running peak. This value is always zero or negative, with more negative values indicating larger declines from the peak.

This formula quantifies the "pain" you experience at any point in time relative to your previous high-water mark. When the portfolio reaches a new high, the drawdown is zero. When the portfolio declines, the drawdown becomes negative, showing how far the portfolio has fallen from its best performance.

The maximum drawdown measures the worst loss over the entire evaluation period by finding the minimum, that is, the most negative, value in the drawdown series:

\text{Max Drawdown} = \min_t(\text{Drawdown}_t)

Here, $\text{Drawdown}_t$ is the percentage decline calculated at each time step $t$ , and the $\min_t$ operator selects the most negative value from the entire series, representing the deepest trough reached during the evaluation period.

In[18]:

Code

def calculate_drawdowns(portfolio_values):
    """
    Calculate drawdown series and maximum drawdown.
    """
    # Running maximum
    running_max = portfolio_values.cummax()

    # Drawdown at each point
    drawdowns = (portfolio_values - running_max) / running_max

    # Maximum drawdown
    max_drawdown = drawdowns.min()

    # Find the dates of the max drawdown
    max_dd_end = drawdowns.idxmin()
    max_dd_start = portfolio_values[:max_dd_end].idxmax()

    return {
        "drawdowns": drawdowns,
        "max_drawdown": max_drawdown,
        "max_dd_start": max_dd_start,
        "max_dd_end": max_dd_end,
    }

Maximum drawdown is particularly important because it relates directly to investor psychology and practical capital requirements. A strategy might have excellent risk-adjusted returns on paper but require holding through a 50% loss, which few investors can tolerate psychologically or financially. Many investors will abandon a strategy during a large drawdown, realizing losses at the worst possible time. Understanding the maximum drawdown helps set realistic expectations and determine appropriate position sizing.

Win Rate and Profit FactorLink Copied

Beyond volatility-based metrics, traders often examine win/loss statistics that describe the distribution of individual trading outcomes. These metrics provide intuition about how the strategy generates its returns and can help identify potential problems.

In[19]:

Code

def calculate_trade_statistics(returns):
    """
    Calculate win rate, profit factor, and related statistics.
    """
    winning_periods = returns[returns > 0]
    losing_periods = returns[returns < 0]

    win_rate = len(winning_periods) / len(returns)

    # Average win vs average loss
    avg_win = winning_periods.mean() if len(winning_periods) > 0 else 0
    avg_loss = abs(losing_periods.mean()) if len(losing_periods) > 0 else 0

    # Profit factor: gross profits / gross losses
    gross_profit = winning_periods.sum()
    gross_loss = abs(losing_periods.sum())
    profit_factor = gross_profit / gross_loss if gross_loss > 0 else np.inf

    # Expectancy: average profit per trade
    expectancy = returns.mean()

    return {
        "win_rate": win_rate,
        "avg_win": avg_win,
        "avg_loss": avg_loss,
        "profit_factor": profit_factor,
        "expectancy": expectancy,
    }

Performance ReportLink Copied

Let's combine these metrics to analyze the strategy from several performance and risk perspectives:

In[20]:

Code

def run_momentum_backtest(prices, lookback=20):
    """
    Run a momentum strategy backtest and return portfolio values.
    """
    returns = prices.pct_change()

    # Momentum signal based on past returns
    # shift(1) ensures signal at t uses data up to t-1
    momentum = returns.rolling(window=lookback).mean().shift(1)

    # Position: +1 when momentum positive, -1 when negative
    position = np.sign(momentum)

    # Strategy returns: position * returns
    strategy_returns = position * returns
    strategy_returns = strategy_returns.dropna()

    # Convert to portfolio values
    portfolio_values = 100000 * (1 + strategy_returns).cumprod()

    return portfolio_values, strategy_returns


# Run backtest on Stock A
portfolio_values, strategy_returns = run_momentum_backtest(prices_df["Stock_A"])

In[21]:

Code

def generate_performance_report(
    portfolio_values, strategy_returns, strategy_name="Strategy"
):
    """
    Generate a comprehensive performance report.
    """
    returns_metrics = calculate_return_metrics(portfolio_values)
    drawdown_metrics = calculate_drawdowns(portfolio_values)
    trade_stats = calculate_trade_statistics(strategy_returns)

    sharpe = calculate_sharpe_ratio(strategy_returns)
    sortino = calculate_sortino_ratio(strategy_returns)
    calmar = (
        returns_metrics["annualized_return"]
        / abs(drawdown_metrics["max_drawdown"])
        if drawdown_metrics["max_drawdown"] != 0
        else np.inf
    )

    report = {
        "Strategy": strategy_name,
        "Total Return": f"{returns_metrics['total_return'] * 100:.2f}%",
        "Annualized Return": f"{returns_metrics['annualized_return'] * 100:.2f}%",
        "Annualized Volatility": f"{returns_metrics['annualized_volatility'] * 100:.2f}%",
        "Sharpe Ratio": f"{sharpe:.2f}",
        "Sortino Ratio": f"{sortino:.2f}",
        "Calmar Ratio": f"{calmar:.2f}",
        "Maximum Drawdown": f"{drawdown_metrics['max_drawdown'] * 100:.2f}%",
        "Win Rate": f"{trade_stats['win_rate'] * 100:.1f}%",
        "Profit Factor": f"{trade_stats['profit_factor']:.2f}",
    }

    return report


report = generate_performance_report(
    portfolio_values, strategy_returns, "Momentum"
)

Out[22]:

Console

==================================================
BACKTEST PERFORMANCE REPORT
==================================================
Strategy...................... Momentum
Total Return.................. -39.21%
Annualized Return............. -23.08%
Annualized Volatility......... 31.18%
Sharpe Ratio.................. -0.74
Sortino Ratio................. -1.17
Calmar Ratio.................. -0.43
Maximum Drawdown.............. -53.74%
Win Rate...................... 48.6%
Profit Factor................. 0.90
==================================================

The performance report quantifies the strategy's risk-return profile through multiple lenses. Metrics like the Sharpe ratio, which measures excess return per unit of risk, and maximum drawdown, which captures the worst peak-to-trough decline, provide a more complete picture than total return alone. By examining these metrics together, you can assess whether the strategy's returns are commensurate with the risks it takes and whether those risks are acceptable for your investment objectives.

Visualizing PerformanceLink Copied

Visual analysis complements numerical metrics by revealing patterns in strategy behavior over time. While summary statistics condense performance into single numbers, charts show how that performance unfolded, making it easier to identify regime changes, unusual periods, and potential concerns.

Out[23]:

Visualization

Line chart of portfolio value over approximately 500 trading days; it rises from $100,000 above $120,000, then falls below the dashed initial-capital reference and finishes near $60,000. — Momentum strategy equity curve showing the portfolio's path over time. After initially rising above $120,000, the portfolio reverses and finishes near $60,000, revealing the strategy's sustained losses in this simulation.

Out[24]:

Visualization

Area chart with coral shading below zero showing drawdown deepening through the backtest and ending near a 50% decline from the portfolio's high-water mark. — Momentum strategy drawdown profile over the backtest period. The chart displays the percentage decline from the running high-water mark, which shows the risk profile and recovery periods.

Out[25]:

Visualization

Histogram of momentum-strategy daily returns centered near zero with a wide negative tail; a dashed coral line marks the mean return of about -0.084%. — Distribution of daily returns for the momentum strategy. The histogram reveals the frequency of return magnitudes, showing the central tendency and the presence of any fat tails.