Case Study - Building a Quantitative Strategy from Scratch › Stage 5: Performance Evaluation
Quantitative FinanceAll chapters Case Study - Building a Quantitative Strategy from Scratch › Stage 5: Performance EvaluationOne section · Sections in this chapter6 / 12 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
A strategy's value cannot be judged by total PnL alone. As we discussed in the portfolio performance measurement chapter, we need risk-adjusted metrics that account for the variability and worst-case behavior of returns.
Computing Performance Metrics
We compute the standard suite of metrics covered in Part IV. Each metric answers a different question about the strategy's quality, and together they provide a multidimensional portrait of performance that no single number could capture.
The annualized return tells us how much the strategy earns per year on average, while the annualized volatility measures the variability of those earnings. The Sharpe ratio, which we encountered in the portfolio theory chapters, combines these two by dividing excess return by volatility, giving us a single number that captures return per unit of risk. A Sharpe ratio above 1.0 is generally considered attractive for a live strategy, while values above 2.0 suggest either exceptional skill or possible overfitting.
The maximum drawdown answers a more visceral question: what is the worst peak-to-trough decline in equity that the strategy has experienced? This metric is arguably more important to you than the Sharpe ratio because it determines whether a strategy is psychologically and financially survivable. A strategy with a Sharpe ratio of 2.0 but a maximum drawdown of 40% may be theoretically attractive but practically undeployable, because you may be forced to shut it down before the recovery arrives. The Calmar ratio bridges these perspectives by dividing annualized return by maximum drawdown, measuring how well the strategy compensates you for enduring its worst period.
The profit factor compares gross profits to gross losses across all trades, and gives a simple measure of edge. A profit factor of 1.5, for example, means the strategy earns $1.50 for every $1.00 it loses. The win rate and average win-to-loss ratio decompose this edge further, revealing whether the strategy wins by being right often (high win rate) or by winning big when it is right (high win/loss ratio).
def compute_performance_metrics(
pnl_series, capital=None, notional_series=None, annual_factor=252
):
"""Compute comprehensive performance metrics for a trading strategy."""
# Compute daily returns based on capital base or average notional
if capital:
daily_returns = pnl_series / capital
elif notional_series is not None:
daily_returns = pnl_series / notional_series.mean()
else:
raise ValueError("Must provide either capital or notional_series")
# Annualized return and volatility
ann_return = daily_returns.mean() * annual_factor
ann_vol = daily_returns.std() * np.sqrt(annual_factor)
# Sharpe ratio (assuming zero risk-free rate for simplicity)
sharpe = ann_return / ann_vol if ann_vol > 0 else 0
# Maximum drawdown
cum_returns = (1 + daily_returns).cumprod()
rolling_max = cum_returns.cummax()
drawdown = (cum_returns - rolling_max) / rolling_max
max_drawdown = drawdown.min()
# Calmar ratio
calmar = ann_return / abs(max_drawdown) if max_drawdown != 0 else 0
# Win rate
active_pnl = pnl_series[pnl_series != 0]
win_rate = (active_pnl > 0).mean() if len(active_pnl) > 0 else 0
# Profit factor
gross_profit = active_pnl[active_pnl > 0].sum()
gross_loss = abs(active_pnl[active_pnl < 0].sum())
profit_factor = gross_profit / gross_loss if gross_loss > 0 else np.inf
# Average win / average loss
avg_win = active_pnl[active_pnl > 0].mean() if (active_pnl > 0).any() else 0
avg_loss = (
abs(active_pnl[active_pnl < 0].mean()) if (active_pnl < 0).any() else 0
)
# Longest drawdown period
is_dd = drawdown < 0
dd_groups = (~is_dd).cumsum()
dd_periods = is_dd.groupby(dd_groups).sum()
longest_dd = dd_periods.max() if len(dd_periods) > 0 else 0
return {
"Annualized Return": f"{ann_return:.2%}",
"Annualized Volatility": f"{ann_vol:.2%}",
"Sharpe Ratio": f"{sharpe:.2f}",
"Maximum Drawdown": f"{max_drawdown:.2%}",
"Calmar Ratio": f"{calmar:.2f}",
"Win Rate": f"{win_rate:.1%}",
"Profit Factor": f"{profit_factor:.2f}",
"Avg Win / Avg Loss": f"{avg_win / avg_loss:.2f}"
if avg_loss > 0
else "N/A",
"Longest Drawdown (days)": f"{int(longest_dd)}",
"Total Trading Days": f"{len(pnl_series)}",
}# In-sample metrics
metrics_train = compute_performance_metrics(
results_train["net_pnl"], capital=1_000_000
)
# Out-of-sample metrics
metrics_test = compute_performance_metrics(
results_test["net_pnl"], capital=1_000_000
)============================================================ Metric In-Sample Out-of-Sample ============================================================ Annualized Return 11.43% 5.57% Annualized Volatility 11.54% 10.85% Sharpe Ratio 0.99 0.51 Maximum Drawdown -11.06% -10.17% Calmar Ratio 1.03 0.55 Win Rate 50.8% 48.8% Profit Factor 1.30 1.14 Avg Win / Avg Loss 1.26 1.19 Longest Drawdown (days) 245 152 Total Trading Days 1000 500 ============================================================
Drawdown Analysis
Maximum drawdown is arguably the most important risk metric for you. It answers the question: "What is the worst cumulative loss you would have experienced?" As we discussed in the chapter on performance measurement, even a high-Sharpe strategy can be behaviorally difficult to trade through deep drawdowns.
Trade Analysis
The distribution of individual trade outcomes shows whether performance depends on a small number of outlier wins. We define a "trade" as a complete round-trip from entry to exit.
def extract_trades(results_df):
"""Extract individual round-trip trades from backtest results."""
positions = results_df["position"].values
pnl = results_df["net_pnl"].values
dates = results_df.index
trades = []
in_trade = False
trade_pnl = 0
trade_start = None
trade_direction = 0
for t in range(len(positions)):
if not in_trade and positions[t] != 0:
# Trade entry
in_trade = True
trade_pnl = pnl[t]
trade_start = dates[t]
trade_direction = positions[t]
elif in_trade and positions[t] == trade_direction:
# Trade continues
trade_pnl += pnl[t]
elif in_trade and positions[t] != trade_direction:
# Trade exit (or reversal)
trades.append(
{
"entry_date": trade_start,
"exit_date": dates[t],
"direction": "Long" if trade_direction > 0 else "Short",
"pnl": trade_pnl,
"duration": (dates[t] - trade_start).days,
}
)
trade_pnl = 0
if positions[t] != 0:
# Immediate reversal
in_trade = True
trade_start = dates[t]
trade_direction = positions[t]
trade_pnl = pnl[t]
else:
in_trade = False
# Close any open trade
if in_trade:
trades.append(
{
"entry_date": trade_start,
"exit_date": dates[-1],
"direction": "Long" if trade_direction > 0 else "Short",
"pnl": trade_pnl,
"duration": (dates[-1] - trade_start).days,
}
)
return pd.DataFrame(trades)trades_test = extract_trades(results_test)=== Out-of-Sample Trade Summary === Total round-trip trades: 12 Winning trades: 7 (58%) Losing trades: 5 (42%) Avg trade PnL: $-5,107.98 Avg winning trade: $16,830.79 Avg losing trade: $-35,822.26 Avg trade duration: 24.1 days Max trade duration: 67 days
The trade summary reports more winning than losing round trips, but the sample is small and includes several large losses. Treat these estimates as a diagnostic for the simulated holdout period, not as precise evidence of a stable win rate or payoff distribution. The multi-week holding periods characterize the strategy as medium-term rather than high-frequency.
The histogram makes the asymmetry visible: positive outcomes are more common, yet several large negative trades pull the arithmetic mean below zero. With only a small set of round trips, neither the shape nor the mean should be extrapolated as a long-run distribution.