Case Study - Building a Quantitative Strategy from Scratch › Stage 2: Data Exploration and Pair Selection

With the idea in hand, we move to data. In practice, you would source historical equity prices from a data vendor and scan a universe of stocks for cointegrated pairs. For this case study, we will generate synthetic data that faithfully reproduces the statistical properties of a cointegrated equity pair. This ensures full reproducibility and lets us control the data-generating process, which is valuable for understanding how the strategy behaves under known conditions.

We construct two price series that share a common stochastic trend (a random walk) plus idiosyncratic noise. This is the classic structure that produces cointegration. The idea is that both stocks are driven by the same underlying factor, such as the price of oil for two energy companies, but each also has its own firm-specific noise that causes temporary deviations. A mean-reverting spread component is layered on top, governed by an autoregressive process with parameter $\phi = 0.95$. Because $\phi$ is less than one, the spread is stationary and tends to pull back toward zero, but because it is close to one, the reversion is slow enough to create tradeable multi-week excursions. This mirrors what we observe in real equity pairs, where the spread can remain far from equilibrium for weeks before converging.

The two series clearly trend together but with visible divergences, exactly the pattern we want to exploit.

Before any analysis, we split the data into an in-sample (training) period for model estimation and an out-of-sample (test) period for unbiased evaluation. As we emphasized in the backtesting chapter, failing to separate these periods is the single most common source of overfitting in quantitative research.

The chronological split preserves the time-series structure, so we estimate parameters on historical data and validate performance on subsequent unseen data.

We now test whether our pair is cointegrated using the Engle-Granger two-step procedure, which we covered conceptually in the time series models chapter. The core logic behind this procedure is elegant: if two price series are truly cointegrated, then a properly chosen linear combination of them should eliminate the non-stationary component entirely, leaving behind a stationary residual. The test works by first finding that linear combination through ordinary least squares regression and then checking whether the resulting residuals behave like a stationary process. If they do, we have evidence that the two prices are tied together by a long-run equilibrium, which is precisely the condition we need for mean reversion trading.

The procedure involves two main steps:

1. Estimate the cointegrating regression: Regress Stock B on Stock A to find the hedge ratio $\beta$ and intercept $\alpha$:

   $$P_{B,t} = \beta P_{A,t} + \alpha + \epsilon_t$$

   where:

   • $P_{B,t}, P_{A,t}$: prices of Stock B and Stock A at time $t$
   • $\beta$: hedge ratio (slope coefficient), representing the number of units of A to hold for each unit of B
   • $\alpha$: intercept term, capturing any constant pricing difference
   • $\epsilon_t$: residual (spread), which should be stationary if the pair is cointegrated

   The hedge ratio $\beta$ deserves careful attention because it is the critical component of the entire strategy. Economically, $\beta$ tells us how many dollars of Stock A we need to sell short (or buy) to neutralize the common factor exposure in one dollar of Stock B. If $\beta = 1.2$, for example, then Stock B is 20% more sensitive to the shared stochastic trend than Stock A, so we need 1.2 shares of A to hedge each share of B. Getting $\beta$ right ensures that our spread isolates the mean-reverting component. Getting it wrong means our spread will inherit a residual trend, turning a mean reversion trade into an unintended directional bet. The intercept $\alpha$ captures any constant offset in the price relationship, such as a persistent premium of one stock over the other. This offset shifts the equilibrium level of the spread but does not affect its mean-reverting dynamics.

2. Test the residuals for stationarity: Apply the Augmented Dickey-Fuller (ADF) test to the regression residuals. If we reject the null of a unit root, the spread is stationary and the pair is cointegrated.

   The ADF test is the standard tool for determining whether a time series contains a unit root, that is, whether it behaves like a random walk. The null hypothesis of the ADF test states that the series has a unit root and is therefore non-stationary. If we can reject this null at a conventional significance level (typically 5%), we conclude that the residuals are stationary, which in the context of the Engle-Granger procedure means the original price series are cointegrated. In a cointegration test, the critical values for the ADF test differ from the standard Dickey-Fuller tables because the residuals are generated from a first-stage regression rather than observed directly. The adfuller function from statsmodels handles these details automatically, but understanding the logic helps avoid misinterpretation.