Time-Series Models for Financial Data › ARIMA Models: Handling Non-Stationarity

When the original series is non-stationary, such as asset prices or GDP, we can often achieve stationarity through differencing. Rather than modeling the original series directly, we model the changes. The ARIMA(p,d,q) model applies ARMA(p,q) to a $d$-times differenced series, combining differencing to achieve stationarity with ARMA to model the resulting stationary dynamics.

Define the difference operator $\nabla = 1 - L$, which computes the change from one period to the next:

$$\nabla Y_t = Y_t - Y_{t-1}$$

where:

• $\nabla$: difference operator
• $Y_t$: value at time $t$
• $Y_{t-1}$: value at time $t-1$

For financial prices, first differencing yields returns. If the original price series has a unit root (is I(1)), then returns will be stationary. Second differencing applies the operator twice:

$$\nabla^2 Y_t = \nabla(\nabla Y_t) = Y_t - 2Y_{t-1} + Y_{t-2}$$

where:

• $\nabla^2 Y_t$: second difference of the series
• $Y_t$: value at time $t$
• $Y_{t-1}$: value at time $t-1$
• $Y_{t-2}$: value at time $t-2$

Second differencing is occasionally needed for economic series with both a unit root and a time-varying trend, though over-differencing can introduce artificial patterns and should be avoided.

An ARIMA(p,d,q) model states that $\nabla^d Y_t$ follows an ARMA(p,q). The integrated component handles the transformation to stationarity, while the ARMA component models the resulting stationary dynamics. We can express this using the difference operator or the lag operator explicitly:

$$\begin{aligned} \Phi(L) \nabla^d Y_t &= c + \Theta(L) \varepsilon_t \\ \Phi(L)(1-L)^d Y_t &= c + \Theta(L) \varepsilon_t && \text{(substituting } \nabla = 1-L \text{)} \end{aligned}$$

where:

• $Y_t$: value of the series at time $t$
• $\nabla$: difference operator
• $\Phi(L)$: AR polynomial
• $(1-L)^d$: differencing operator of order $d$
• $c$: constant term
• $\Theta(L)$: MA polynomial
• $\varepsilon_t$: white noise error term

Common specifications include:

• ARIMA(0,1,0): Random walk $Y_t = Y_{t-1} + \varepsilon_t$
• ARIMA(0,1,1): Random walk plus MA(1) on returns
• ARIMA(1,1,0): AR(1) on returns (first differences)
• ARIMA(1,1,1): ARMA(1,1) on returns

When $d \geq 1$, the constant $c$ induces a deterministic trend in the original series. For ARIMA(0,1,0) with constant:

$$Y_t - Y_{t-1} = c + \varepsilon_t$$

where:

• $Y_t$: value of the series at time $t$
• $Y_{t-1}$: value at time $t-1$
• $c$: constant term (drift)
• $\varepsilon_t$: white noise error term

This says that each period, the series increases by $c$ on average plus a random shock. Summing the increments from $t=1$ to $T$ reveals the cumulative effect of the drift and shocks:

$$\begin{aligned} \sum_{t=1}^T (Y_t - Y_{t-1}) &= \sum_{t=1}^T (c + \varepsilon_t) \\ Y_T - Y_0 &= cT + \sum_{t=1}^T \varepsilon_t && \text{(telescoping sum)} \\ Y_T &= Y_0 + cT + \sum_{t=1}^T \varepsilon_t && \text{(rearrange)} \end{aligned}$$

where:

• $Y_T$: value at time $T$
• $Y_0$: initial value
• $cT$: deterministic trend component
• $\sum \varepsilon_t$: stochastic trend component

The term $cT$ is a linear trend that grows steadily with time. The accumulated shocks $\sum \varepsilon_t$ form a stochastic trend that wanders randomly. This is the random walk with drift model, commonly used for log stock prices. It captures the empirical observation that stock prices tend to rise over time (positive drift) while exhibiting substantial random variation around this trend.