Time-Series Models for Financial Data › ARMA Models: Combining AR and MA

Real financial time series often exhibit both autoregressive and moving average characteristics. Neither pure AR nor pure MA models may adequately capture the dependence structure. The ARMA(p,q) model combines both components:

$$Y_t = c + \phi_1 Y_{t-1} + \cdots + \phi_p Y_{t-p} + \varepsilon_t + \theta_1 \varepsilon_{t-1} + \cdots + \theta_q \varepsilon_{t-q}$$

where:

• $Y_t$: value of the series at time $t$
• $c$: constant term
• $\phi_i$: autoregressive coefficients
• $\theta_j$: moving average coefficients
• $\varepsilon_t$: white noise terms

In lag operator notation:

$$\Phi(L) Y_t = c + \Theta(L) \varepsilon_t$$

where:

• $\Phi(L)$: autoregressive polynomial
• $c$: constant term
• $\Theta(L)$: moving average polynomial

The principle of parsimony motivates ARMA. Sometimes a pure AR model requires many lags to capture the autocorrelation structure, but a mixed ARMA model achieves the same fit with fewer parameters. This parsimony matters for both statistical and practical reasons. Models with fewer parameters are easier to interpret, more stable to estimate, and less prone to overfitting.

Consider an AR(1) process. Its ACF decays geometrically. An MA($\infty$) could replicate this pattern, but that would require infinitely many parameters. Conversely, an MA(1) has an ACF that cuts off after lag 1, but an AR($\infty$) could approximate it. ARMA provides flexibility to match complex autocorrelation patterns economically, using a small number of AR and MA terms to capture structures that would require many terms in either pure representation.

The ARMA(1,1) specification combines the first-order properties of both models, and gives a rich but still tractable framework:

$$Y_t = c + \phi Y_{t-1} + \varepsilon_t + \theta \varepsilon_{t-1}$$

where:

• $Y_t$: value of the series at time $t$
• $Y_{t-1}$: value of the series at time $t-1$
• $c$: constant term
• $\phi$: autoregressive coefficient
• $\theta$: moving average coefficient
• $\varepsilon_t$: white noise error term
• $\varepsilon_{t-1}$: white noise error term at time $t-1$

Its ACF starts at:

$$\rho_1 = \frac{(1+\phi\theta)(\phi+\theta)}{1+\theta^2+2\phi\theta}$$

where:

• $\rho_1$: autocorrelation at lag 1
• $\phi$: AR coefficient
• $\theta$: MA coefficient

and then decays geometrically following the AR structure:

$$\rho_k = \phi \rho_{k-1} \quad \text{for } k \geq 2$$

where:

• $\rho_k$: autocorrelation at lag $k$
• $\phi$: autoregressive coefficient
• $\rho_{k-1}$: autocorrelation at lag $k-1$

The MA component affects only the starting value of the autocorrelation sequence, while the AR component determines the rate of decay. The PACF also decays rather than cutting off, which is characteristic of processes with MA components. This makes ARMA models harder to identify than pure AR or MA processes, since neither the ACF nor PACF shows a clean cutoff pattern.

An ARMA(p,q) model is:

• Stationary if all roots of $\Phi(z) = 0$ lie outside the unit circle
• Invertible if all roots of $\Theta(z) = 0$ lie outside the unit circle

We typically work only with stationary and invertible representations. Stationarity ensures the process has stable statistical properties, while invertibility ensures unique parameter identification and allows us to recover the underlying shocks from observed data.