SARIMA (Seasonal ARIMA) › Formula

The SARIMA model is denoted as SARIMA(p,d,q)(P,D,Q)s, where the first set of parameters (p,d,q) refers to the non-seasonal components and the second set (P,D,Q) refers to the seasonal components, with s being the seasonal period.

Imagine you're forecasting monthly sales for a retail store. You notice two distinct patterns: sales tend to be higher this month if last month was high (short-term memory), and sales in December are consistently higher than in January (seasonal pattern). A simple ARIMA model can handle the first pattern, but it struggles with the second because seasonal effects operate on a different timescale.comparing December to November misses the comparison to December of last year.

This is the fundamental challenge SARIMA addresses: time series often exhibit both short-term dependencies (consecutive observations are related) and long-term seasonal cycles (observations separated by the seasonal period are related). The SARIMA model structure emerges naturally from this dual nature of seasonal time series.

Before we can build a predictive model, we need to understand an important concept: stationarity. A stationary time series has a stable distribution and dependence structure over time. Think of it like a well-behaved process where the rules don't change.if we understand how it behaved in the past, we can predict its future.

Most real-world time series are non-stationary. Sales might trend upward over years, or temperatures might have seasonal cycles. These changing patterns make prediction difficult because the "rules" keep changing. This is why differencing is important: it maps a non-stationary series into a stationary one by removing trends and seasonal effects, creating a stable basis for modeling.

Differencing is the mathematical tool that makes non-stationary series stationary. The key insight is simple: instead of modeling the raw values (which change over time), we model the changes between values (which are more stable).

The SARIMA model uses two types of differencing:

1. Non-seasonal differencing removes trends by computing differences between consecutive observations. If sales increase by roughly the same amount each month, taking differences removes that trend.
2. Seasonal differencing removes seasonal patterns by computing differences between observations separated by the seasonal period. If December sales are typically higher than other months, comparing each month to the same month last year removes that seasonal effect.

Mathematically, we express differencing using the backshift operator B, where $BX_t = X_{t-1}$ means "shift the series back by one period." This notation allows us to write differencing compactly:

$$\nabla^d \nabla_s^D X_t = (1-B)^d (1-B^s)^D X_t$$

The notation has four parts:

• $\nabla^d$: Non-seasonal differencing applied d times
• $\nabla_s^D$: Seasonal differencing applied D times with seasonal period s
• $(1-B)^d$: The mathematical operation that computes d-th order differences
• $(1-B^s)^D$: The operation that computes D-th order seasonal differences

To see why this works, consider first-order non-seasonal differencing (d=1): we compute $X_t - X_{t-1}$. If the original series had a constant upward trend, the differenced series will have a constant mean near zero. For seasonal differencing with s=12, we compute $X_t - X_{t-12}$, comparing each month to the same month last year. This removes the annual seasonal pattern while preserving other relationships.

This visualization demonstrates the differencing transformation process. The top panel shows the original non-stationary series with both an upward trend and seasonal fluctuations. The middle panel shows the result after non-seasonal differencing: the trend has been removed, leaving a series that oscillates around a stable mean, but seasonal patterns remain. The bottom panel shows the final result after seasonal differencing: both trend and seasonality have been removed, creating a stationary series suitable for ARMA modeling. Notice how the mean approaches zero and the variance becomes more stable after each differencing step.

Once we have a stationary series through differencing, we can model how current values depend on past values. This is where autoregressive (AR) terms come in. The intuition is straightforward: in many time series, today's value is related to yesterday's value, and yesterday's value is related to the day before, creating a chain of dependencies.

SARIMA extends this idea to two timescales:

• Non-seasonal AR terms capture short-term dependencies: how this month relates to last month
• Seasonal AR terms capture long-term seasonal dependencies: how this December relates to last December

The non-seasonal autoregressive polynomial is:

$$\phi(B) = 1 - \phi_1 B - \phi_2 B^2 - \cdots - \phi_p B^p$$

Each term $\phi_i B^i$ represents the influence of the value i periods ago. The coefficient $\phi_i$ tells us how strong that influence is. For example, if $\phi_1 = 0.7$, it means that 70% of the current value's deviation from the mean can be explained by the previous value's deviation.

The seasonal autoregressive polynomial works similarly but at seasonal lags:

$$\Phi(B^s) = 1 - \Phi_1 B^s - \Phi_2 B^{2s} - \cdots - \Phi_P B^{Ps}$$

Here, $B^s$ shifts back by s periods (e.g., 12 months), so $\Phi_1 B^{12}$ captures how this month relates to the same month last year. The seasonal AR terms allow the model to "remember" seasonal patterns across years.

Autoregressive terms capture how values depend on past values, but they miss an important source of information: past forecast errors. If we consistently over-predict in certain situations, we should adjust our forecasts accordingly. This is what moving average (MA) terms do.they allow the model to learn from its mistakes.

Moving average terms capture how current values depend on past forecast errors. Like AR terms, SARIMA includes both non-seasonal and seasonal MA components:

• Non-seasonal MA terms: How this month's forecast error relates to last month's forecast error
• Seasonal MA terms: How this month's forecast error relates to the forecast error from the same month last year

The non-seasonal moving average polynomial is:

$$\theta(B) = 1 + \theta_1 B + \theta_2 B^2 + \cdots + \theta_q B^q$$

Note the plus signs here (unlike the minus signs in AR terms). This reflects that MA terms model the error structure, not the value structure. A positive $\theta_1$ means that if we over-predicted last period, we're likely to over-predict this period too (the errors are correlated).

The seasonal moving average polynomial is:

$$\Theta(B^s) = 1 + \Theta_1 B^s + \Theta_2 B^{2s} + \cdots + \Theta_Q B^{Qs}$$

This captures how forecast errors from the same season in previous years influence current errors. For example, if we consistently under-predict December sales, the seasonal MA term helps the model learn this pattern.

Now we can assemble the complete SARIMA model by combining differencing, autoregressive terms, and moving average terms:

$$\phi(B) \Phi(B^s) \nabla^d \nabla_s^D X_t = \theta(B) \Theta(B^s) \epsilon_t$$

This equation systematically combines all the components we've discussed:

• Left side: The differenced series ($\nabla^d \nabla_s^D X_t$) is modeled by autoregressive polynomials that capture dependencies at both short-term ($\phi(B)$) and seasonal ($\Phi(B^s)$) timescales
• Right side: The error structure is modeled by moving average polynomials at both short-term ($\theta(B)$) and seasonal ($\Theta(B^s)$) timescales
• Error term: $\epsilon_t$ represents white noise.random, unpredictable fluctuations with mean 0 and constant variance $\sigma^2$

The multiplicative structure (AR terms multiplied together, MA terms multiplied together) is important. It means that seasonal and non-seasonal effects interact. For example, the interaction between non-seasonal and seasonal AR terms captures how this month's relationship to last month might be different in December (holiday season) versus January (post-holiday).

The SARIMA formulation works because it mirrors the actual structure of seasonal time series. Real-world seasonal data exhibits dependencies at multiple timescales simultaneously:

1. Short-term memory: Recent values influence current values (captured by non-seasonal AR terms)
2. Seasonal memory: Values from the same season in previous years influence current values (captured by seasonal AR terms)
3. Error learning: Past forecast errors inform current predictions (captured by MA terms at both timescales)
4. Stability: Differencing ensures the underlying process is stationary, making all these relationships stable and predictable

The backshift operator B allows us to express complex lag structures compactly. The polynomial notation makes it clear how different lags combine, and the multiplicative structure captures the interaction between seasonal and non-seasonal effects.

For example, in monthly sales data, the model might learn that:

• This month's sales depend on last month's sales (non-seasonal AR)
• December sales depend on previous December sales (seasonal AR)
• These effects interact.the December-to-December relationship might be stronger than the January-to-January relationship
• Forecast errors from holiday seasons tend to persist (seasonal MA), requiring adjustments in future holiday forecasts

This structure allows SARIMA to capture dependencies across multiple timescales in real seasonal time series, making it effective for seasonal forecasting.

The SARIMA model has several important mathematical properties that make it suitable for time series forecasting. The model is stationary when the roots of both the non-seasonal autoregressive polynomial $\phi(B)$ and the seasonal autoregressive polynomial $\Phi(B^s)$ lie outside the unit circle in the complex plane. This condition ensures that the time series doesn't exhibit explosive growth or decay.

The model is also invertible when the roots of both the non-seasonal moving average polynomial $\theta(B)$ and the seasonal moving average polynomial $\Theta(B^s)$ lie outside the unit circle. Invertibility ensures that the model can be expressed in terms of past observations rather than past forecast errors, which is important for forecasting and interpretation.

The forecast function of a SARIMA model has a specific structure: short-term forecasts are dominated by the autoregressive and moving average terms, while long-term forecasts converge to the seasonal pattern. This property makes SARIMA particularly useful for forecasting seasonal time series, as it can capture both immediate dependencies and long-term seasonal cycles.