N-BEATS › Formula

Imagine you're analyzing monthly sales data for a retail store. The numbers tell a story: steady growth over the years, but also predictable spikes every December. The challenge is that these two patterns, the long-term growth and the seasonal cycles, are woven together in a single stream of numbers. How do we separate them? More importantly, how do we predict future values when both patterns might evolve in complex ways?

This is the fundamental problem N-BEATS addresses: forecasting time series that contain multiple overlapping patterns. The solution lies in a mathematical idea called basis expansion, the same principle that allows us to decompose a musical chord into individual notes, or break down a complex signal into simpler frequencies.

Before diving into formulas, let's build intuition. When you look at a time series, what you're seeing is the sum of different underlying forces:

1. Long-term trends: The overall direction. Is the business growing, declining, or stable?
2. Seasonal patterns: Repeating cycles. Do sales peak in certain months, days, or seasons?
3. Random noise: Unpredictable fluctuations that we can't model

The key insight is that if we can mathematically separate these components, we can:

• Understand what's driving our forecasts (interpretability)
• Extrapolate each component independently (flexibility)
• Learn the right combination from data (adaptability)

This separation is exactly what N-BEATS does. The model doesn't just predict numbers. It decomposes the time series into interpretable building blocks, then combines them to produce forecasts.

To make this intuition precise, N-BEATS expresses any time series value $y_t$ as the sum of trend components, seasonal components, and a small error term. This is the fundamental decomposition structure:

$$y_t = \sum_{k=1}^{K} g_k^t(x_t) + \sum_{k=1}^{K} g_k^s(x_t) + \epsilon_t$$

where:

• $y_t$: observed value at time $t$
• $g_k^t(x_t)$: contribution of the $k$-th trend basis function at time $t$ (captures long-term patterns)
• $g_k^s(x_t)$: contribution of the $k$-th seasonal basis function at time $t$ (captures periodic patterns)
• $K$: number of basis functions for each component type
• $\epsilon_t$: error term (residual noise we can't predict)
• $x_t$: input features at time $t$ (typically a window of historical values)

Why this structure matters: This equation tells us that every observation is built from two types of interpretable components plus noise. The trend components capture long-term direction and growth patterns, while the seasonal components capture repeating cycles. By separating these, we can understand both what the forecast is and why it takes that value.

Note: When making forecasts, this decomposition is applied at each forecast horizon $h$. The basis functions $g_k^t$ and $g_k^s$ are evaluated using coefficients learned by the neural network, which we'll see in detail below.

How do we choose the right basis functions $g_k^t$ and $g_k^s$? We need functions that:

1. Can represent the types of patterns we see in real time series
2. Allow us to learn how much each function contributes to the forecast
3. Maintain interpretability so we can understand what's driving predictions

This is where N-BEATS' mathematical design addresses each component systematically.

When we examine a time series, trends reveal the underlying direction of change. But trends come in many shapes:

• A startup's revenue might grow steadily month after month (linear growth)
• A company gaining market share might start slow and accelerate as it scales (quadratic growth)
• A product lifecycle might show growth, then plateau, then decline (cubic with inflection points)

The challenge: We don't know in advance which pattern our time series will exhibit. We need a flexible way to represent all these possibilities.

Why polynomials? Polynomials are mathematical functions that can represent different types of growth patterns. The key insight is that we don't have to choose just one. We can include multiple polynomial degrees and let the model learn how much each one contributes. This is the foundation of N-BEATS' trend modeling.

For a forecast horizon $H$ (the number of future periods we want to predict), we compute the trend component at each forecast step $h$. The trend component at horizon $h$ is:

$$\text{trend}_h = \sum_{k=1}^{K} \alpha_{k,h} \cdot \left(\frac{h}{H}\right)^k$$

where:

• $\alpha_{k,h}$: learned coefficient for the $k$-th trend basis function at forecast horizon $h$
• $H$: total forecast horizon (e.g., 6 months, 12 months)
• $h$: specific forecast step within the horizon ($h \in \{1, 2, \ldots, H\}$)
• $k$: polynomial degree (typically $k \in \{1, 2, 3\}$)
• $K$: number of polynomial basis functions used

Understanding the normalization: The term $\left(\frac{h}{H}\right)^k$ is carefully designed. By normalizing the horizon $h$ by the total horizon $H$, we create a scale that ranges from 0 to 1. This normalization is important because it makes the basis functions scale-invariant. The same model architecture works whether we're forecasting 6 months or 12 months ahead. Without this normalization, we'd need different models for different forecast horizons.

How the exponent $k$ shapes the pattern:

The exponent $k$ determines the mathematical shape of the trend pattern:

1. $k=1$ (linear): The function $\frac{h}{H}$ creates a straight line. This works well for steady, consistent growth or decline. If the learned coefficient $\alpha_{1,h}$ is positive, the trend increases linearly; if negative, it decreases linearly.

2. $k=2$ (quadratic): The function $\left(\frac{h}{H}\right)^2$ creates a curve that accelerates or decelerates. This captures scenarios where growth starts slow and speeds up (positive acceleration), or starts fast and slows down (negative acceleration).

3. $k=3$ (cubic): The function $\left(\frac{h}{H}\right)^3$ creates more complex curves with inflection points, places where the curve changes direction. This allows the model to capture trends that grow before plateauing and declining, or other complex directional changes.

The power of learned coefficients: The coefficients $\alpha_{k,h}$ make this approach flexible. Rather than forcing a single polynomial degree, N-BEATS learns coefficients for multiple degrees simultaneously. The neural network examines the historical data and automatically determines: "This time series needs 70% linear trend, 20% quadratic trend, and 10% cubic trend."

This automatic selection makes N-BEATS both flexible (it adapts to different patterns) and interpretable (we can see which trend components are driving the forecast). The model fits a curve by decomposing the trend into understandable components.

While trends capture long-term direction, seasonal patterns capture repeating cycles. Think of retail sales: they might peak every December (yearly seasonality), or show weekly patterns with higher sales on weekends. These patterns repeat at regular intervals, creating waves in the time series.

The challenge: Seasonal patterns can have different shapes. Some are smooth sine waves, others have multiple peaks and valleys within a single cycle. How do we represent all these possibilities?

Why trigonometric functions? Trigonometric functions (sine and cosine) are the natural mathematical language for periodic patterns. This isn't arbitrary. It's grounded in Fourier analysis, which tells us that any periodic signal can be decomposed into sine and cosine waves. This theoretical foundation gives us confidence that trigonometric basis functions can represent the seasonal patterns we see in real time series.

The seasonal component at forecast horizon $h$ is:

$$\text{seasonal}_h = \sum_{k=1}^{K} \left[ \beta_{k,h} \cdot \sin\left(\frac{2\pi k h}{P}\right) + \gamma_{k,h} \cdot \cos\left(\frac{2\pi k h}{P}\right) \right]$$

where:

• $\beta_{k,h}, \gamma_{k,h}$: learned coefficients for sine and cosine components of the $k$-th harmonic at forecast horizon $h$
• $P$: seasonal period (e.g., 12 for monthly data with yearly seasonality)
• $k$: harmonic frequency (typically $k \in \{1, 2, 3, 4\}$)
• $h$: specific forecast step within the horizon ($h \in \{1, 2, \ldots, H\}$)
• $K$: number of harmonic basis functions used

Why both sine and cosine? This is an important question. A pure sine wave starts at zero and oscillates, but real seasonal patterns can start at any point in the cycle. For example, retail sales might peak in December (month 12), but a sine wave that starts at zero would peak at a different time.

The solution lies in phase shifts. Mathematically, any sinusoidal pattern can be written as $A \sin(\omega t + \phi)$, where $\phi$ is the phase shift. Using trigonometric identities, this can be expanded to $A_1 \sin(\omega t) + A_2 \cos(\omega t)$. By combining sine and cosine with learned coefficients, we can represent any phase-shifted pattern. The coefficients $\beta_{k,h}$ and $\gamma_{k,h}$ automatically learn the right combination to match the phase of the actual seasonal pattern.

Understanding harmonics with parameter $k$: The parameter $k$ controls the frequency of oscillations. When $k=1$, we capture the fundamental seasonal pattern, the main yearly cycle, for example. But real seasonal patterns are often more complex.

Consider a retail time series: it might have a yearly cycle (higher in December) but also sub-cycles within the year (higher in spring and fall). Higher values of $k$ capture these harmonics:

• $k=1$: The fundamental pattern (one complete cycle per period)
• $k=2$: Patterns that repeat twice per cycle (e.g., spring and fall peaks)
• $k=3$: Patterns that repeat three times per cycle
• $k=4$: Even more complex patterns with multiple peaks and valleys

By including multiple harmonics, N-BEATS can represent complex seasonal patterns that have multiple peaks and valleys within a single cycle. The model learns which harmonics are needed for each specific time series.

The role of period $P$: The period $P$ determines the length of one complete cycle. For monthly data with yearly seasonality, $P=12$ means the pattern repeats every 12 months. The model learns the coefficients $\beta_{k,h}$ and $\gamma_{k,h}$ to determine both:

1. Amplitude: How strong the seasonal effect is (the height of the peaks)
2. Phase: When the peaks and valleys occur (the timing within the cycle)

This automatic learning allows the model to adapt to different seasonal patterns without manual tuning. The same architecture can handle yearly seasonality ($P=12$), quarterly patterns ($P=4$), or weekly patterns ($P=7$), simply by adjusting the period parameter.

N-BEATS learns the coefficients $\alpha_{k,h}$, $\beta_{k,h}$, and $\gamma_{k,h}$ automatically from data with a neural network instead of relying on fixed values chosen by hand. This automatic learning makes N-BEATS adaptive and general-purpose.

The network examines historical patterns in your specific time series and automatically determines: "For this particular dataset, I need 60% linear trend, 30% quadratic trend, and 10% cubic trend, with a seasonal pattern that peaks in month 6 and has a secondary peak in month 12." Feed it different data, and it learns a different decomposition.

Why a neural network? The relationship between historical patterns and future coefficients is complex and non-linear. Consider this: if recent values are increasing rapidly, we might need strong linear and quadratic trend components. But if the increase is accelerating, we might need more quadratic than linear. If there's a seasonal peak approaching, we need specific sine/cosine coefficients to capture that phase.

A simple linear model couldn't capture how acceleration and seasonal phase affect the coefficients. The neural network learns these patterns through training, discovering relationships that would be difficult to specify manually.

The learning mechanism: The network takes a window of historical values as input and outputs all the coefficients needed for forecasting:

$$\boldsymbol{\theta} = \text{NN}(\mathbf{x}_t)$$

where:

• $\boldsymbol{\theta} = [\alpha_{1,1}, \alpha_{1,2}, \ldots, \beta_{1,1}, \beta_{1,2}, \ldots, \gamma_{1,1}, \gamma_{1,2}, \ldots]^T$: vector of all coefficients
• $\mathbf{x}_t$: input features (typically a window of historical values)
• $\text{NN}(\cdot)$: neural network function

How the network processes information: The neural network consists of multiple fully connected layers that transform the input historical data into the coefficient vector. Think of each layer as learning increasingly abstract patterns:

1. First layer: Detects simple features like "recent values are increasing" or "there's a peak in the last 12 months"
2. Second layer: Combines these simple features into more complex patterns like "accelerating growth with seasonal peaks" or "steady decline with quarterly cycles"
3. Final layer: Translates these abstract patterns into the specific coefficients needed for forecasting

The mathematical structure: Each layer applies a linear transformation followed by a ReLU activation:

$$\mathbf{h}_1 = \text{ReLU}(\mathbf{W}_1 \mathbf{x}_t + \mathbf{b}_1)$$ $$\mathbf{h}_2 = \text{ReLU}(\mathbf{W}_2 \mathbf{h}_1 + \mathbf{b}_2)$$ $$\boldsymbol{\theta} = \mathbf{W}_3 \mathbf{h}_2 + \mathbf{b}_3$$

where:

• $\mathbf{x}_t$: input vector (window of historical time series values)
• $\mathbf{h}_1, \mathbf{h}_2$: hidden layer activations (intermediate representations)
• $\mathbf{W}_i$: weight matrix for layer $i$ (learned parameters that transform the input)
• $\mathbf{b}_i$: bias vector for layer $i$ (learned parameters that shift the transformation)
• $\boldsymbol{\theta}$: output vector containing all learned coefficients $[\alpha_{1,1}, \alpha_{1,2}, \ldots, \beta_{1,1}, \beta_{1,2}, \ldots, \gamma_{1,1}, \gamma_{1,2}, \ldots]^T$
• $\text{ReLU}(z) = \max(0, z)$: Rectified Linear Unit activation function (applied element-wise)

Why ReLU activation? The ReLU (Rectified Linear Unit) activation function introduces non-linearity by setting negative values to zero. This is important because it allows the network to learn complex, non-linear relationships between historical patterns and future coefficients. Without this non-linearity, the network would only be able to learn linear relationships, severely limiting its ability to capture the complex patterns in time series data.

Interpreting coefficients: The final layer outputs the coefficient vector $\boldsymbol{\theta}$ without activation, as coefficients can be positive or negative. This design choice is important:

• A positive coefficient means that basis function contributes positively to the forecast (e.g., increasing trend, seasonal peak)
• A negative coefficient means it contributes negatively (e.g., decreasing trend, seasonal trough)

This allows the model to represent both upward and downward patterns, and to cancel out components when needed to match the actual time series behavior.

We've seen how N-BEATS represents time series using basis functions and how the neural network learns coefficients. But here's an important question: How does the network know if it's learning the right coefficients?

the loss function addresses this. It measures how well the model's predictions match the actual values. Without this feedback mechanism, the network would have no way to know whether its learned coefficients are producing good forecasts.

The optimization objective: The model learns the optimal coefficients by minimizing prediction error on training data. The loss function measures how far the model's predictions are from the actual values using mean squared error:

$$\mathcal{L} = \frac{1}{N} \sum_{i=1}^{N} (y_i - \hat{y}_i)^2$$

where:

• $N$: total number of predictions (across all training samples and forecast horizons)
• $\hat{y}_i$: predicted value for the $i$-th prediction
• $y_i$: actual value for the $i$-th prediction
• $\mathcal{L}$: mean squared error loss

In practice, for multi-step forecasting with $T$ training samples and forecast horizon $H$, we have $N = T \cdot H$ total predictions, and the loss averages the squared errors across all of them.

Why mean squared error? This is the mean squared error (MSE), which penalizes large prediction errors more than small ones. Squaring the errors means that being off by 10 units is penalized 100 times more than being off by 1 unit. This encourages the model to avoid large mistakes, which is particularly important in forecasting where a few large errors can be more costly than many small ones.

How predictions are computed: For each forecast horizon $h$, the predicted value $\hat{y}_{t+h}$ is computed by combining the trend and seasonal components:

$$\hat{y}_{t+h} = \text{trend}_h + \text{seasonal}_h = \sum_{k=1}^{K} \alpha_{k,h} \cdot \left(\frac{h}{H}\right)^k + \sum_{k=1}^{K} \left[ \beta_{k,h} \cdot \sin\left(\frac{2\pi k h}{P}\right) + \gamma_{k,h} \cdot \cos\left(\frac{2\pi k h}{P}\right) \right]$$

where:

• $\hat{y}_{t+h}$: predicted value at time $t+h$ (forecast $h$ steps ahead)
• $\text{trend}_h$: trend component at forecast horizon $h$
• $\text{seasonal}_h$: seasonal component at forecast horizon $h$
• All other symbols are as defined previously

This is where everything comes together: the basis functions are evaluated at each forecast step $h$ using the learned coefficients, then combined additively to produce the forecast.

The learning process: During training, the neural network adjusts its weights through backpropagation to minimize this loss. The process works iteratively:

1. The network makes a prediction using current coefficients
2. It compares the prediction to the actual value
3. It calculates the loss (how far off it was)
4. It adjusts its weights to reduce the loss
5. Repeat over many iterations

Over many iterations, the network learns to extract coefficients that produce accurate forecasts.

The deeper insight: The loss function measures accuracy and also guides the decomposition. When the model minimizes the loss, it's simultaneously learning:

1. How to separate trend from seasonality: The model needs to figure out which patterns are long-term trends versus repeating cycles
2. Which polynomial degrees are needed: Does this time series need linear, quadratic, or cubic trend components?
3. Which harmonics are needed: What seasonal frequencies are present in the data?
4. The appropriate coefficients: How much should each basis function contribute?

All of this happens automatically through the optimization process. The model doesn't need us to tell it what patterns to look for. It discovers them by minimizing prediction error. This automatic discovery makes N-BEATS both effective (it adapts to different patterns) and general-purpose (the same architecture works across diverse forecasting problems).

We've now built up the complete mathematical framework, from decomposition to basis functions to neural network learning. Let's step back and understand why this particular combination of ideas is so effective for time series forecasting. Each design choice serves a specific purpose, and together they create a system that is both theoretically sound and practically powerful.

1. Theoretical foundation: The basis expansion approach is grounded in fundamental mathematical theory. Two key theorems guarantee that our approach can represent the patterns we need:

• Weierstrass approximation theorem: Polynomials can approximate any smooth function to arbitrary precision. This means our polynomial trend basis functions can represent any smooth growth pattern we encounter.
• Fourier's theorem: Trigonometric functions can represent any periodic function. This means our trigonometric seasonal basis functions can capture any repeating pattern.

By combining both, we get a framework that's theoretically sound: we know mathematically that our basis functions are capable of representing the types of patterns we see in real time series. This theoretical guarantee gives us confidence that the model architecture isn't fundamentally limited in what it can learn.

2. Additive decomposition for interpretability: The choice to add trend and seasonal components together (rather than multiply them) is important for interpretability. When components are additive, they don't interfere with each other. This means:

• We can examine the trend component and understand long-term growth patterns without seasonal fluctuations obscuring our view
• We can examine the seasonal component and understand periodic patterns without the trend distorting the seasonal shape
• We can see exactly how much each component contributes to the final forecast

This separation makes N-BEATS interpretable: we can see what's driving our forecasts. The additive structure also avoids imposing strong assumptions about how trend and seasonality interact.

1. Automatic adaptation through learning: The neural network learning mechanism maps this mathematical framework from a theoretical possibility into a practical tool. The network learns coefficients and the right decomposition for each specific time series.

When you feed it historical sales data, it might learn: "This time series needs a strong linear trend with a moderate seasonal pattern that peaks in December." Feed it different data, and it learns a different decomposition. This automatic adaptation makes N-BEATS general-purpose: the same architecture works across diverse forecasting problems without requiring domain-specific knowledge or extensive hyperparameter tuning.

4. Scale invariance: The normalization in the trend basis functions ($\frac{h}{H}$) and the period-based scaling in seasonal functions ($\frac{2\pi k h}{P}$) make the model scale-invariant. This means:

• The same model architecture can work with different forecast horizons (6 months, 12 months, etc.) without modification
• The same architecture can handle different seasonal periods (yearly, quarterly, weekly) by adjusting the period parameter
• The normalization automatically adjusts the basis functions to the appropriate scale

This design choice means we don't need separate models for different forecasting scenarios. One architecture adapts to them all.

The synthesis: Together, these mathematical properties create an interpretable forecasting system that remains flexible enough to adapt to different time series. The theoretical foundation ensures we're not missing important patterns, the additive decomposition ensures we can understand what we're learning, the neural network ensures we can adapt to any specific time series automatically, and the scale invariance ensures the same architecture works across different forecasting scenarios.

This is the power of N-BEATS: it combines mathematical rigor with practical flexibility, creating a model that is both theoretically sound and practically useful.