N-HiTS › Formula

Imagine you're forecasting retail sales. A sudden spike on Tuesday might catch your attention, but is it meaningful? To answer that, you need context: Is this part of a weekly shopping pattern? Does it align with monthly seasonal trends? Or is it just noise? The challenge is that time series contain patterns operating at fundamentally different speeds: daily fluctuations overlay weekly cycles, which nest within monthly seasonality, all embedded in yearly trends.

Traditional forecasting methods often force a choice: focus on fine-grained details and miss the big picture, or smooth everything and lose important short-term signals. But what if we didn't have to choose? What if we could simultaneously understand both the forest and the trees?

NHITS solves this with a simple insight: we should process time series at multiple resolutions, just as we naturally think about time. When you examine a stock chart, you might zoom in to see minute-by-minute volatility, then zoom out to understand the monthly trend. NHITS does exactly this mathematically, creating multiple "views" of the same time series, each optimized for a different temporal scale, then combining these perspectives into a unified forecast.

The Core Problem: A single time series contains information at multiple temporal scales, but a single neural network struggles to learn all scales simultaneously. High-frequency noise can overwhelm long-term trends, and trying to capture everything at once leads to mediocre performance across all scales.

The Solution: Create a hierarchy, a pyramid of views where each level focuses on a specific temporal resolution. This changes how we approach the problem. Instead of asking one network to do everything, we ask multiple specialized networks to each handle what they do best.

Why We Need Multiple Scales: The IntuitionLink Copied

Consider what happens when you look at a time series through different "lenses":

• Fine-grained view: Every point matters
  You see daily fluctuations and immediate patterns, along with noise. This detail is important for capturing short-term dynamics, but it's also where noise lives.

• Medium-grained view: Trends emerge
  When you average pairs of consecutive points, noise cancels out while trends become clearer. Weekly patterns start to emerge.

• Coarse-grained view: The big picture
  Averaging over larger windows reveals long-term trends and seasonal patterns that were invisible in the noisy original data.

The central insight is that each scale reveals different information. Fine scales capture immediate dynamics; coarse scales reveal structural patterns. We need both, but they're best learned separately.

The Mathematical Framework: DownsamplingLink Copied

To create these multiple views, we systematically reduce temporal resolution through downsampling. For a time series $x_t$ with length $T$, we create $L$ hierarchical levels, where level $l$ has downsampling factor $s_l$.

The downsampled representation at level $l$ is:

$$x^{(l)}_t = \text{Downsample}(x_t, s_l)$$

where:

• $x_t$: original time series with $T$ time steps
• $l$: hierarchical level index ($l = 0, 1, 2, \ldots, L-1$)
• $s_l$: downsampling factor for level $l$ (e.g., $s_0 = 1$, $s_1 = 2$, $s_2 = 4$)
• $x^{(l)}_t$: downsampled time series at level $l$
• $t$: time index in the downsampled series

What does downsampling do? The most natural approach averages consecutive values, creating a smoother, lower-resolution version:

$$\text{Downsample}(x_t, s) = \frac{1}{s} \sum_{i=0}^{s-1} x_{t \cdot s + i}$$

where:

• $s$: downsampling factor (window size for averaging)
• $x_t$: original time series values
• $t$: index in the downsampled series (each $t$ corresponds to a window in the original)
• $i$: offset within the averaging window ($i = 0, 1, \ldots, s-1$)
• $x_{t \cdot s + i}$: original time series value at position $t \cdot s + i$

This formula tells us to:

1. Partition the time series into consecutive windows of size $s$
2. Average all values within each window
3. Produce a new time series with $1/s$ the original temporal resolution

Why averaging? Averaging serves two important purposes:

• Noise reduction: High-frequency noise tends to cancel out when averaged, while true patterns persist
• Scale bridging: It creates a natural mathematical connection between different temporal resolutions

The Hierarchy in Action:

• Level 0 ($s_0 = 1$): The original time series $x^{(0)}_t = x_t$, with every detail preserved
• Level 1 ($s_1 = 2$): Each point averages two consecutive originals, reducing noise and making trends clearer
• Level 2 ($s_2 = 4$): Each point averages four originals, allowing long-term patterns to emerge
• Level 3 ($s_3 = 8$): Even coarser, with seasonal and cyclical patterns dominating

This creates a pyramid: the base contains all details, while higher levels reveal increasingly broad patterns. Each level is a different "zoom level" on the same underlying data.

We now have multiple views of our time series, each revealing different temporal patterns. This raises an important question: Should we use one network to learn all scales, or separate networks for each?

The answer lies in understanding what each network needs to learn. Consider the difference between predicting tomorrow's stock price versus next year's trend:

• Fine-scale prediction requires handling rapid fluctuations and immediate market reactions amid noise
• Coarse-scale prediction requires understanding long-term economic cycles, seasonal patterns, and structural trends

These are fundamentally different learning problems. A single network trying to do both faces a conflict: it needs to simultaneously be sensitive to noise (for fine scales) and ignore noise (for coarse scales). This is why specialization matters.

The Prediction FormulaLink Copied

At each hierarchical level $l$, we employ a dedicated neural network $f^{(l)}$ that learns patterns specific to that temporal scale. The prediction at level $l$ is:

$$\hat{x}^{(l)}_t = f^{(l)}(x^{(l)}_{t-k:t-1})$$

where:

• $l$: hierarchical level index ($l = 0, 1, 2, \ldots, L-1$)
• $f^{(l)}$: neural network specialized for patterns at scale $s_l$ (a function mapping input windows to predictions)
• $x^{(l)}_{t-k:t-1}$: input window containing the past $k$ time steps at level $l$'s resolution
  • $k$: input window size (number of past time steps used for prediction)
  • $t-k:t-1$: time indices from $t-k$ to $t-1$ (exclusive of $t$)
• $\hat{x}^{(l)}_t$: predicted future value(s) at level $l$'s native resolution
  • The hat ($\hat{}$) indicates this is a predicted value, not an observed value

Why this works: Each network operates in a space optimized for its scale. The fine-scale network $f^{(0)}$ sees high-resolution data with all its noise and volatility, exactly what it needs to learn short-term patterns. The coarse-scale network $f^{(2)}$ sees smoothed, downsampled data where long-term trends are clear, exactly what it needs to learn seasonal patterns.

The Architecture: Encoder-Processor-DecoderLink Copied

Each neural network $f^{(l)}$ is typically implemented as a multi-layer perceptron (MLP) with three components:

1. Encoder layers: Transform the input window $x^{(l)}_{t-k:t-1}$ into a rich internal representation that captures relevant patterns at scale $s_l$

2. Processing layers: Learn complex non-linear relationships specific to the temporal scale. These layers discover how past values at this resolution relate to future values

3. Decoder layers: Generate predictions $\hat{x}^{(l)}_t$ based on the learned representation

This architecture allows each network to specialize: fine-scale networks learn to predict immediate fluctuations, while coarse-scale networks learn to predict broader movements and seasonal patterns. The separation prevents the learning conflicts that plague single-scale approaches.

We've made predictions at multiple scales, but they exist in different "spaces": Level 0 predictions are at the original resolution, Level 1 predictions are at half resolution, and Level 2 predictions are at quarter resolution. How do we combine these into a single, coherent forecast?

This is the reconstruction problem: we need to bring all predictions to a common resolution, then intelligently merge them. The solution has two parts: upsampling (bringing everything to the same resolution) and weighted combination (merging the multi-scale insights).

Step 1: Upsampling: Bringing Coarse Predictions to Fine ResolutionLink Copied

A coarse-scale prediction like $\hat{x}^{(2)}_t$ represents a trend over multiple fine-scale time steps. To combine it with fine-scale predictions, we need to expand it back to the original resolution through upsampling:

$$\text{Upsample}(\hat{x}^{(l)}_t, s_l)$$

where:

• $\hat{x}^{(l)}_t$: prediction at hierarchical level $l$ (at level $l$'s native resolution)
• $s_l$: downsampling factor for level $l$ (determines how many times each value is repeated)
• Returns: upsampled prediction at the original temporal resolution

The simplest and most interpretable approach is repetition: each coarse-scale value is repeated $s_l$ times to fill the higher-resolution timeline:

$$\text{Upsample}(\hat{x}^{(l)}_t, s) = \text{Repeat}(\hat{x}^{(l)}_t, s)$$

where:

• $\hat{x}^{(l)}_t$: coarse-scale prediction at level $l$
• $s$: upsampling factor (number of repetitions per value, equal to the downsampling factor $s_l$)
• $\text{Repeat}(\hat{x}^{(l)}_t, s)$: each value in $\hat{x}^{(l)}_t$ is repeated $s$ times consecutively

Why repetition? This has a clear interpretation: the coarse-scale network is saying "my best estimate for the trend over these $s$ time steps is this value." Repeating it maintains that interpretation while bringing it to the fine resolution needed for combination.

More advanced interpolation methods (linear, cubic) can also be used, but repetition often works well and preserves the semantic meaning of coarse-scale predictions.

Step 2: Weighted Combination: Merging Multi-Scale InsightsLink Copied

With all predictions now at the same resolution, we combine them using learnable weights that determine each scale's contribution:

$$\hat{x}_t = \sum_{l=0}^{L-1} \alpha^{(l)} \cdot \text{Upsample}(\hat{x}^{(l)}_t, s_l)$$

where:

• $\hat{x}_t$: final combined prediction at time $t$ (at original temporal resolution)
• $l$: hierarchical level index, ranging from $0$ to $L-1$
• $L$: total number of hierarchical levels
• $\alpha^{(l)}$: learnable combination weight for level $l$ (learned during training, typically normalized so $\sum_{l=0}^{L-1} \alpha^{(l)} = 1$)
• $\hat{x}^{(l)}_t$: prediction from level $l$ (at level $l$'s native resolution)
• $s_l$: downsampling factor for level $l$ (used for upsampling)
• $\text{Upsample}(\hat{x}^{(l)}_t, s_l)$: upsampled prediction from level $l$ (at original resolution)

The weights $\alpha^{(l)}$ are learned during training and represent the model's confidence in each scale's predictive power. This design is powerful for three reasons:

1. Adaptive weighting: Different time series require different scale balances. For volatile financial data, short-term patterns might dominate (large $\alpha^{(0)}$). For seasonal sales data, long-term trends might be more important (large $\alpha^{(L-1)}$). The model adapts automatically.

2. Automatic scale selection: Rather than manually deciding which scales matter, the model learns this from data. If a particular scale isn't helpful, its weight will be small.

3. Interpretability: By examining learned weights, we understand which temporal scales drive the forecast. This provides insight into the data's structure and the model's reasoning.

The intuition: Think of this as a weighted vote. Each hierarchical level casts a "vote" (its prediction) about the future, and the weights determine how much we trust each vote. The final forecast is a consensus that respects both immediate dynamics and long-term trends.

Training a hierarchical system presents a unique challenge: we have multiple networks, each operating at different resolutions, and we need them to work together coherently. How do we design a loss function that ensures all levels learn meaningful patterns?

The answer is a multi-scale loss that supervises each level at its native resolution:

$$\mathcal{L} = \sum_{l=0}^{L-1} w^{(l)} \cdot \text{MSE}(x^{(l)}_t, \hat{x}^{(l)}_t)$$

where:

• $\mathcal{L}$: total multi-scale loss (scalar value to be minimized during training)
• $l$: hierarchical level index, ranging from $0$ to $L-1$
• $L$: total number of hierarchical levels
• $w^{(l)}$: loss weight for level $l$ (can be set to balance contributions across scales, or set to $1/L$ for equal weighting)
• $\text{MSE}(x^{(l)}_t, \hat{x}^{(l)}_t)$: mean squared error between observed and predicted values at level $l$
  • $x^{(l)}_t$: observed (ground truth) values at level $l$'s resolution (downsampled from original $x_t$)
  • $\hat{x}^{(l)}_t$: predicted values at level $l$'s resolution (output of network $f^{(l)}$)
  • Computed as: $\text{MSE}(x^{(l)}_t, \hat{x}^{(l)}_t) = \frac{1}{N} \sum_{t=0}^{N-1} (x^{(l)}_t - \hat{x}^{(l)}_t)^2$ where $N$ is the number of time steps at level $l$ (indices range from $t=0$ to $t=N-1$)

Why this design works:

1. Direct supervision at native resolution: Each network $f^{(l)}$ receives a learning signal at its own temporal scale. The fine-scale network learns from fine-scale targets; the coarse-scale network learns from coarse-scale targets. This prevents the mismatch that occurs when trying to learn coarse patterns from fine-scale data.

2. Balanced learning across scales: The weights $w^{(l)}$ can be adjusted to ensure all scales contribute to the prediction. Without this, fine-scale patterns might dominate simply because there are more data points at fine resolutions. The weights ensure each scale gets appropriate attention.

3. Coherent multi-scale predictions: By training all levels jointly, the model learns to make predictions that are consistent across scales. A coarse-scale prediction should align with the smoothed version of fine-scale predictions. This coherence emerges naturally from joint training.

The training process: During each training step, we compute predictions at all hierarchical levels, measure errors at each level's native resolution, and update all networks simultaneously. This ensures that each network specializes in its scale while contributing to a unified forecasting system.

The hierarchical structure of NHITS provides mathematical advantages that explain why it often outperforms single-scale approaches. Let's examine four key properties:

1. Multi-Scale Pattern Capture: The decomposition creates a mathematical guarantee: both high-frequency patterns (fine scales) and low-frequency patterns (coarse scales) have dedicated neural networks optimized for their specific characteristics. This solves a fundamental problem: in single-scale models, high-frequency noise often dominates the loss function, obscuring long-term trends. By separating scales, each network can focus on patterns appropriate to its resolution without interference from other scales.

2. Smooth Interpolation: The upsampling mechanism, combined with learned combination weights, creates mathematically smooth transitions between temporal scales. This prevents the jagged artifacts that arise from naive scale mixing (like simply concatenating features from different scales). The final prediction is temporally coherent because the combination respects the natural structure of multi-scale data.

3. Adaptive Scale Importance: The learnable weights $\alpha^{(l)}$ provide automatic scale selection. During training, the model discovers which temporal scales are most predictive for the specific forecasting task. This adaptation eliminates the need for manual scale selection and ensures the model uses the most relevant scales for each time series.

4. Hierarchical Regularization: By processing the same information at multiple resolutions and requiring consistency across scales, the model implicitly regularizes its predictions. Predictions should make sense at both fine and coarse resolutions, which helps prevent overfitting to noise at any single scale. This is a form of multi-scale regularization that's naturally built into the architecture.

How these properties work together: The decomposition enables specialization (Property 1), the upsampling enables smooth combination (Property 2), the learnable weights enable adaptation (Property 3), and the multi-scale structure provides regularization (Property 4). Together, they create a framework that's both powerful and principled, exactly what's needed for complex, multi-scale time series forecasting.