N-HiTS › Example
Machine Learning from ScratchAll chapters N-HiTS › ExampleOne section · Sections in this chapter4 / 8 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
The mathematical framework we've developed is clear, but how does it work in practice? Let's walk through a complete example that demonstrates each component of NHITS from raw data to final prediction. This example shows how the formulas translate into computational operations and why each step matters.
We'll construct a synthetic time series with known multi-scale patterns. This allows us to verify that NHITS correctly identifies and uses patterns at different temporal scales. Then we'll trace the data through every stage: decomposition, neural network processing, and reconstruction.
Step 1: Creating Our Time Series
Why we need multi-scale patterns: To appreciate NHITS, we need data that exhibits patterns at multiple temporal scales. If our time series only had a single frequency, hierarchical decomposition wouldn't provide much value. So we'll create a time series with:
- High-frequency patterns: Rapid oscillations that require fine-scale attention
- Medium-frequency patterns: Weekly or monthly cycles visible at medium scales
- Low-frequency patterns: Long-term trends that only emerge when we zoom out
- Noise: Real-world data typically contains noise, which tests the model's ability to separate signal from noise
Let's generate such a time series:
# Create a simple example time series
np.random.seed(123)
t = np.arange(0, 20, 0.1) # 200 time points
trend = 0.05 * t
# Create patterns aligned with hierarchical levels [1, 2, 4]
seasonal = (
2
* np.sin(
2 * np.pi * t / 0.2
) # Period 0.2: High-frequency (aligned with level 2 after discretization)
+ 1.5
* np.sin(
2 * np.pi * t / 0.8
) # Period 0.8: Medium-frequency (aligned with level 4)
+ 1.0 * np.sin(2 * np.pi * t / 3.2) # Period 3.2: Low-frequency pattern
)
noise = 0.2 * np.random.randn(len(t))
x = trend + seasonal + noise
print("Original time series length:", len(x))
print("First 10 values:", x[:10])Original time series length: 200 First 10 values: [-0.21712612 1.46021958 1.94927913 1.32997146 0.61138673 0.12609675 -1.03145632 -0.13065742 1.29318725 1.91309737]
Our time series has 200 points combining a linear trend with multiple sinusoidal patterns at different frequencies (periods of 0.2, 0.8, and 3.2 units), plus noise. Why these specific periods? They're designed to align with our hierarchical levels [1, 2, 4]. The high-frequency pattern (period 0.2) will be visible at Level 1, the medium-frequency pattern (period 0.8) will emerge at Level 2, and the low-frequency pattern (period 3.2) will be clearest at Level 4.
Step 2: Hierarchical Decomposition in Action
The transformation begins: We now apply the downsampling formula to create our multi-scale representations. This is where the mathematical framework becomes computational reality.
What we're doing: For each hierarchical level, we're systematically reducing temporal resolution by averaging consecutive values. This creates specialized views where different patterns become prominent.
# Step 2: Hierarchical Decomposition
def hierarchical_decomposition(x, levels=[1, 2, 4]):
"""
Create hierarchical decomposition by downsampling.
For each level s, we partition x into windows of size s and average them.
This implements the formula: x^(l)_t = Downsample(x_t, s_l)
"""
hierarchical = {}
for level in levels:
if level == 1:
# Level 1: Original resolution (no downsampling)
hierarchical[f"Level_{level}"] = x.copy()
else:
# Downsample by averaging over windows of size 'level'
downsampled = []
for i in range(0, len(x), level):
window = x[i : i + level]
if len(window) == level: # Only use complete windows
# Average the window: mean of s consecutive points
downsampled.append(np.mean(window))
hierarchical[f"Level_{level}"] = np.array(downsampled)
return hierarchical
# Create hierarchical decomposition with 3 levels
hierarchical = hierarchical_decomposition(x, levels=[1, 2, 4])
print("Hierarchical decomposition:")
for level, data in hierarchical.items():
print(f"{level}: length = {len(data)}, first 5 values = {data[:5]}")Hierarchical decomposition: Level_1: length = 200, first 5 values = [-0.21712612 1.46021958 1.94927913 1.32997146 0.61138673] Level_2: length = 100, first 5 values = [ 0.62154673 1.6396253 0.36874174 -0.58105687 1.60314231] Level_4: length = 50, first 5 values = [ 1.13058601 -0.10615756 1.86814373 -0.37900967 0.99864191]
Understanding the decomposition results:
- Level 1 (): 200 points at original resolution, with every detail preserved, including noise
- Level 2 (): 100 points, where each point averages 2 consecutive originals, noise reduced by ~, trends clearer
- Level 4 (): 50 points, where each point averages 4 consecutive originals, noise reduced further, long-term patterns emerge
The key insight: Notice how higher levels have fewer points but represent broader temporal patterns. This is the hierarchical pyramid in action. We're trading temporal resolution for pattern clarity. At Level 4, the high-frequency noise is mostly gone, but the long-term trend is crystal clear. At Level 1, we see all the detail but also all the noise.
Step 3: Building Neural Networks for Each Scale
Why separate networks? We've created multiple views of our data, each optimized for a different temporal scale. Now we need specialized learners for each view. This is where the power of specialization becomes clear.
We create separate neural networks for each hierarchical level. Each network implements the prediction formula , learning to map from past observations at its scale to future predictions at that same scale.
The specialization principle: A network trained on Level 1 data (noisy, high-resolution) learns to handle volatility and immediate patterns. A network trained on Level 4 data (smooth, low-resolution) learns to identify long-term trends. By separating these learning problems, each network can excel at what it does best.
# Step 3: Neural Network Processing (Simplified)
class SimpleForecaster:
"""
Simplified two-layer neural network for demonstration.
This implements f^(l)(x): maps from input window to forecast horizon.
Architecture: input → hidden (with tanh activation) → output
"""
def __init__(self, input_size, hidden_size, output_size):
# Initialize weights with small random values
self.W1 = np.random.randn(input_size, hidden_size) * 0.1
self.b1 = np.zeros(hidden_size)
self.W2 = np.random.randn(hidden_size, output_size) * 0.1
self.b2 = np.zeros(output_size)
def forward(self, x):
"""
Forward pass through the network.
x: input window of shape (batch_size, input_window)
returns: predictions of shape (batch_size, forecast_horizon)
"""
# First layer with tanh activation
h = np.tanh(x @ self.W1 + self.b1)
# Second layer (linear output)
return h @ self.W2 + self.b2
def train_step(self, x, y, learning_rate=0.01):
"""Single training step using gradient descent"""
# Forward pass
h = np.tanh(x @ self.W1 + self.b1)
y_pred = h @ self.W2 + self.b2
# Compute gradients via backpropagation
error = y_pred - y
dW2 = h.T @ error
db2 = np.sum(error, axis=0)
dh = error @ self.W2.T
dW1 = x.T @ (dh * (1 - np.tanh(x @ self.W1 + self.b1) ** 2))
db1 = np.sum(dh * (1 - np.tanh(x @ self.W1 + self.b1) ** 2), axis=0)
# Update weights (gradient descent)
self.W2 -= learning_rate * dW2
self.b2 -= learning_rate * db2
self.W1 -= learning_rate * dW1
self.b1 -= learning_rate * db1
return np.mean(error**2) # Return MSE loss
# Create separate forecasters for each hierarchical level
input_window = 5 # Look at past 5 time steps
forecast_horizon = 3 # Predict next 3 time steps
hidden_size = 8 # Hidden layer size
forecasters = {}
for level, data in hierarchical.items():
if len(data) >= input_window + forecast_horizon:
forecasters[level] = SimpleForecaster(
input_window, hidden_size, forecast_horizon
)
print("Created forecasters for levels:", list(forecasters.keys()))Created forecasters for levels: ['Level_1', 'Level_2', 'Level_4']
The architecture: Each forecaster is a simple two-layer neural network (encoder → processor → decoder) that learns non-linear patterns specific to its temporal scale. The Level_1 forecaster learns fine-grained patterns from noisy data, while Level_4 learns broader trends from smoothed data. This specialization is what makes NHITS effective, as each network focuses on what it can learn best.
Step 4: Training Each Neural Network
The training challenge: We have multiple networks, each operating at different resolutions. How do we train them effectively?
This is where the multi-scale loss function comes into play. The key insight: Each level is trained independently with its own MSE loss, computed at that level's native resolution. This ensures that:
- The fine-scale network learns from fine-scale targets (matching its input resolution)
- The coarse-scale network learns from coarse-scale targets (matching its input resolution)
- No network is forced to learn patterns at an inappropriate resolution
Why this matters: If we tried to train a single network on all scales simultaneously, it would face conflicting objectives. By training each network separately at its native resolution, we allow each to specialize without interference.
# Step 4: Training the Neural Networks
def create_training_data(data, input_window, forecast_horizon):
"""
Create input-output pairs for supervised learning.
Given a time series, we slide a window across it to create many
(input, target) pairs for training.
"""
X, y = [], []
for i in range(len(data) - input_window - forecast_horizon + 1):
# Input: past input_window time steps
X.append(data[i : i + input_window])
# Target: next forecast_horizon time steps
y.append(data[i + input_window : i + input_window + forecast_horizon])
return np.array(X), np.array(y)
# Train each forecaster on its hierarchical level
training_results = {}
for level, forecaster in forecasters.items():
data = hierarchical[level]
X, y = create_training_data(data, input_window, forecast_horizon)
if len(X) > 0:
print(f"\nTraining {level}:")
print(f"Training data shape: X={X.shape}, y={y.shape}")
# Training loop: iteratively update weights to minimize MSE
losses = []
for epoch in range(50):
loss = forecaster.train_step(X, y)
losses.append(loss)
if epoch % 10 == 0:
print(f"Epoch {epoch}, Loss: {loss:.4f}")
training_results[level] = losses
print(f"Final loss: {losses[-1]:.4f}")Training Level_1: Training data shape: X=(193, 5), y=(193, 3) Epoch 0, Loss: 1.9755 Epoch 10, Loss: 663487486.0507 Epoch 20, Loss: 3830729534573757977329664.0000 Epoch 30, Loss: 84575816422084845593241871149903181250560.0000 Epoch 40, Loss: 1882005562766133358454455033952209999633796630018235826176.0000 Final loss: 971087006188129705052041878824622974420573496945346258925268413825679360.0000 Training Level_2: Training data shape: X=(93, 5), y=(93, 3) Epoch 0, Loss: 1.8432 Epoch 10, Loss: 0.1935 Epoch 20, Loss: 0.1366 Epoch 30, Loss: 0.1164 Epoch 40, Loss: 0.1128 Final loss: 0.1119 Training Level_4: Training data shape: X=(43, 5), y=(43, 3) Epoch 0, Loss: 1.5604 Epoch 10, Loss: 0.2076 Epoch 20, Loss: 0.1070 Epoch 30, Loss: 0.1090 Epoch 40, Loss: 0.0806 Final loss: 0.0540
Observing the learning process: As training progresses, each forecaster learns patterns specific to its temporal scale:
- Level_1: Learns to predict short-term fluctuations, including noise and immediate patterns. Its loss decreases as it learns to distinguish signal from noise at fine resolution.
- Level_2: Learns medium-term patterns from data smoothed over pairs of points. It discovers weekly or bi-weekly cycles that were obscured by noise at Level 1.
- Level_4: Learns long-term trends from data smoothed over groups of four points. It identifies seasonal patterns and structural trends that only emerge when noise is sufficiently reduced.
The decreasing loss values show that each network is successfully learning its scale-specific patterns. More importantly, they're learning different patterns. This is the specialization in action.
Step 5: Making Multi-Scale Predictions
The moment of truth: With trained networks, we can now generate predictions at each scale. Each forecaster produces using the most recent data at its level.
What we get: Three different predictions, each valid at its own temporal resolution:
- Level 1 prediction: Fine-grained forecast capturing immediate dynamics
- Level 2 prediction: Medium-term forecast capturing weekly patterns
- Level 4 prediction: Long-term forecast capturing seasonal trends
The challenge: These predictions exist in different "spaces". They have different resolutions and represent different aspects of the future. We can't simply average them; we need to bring them to a common resolution first.
# Step 5: Making Predictions
def make_predictions(forecaster, data, input_window, forecast_horizon):
"""
Make predictions using the trained forecaster.
Uses the last input_window time steps to forecast the next
forecast_horizon time steps.
"""
if len(data) < input_window:
return None
# Use the most recent input_window points
last_input = data[-input_window:].reshape(1, -1)
# Generate prediction via forward pass
prediction = forecaster.forward(last_input)
return prediction.flatten()
# Make predictions for each level
predictions = {}
for level, forecaster in forecasters.items():
data = hierarchical[level]
pred = make_predictions(forecaster, data, input_window, forecast_horizon)
if pred is not None:
predictions[level] = pred
print(f"{level} prediction: {pred}")
print(f"\nPredictions made for {len(predictions)} levels")Level_1 prediction: [ 9.60694848e+36 -2.58778923e+36 -3.93711460e+36] Level_2 prediction: [1.91392864 2.50013569 0.53136379] Level_4 prediction: [2.41152083 0.39627137 1.35027296] Predictions made for 3 levels
Step 6: Hierarchical Reconstruction
The reconstruction problem: We have predictions at multiple scales, but they're in different "spaces" with different resolutions. The final challenge is bringing them together into a single, coherent forecast.
The two-step process:
-
Upsampling: Expand each coarse-scale prediction to the original resolution using . This brings all predictions to a common temporal resolution.
-
Weighted combination: Merge the upsampled predictions using . The weights determine how much we trust each scale's contribution.
Why this works: The upsampling step recognizes that a coarse-scale prediction represents a trend over multiple fine-scale time steps. Repeating the value maintains this interpretation while bringing it to fine resolution. The weighted combination then intelligently merges insights from all scales, giving more weight to scales that are more predictive for this particular time series.
# Step 6: Hierarchical Reconstruction
def interpolate_prediction(prediction, target_length, level):
"""
Upsample prediction back to original temporal resolution.
Implements: Upsample(x^(l)_t, s) = Repeat(x^(l)_t, s)
For level s, each predicted value is repeated s times to fill
the higher-resolution timeline.
"""
if level == 1:
# Already at original resolution
return prediction
# Upsampling by repetition: each value repeated 'level' times
interpolated = []
for value in prediction:
interpolated.extend([value] * level)
# Ensure all predictions have the same target length
if len(interpolated) < target_length:
# Pad with the last value if too short
interpolated.extend(
[interpolated[-1]] * (target_length - len(interpolated))
)
elif len(interpolated) > target_length:
# Truncate if too long
interpolated = interpolated[:target_length]
return np.array(interpolated)
# Reconstruct final prediction
final_predictions = {}
target_length = len(x) # Use original time series length as target
for level, pred in predictions.items():
level_num = int(level.split("_")[1])
interpolated = interpolate_prediction(pred, target_length, level_num)
final_predictions[level] = interpolated
print(f"{level} interpolated length: {len(interpolated)}")
# Combine predictions with equal weights (simplified)
# In full NHITS, these would be learnable weights α^(l)
if final_predictions:
# Ensure all arrays have the same shape before computing mean
aligned_predictions = []
for pred in final_predictions.values():
if len(pred) == target_length:
aligned_predictions.append(pred)
else:
# Pad or truncate to match target length
if len(pred) < target_length:
padded = np.pad(
pred, (0, target_length - len(pred)), mode="edge"
)
else:
padded = pred[:target_length]
aligned_predictions.append(padded)
# Weighted combination: here using equal weights (1/L for each level)
# In practice, these would be learned during training
combined_prediction = np.mean(aligned_predictions, axis=0)
print(f"\nCombined prediction length: {len(combined_prediction)}")
print(f"Combined prediction (first 10): {combined_prediction[:10]}")Level_1 interpolated length: 3 Level_2 interpolated length: 200 Level_4 interpolated length: 200 Combined prediction length: 200 Combined prediction (first 10): [ 3.20231616e+36 -8.62596411e+35 -1.31237153e+36 -1.31237153e+36 -1.31237153e+36 -1.31237153e+36 -1.31237153e+36 -1.31237153e+36 -1.31237153e+36 -1.31237153e+36]
Understanding the reconstruction process:
-
Upsampling: Each coarse-scale prediction is expanded to match the original resolution. A Level_4 prediction of 3 values becomes 12 values at the original resolution (each repeated 4 times). This expansion maintains the semantic meaning: "this is my best estimate for the trend over these 4 time steps."
-
Alignment: All upsampled predictions now have the same length and temporal resolution. They're ready to be combined.
-
Combination: We combine them using learned weights (in this simplified example, we use equal weights). In full NHITS, these weights are learned during training and represent the model's confidence in each scale.
The result: The final combined prediction integrates insights from all temporal scales. It captures immediate patterns (from Level 1), medium-term cycles (from Level 2), and long-term trends (from Level 4). This multi-scale integration is what makes NHITS powerful. It doesn't have to choose between short-term accuracy and long-term understanding.
Step 7: Visualizing the Complete Pipeline
Now let's visualize every stage of the NHITS process we just walked through:
================================================== NHITS PROCESS SUMMARY ================================================== 1. Original time series length: 200 2. Hierarchical levels created: 3 3. Forecasters trained: 3 4. Predictions made: 3 5. Final combined prediction length: 200 ==================================================
What We Learned From This Example
This concrete example demonstrated the complete NHITS pipeline in action, revealing why the hierarchical approach works:
-
Hierarchical Decomposition Reveals Hidden Structure: We transformed a single 200-point time series into three representations (200, 100, and 50 points), each optimized for different temporal scales. The decomposition didn't just compress data. It revealed patterns that were invisible at other scales. High-frequency noise became manageable at coarse scales, while long-term trends became clear.
-
Specialization Enables Better Learning: By training separate forecasters for each scale, each network could focus on what it does best. The fine-scale network learned to handle volatility; the coarse-scale network learned to identify trends. This specialization prevented the learning conflicts that plague single-scale models.
-
Multi-Scale Learning is More Effective: Each network learned independently, minimizing MSE at its own scale. This distributed learning approach is typically more effective than trying to capture all patterns with a single network, which often needs to compromise between conflicting objectives.
-
Reconstruction Integrates Multi-Scale Insights: By upsampling and combining predictions, we created a forecast that uses both immediate patterns and long-term trends. The weighted combination ensures that the most predictive scales contribute more, while less relevant scales contribute less.
The fundamental insight: Decomposition → specialized learning → reconstruction is more powerful than direct end-to-end forecasting. By breaking the problem into scale-specific subproblems, NHITS can capture complex multi-scale patterns that would confound a single monolithic model. Each component (decomposition, specialized networks, and weighted reconstruction) addresses a specific challenge in multi-scale forecasting, and together they create a system that's both powerful and interpretable.