Forecasting Baselines, Temporal Validation, and Backtesting › Implementation in NumPy and pandas

The horizon table makes the degradation of each baseline visible without a plot. A column that grows with the horizon shows increasing average absolute error on these origins; a flat column shows similar average error across the displayed horizons. Neither pattern alone establishes that a forecasting assumption holds or fails for the underlying process.

This section summarizes the key parameters documented above and the main functions used in the implementation. Below are the parameters that control the validation design and the baselines.

initial_train is the size of the first training window. It must be larger than the seasonal period so that seasonal-naive and MASE calculations are defined. In a rolling window, it also acts as the window length in this implementation. A larger initial window provides more fitting observations but can include stale regimes. On a shared origin schedule, a smaller window drops older observations no later and can increase estimation variance in fitted models; the naive and seasonal-naive rules here use only the latest observation or cycle once enough history is available. The choice is a bias-variance trade-off expressed in time, not in model capacity.

The parameter horizon is the maximum lead time. It determines which targets are evaluable and how the error table is grouped. If the business decision is a four-week order, the horizon should be 28 days, not one day. A model that is excellent one day ahead but poor 28 days ahead is not a solution to the four-week problem. The horizon should be chosen from the decision cadence, not from the desire to make the error curve look smooth.

The parameter step is the spacing between forecast origins. A step of 1 uses every possible origin; adjacent target blocks share $H-1$ dates. A step equal to the horizon creates disjoint target blocks and generally yields fewer evaluations, with no strict reduction required when $H=1$ or only one origin is evaluable. The step should be stated in the report because it affects the dependence between errors and the apparent sample size. A dense grid of origins is not automatically better evidence; the dependence among its errors depends on the horizon, history policy and data-generating process.

The parameter window chooses expanding or rolling history. Expanding history is appropriate when the process is stable and old data remain relevant. Rolling history is appropriate when the process changes. The choice should be made from training evidence, not from the test score. In practice, it is often worth reporting both windows as a sensitivity check, especially when a structural break is suspected.

The parameter season is the seasonal period used by seasonal naive and by the MASE scale. It should reflect the data frequency and the business calendar. Daily data with a weekly pattern usually use $m=7$. Monthly data with an annual pattern usually use $m=12$. If the seasonality is not stable, a fixed $m$ can be misleading. The seasonal period is a modeling assumption, and it should be defended in the same way as the choice of model.

The parameter scale is the denominator used for MASE. In this implementation it is computed once from the initial training window. A per-origin scale is also possible, but it should be documented. A zero or near-zero scale makes MASE unstable, just as zero targets make MAPE unstable. The scale should be treated as part of the reporting contract, not as an invisible implementation detail.

naive_forecast returns a constant array equal to the last observation. It is a simple last-value baseline and the standard random-walk benchmark. Comparing this baseline at matched origins and horizons tests the predictive value of the most recent level under the chosen loss. It is also the first place to look when a complex model appears suspiciously accurate, because an unavailable feature can produce an invalid comparison, even though a large legitimate improvement is also possible.

seasonal_naive_forecast tiles the last seasonal cycle until the requested horizon is filled. It is the standard seasonal random-walk benchmark. Comparing this baseline at matched origins and horizons tests the predictive value of repeating the latest seasonal pattern under the chosen loss. The function uses only the last season observations, so within that forecast block it cannot smooth their noise or extrapolate a new level; that is both its simplicity and its limitation.

drift_forecast estimates a slope from the first and last observations and adds that slope at each future step. It is the standard random-walk-with-drift benchmark. Because it uses endpoints, it is sensitive to outliers at the boundaries; a least-squares trend uses the interior observations and may be less sensitive to a noisy endpoint when a linear trend model is appropriate. The endpoint sensitivity is not a bug in the implementation; it is the defining assumption of the canonical drift baseline.

rolling_origin_backtest is the validation engine. It advances the origin, slices the available history, calls each forecaster, and records the realized error at every horizon. The slicing is the most important line in the function: it must never include the target block in the history. The function raises an error if a forecaster returns the wrong number of horizons, which prevents silent shape mismatches. The shape check catches an invalid forecaster return before its values enter the error table.

seasonal_scale computes the in-sample seasonal-naive scale for MASE. It uses only the training window passed to it. Keeping the scale function separate makes the leakage boundary explicit: the scale is estimated from training history, then used to normalize out-of-sample errors. summarize_errors groups by window and model; the separate horizon table groups by model and horizon. The grouping is where we turn forecast errors into a report that a decision-maker can interpret. Because the long table retains origins and horizons, the summary can always be traced back to the individual forecasts that produced it.