Forecasting Baselines, Temporal Validation, and Backtesting

A demand planner faces a concrete problem: every Monday morning, using sales that are final only through Sunday night, they must forecast the next four weeks. The forecast is not a claim about a randomly selected future row. It is a claim about specific future dates, made from a specific information set, and it will be scored at several horizons.

Forecasting baselines are the simplest forecasts that respect time. The naive forecast repeats the last observed value. The seasonal-naive forecast repeats the last observed seasonal cycle. The canonical drift forecast continues the line through the first and last observations; fitting a separate trend is a different stated model. These methods are deliberately simple. Their job is to establish what a forecast must beat before we add exponential smoothing, ARIMA, gradient boosting, or neural networks. They also expose whether our validation is honest: a large improvement merits an information-availability audit, but it is not itself evidence of leakage.

This chapter treats forecasting as a sequence of forecast origins. At origin (o), we know the past up to (o) and must produce forecasts for (o+1,\ldots,o+H). The targets are future values (y_{o+1},\ldots,y_{o+H}), and none of them may enter the features, scaling, imputation, or model selection for that origin. We then move the origin forward and repeat the exercise. This is backtesting. It is the time-series analogue of cross-validation, but with an explicit arrow of time and an explicit forecast horizon.

We should keep this problem distinct from time-to-event analysis. In survival analysis, the outcome is a time to an event. Right-censoring records that the event has not occurred by the last observed follow-up time. In forecasting, the outcome is a value at a future date, such as demand next Tuesday. A missing future demand value is not a censored survival time; it is simply unavailable because the future has not happened. We can borrow careful thinking about information availability, but we should not blur the two targets.

These three baseline formulas are transparent and avoid iterative model fitting. The seasonal period and history policy still require choices made from training evidence. Their accuracy and cost depend on the series and implementation. If a seasonal-naive forecast is already accurate enough for inventory planning, a complex model must justify its additional maintenance cost, monitoring burden, and failure modes. Baselines therefore provide a minimum acceptable standard rather than a decorative comparison.

They also provide a reference that can prompt a leakage audit. Suppose a gradient-boosted model reports an implausibly low error on next-day demand. Comparing it with a naive forecast at the same origins can prompt an information-availability audit. A rolling average of realized responses that includes an unavailable target-day outcome, a promotion flag unavailable at the forecast origin, or a scaler fitted using values released after that origin violates that origin's information boundary. These violations can distort the comparison even when they produce no accuracy gain. The gap can prompt an audit; a large legitimate improvement is possible, and the gap alone does not establish leakage. Baselines also help us audit the information available to each forecast.

Finally, baselines make horizon-specific reporting natural. A naive forecast may be reasonable one day ahead and poor four weeks ahead. Seasonal naive can be strong at multiples of the seasonal period and weaker between them. Drift can capture a persistent trend but can also extrapolate a temporary run of high values. Reporting error by horizon, with the number of origins and the forecast window stated explicitly, turns a single aggregate score into an interpretable forecast audit.

The main disadvantage is structural. A naive forecast assumes that the future conditional mean is close to the last observation. A seasonal-naive forecast assumes that the seasonal pattern repeats exactly. A drift forecast assumes that a linear trend estimated from two endpoints continues unchanged. These assumptions can fail under promotions, price changes, supply constraints, holidays, regime changes, and product life cycles. The three rules as defined here do not use covariates such as price, weather, or marketing spend. Covariate-based simple benchmarks are possible, but they are different forecasting rules.

Baselines also provide no built-in uncertainty model. We can construct empirical intervals from backtest errors, but those intervals inherit the dependence structure of the validation design. If we evaluate daily origins with a 28-day horizon, consecutive forecast errors overlap heavily. The number of error rows can look large while the number of independent forecast experiments is much smaller. Treating correlated errors as independent can misstate uncertainty. The direction and size depend on their covariance structure.

The validation itself has costs and pitfalls. Rolling-origin backtesting repeats forecast production and scoring. Model refitting follows the deployment policy and can add substantial cost. Expanding windows reuse all history, which may be inappropriate after a structural break. Rolling windows discard old data, which reduces sample size and can increase variance. A fixed seasonal period may be wrong after calendar changes. Percentage-error measures can explode near zero, and a single zero-demand day can make MAPE undefined. These are not arguments against baselines; they are reasons to report the validation design and the error scale explicitly.

Forecasting notation earns its keep by keeping two times apart: the time when the forecast is made and the time it is meant to describe. If we blur those times, the model can accidentally learn from the very future it is supposed to predict. The formulas below build a precise contract around that separation. We first define the forecast origin and horizon, then the information set, then the three baseline rules, then the backtest that turns a single forecast into repeated evidence, and finally the error measures that summarize that evidence.

Let the observed series be (y_1,\ldots,y_T), where (T) is the last time index with a realized target. We use one-based time subscripts in the equations. In code, Python indices are zero-based, so the observation at time (t) is stored at position (t-1). A forecast made at origin (o) for target time (o+h) is written

A one-step-ahead forecast has $h=1$, a four-week-ahead forecast has $h=28$ if the data are daily and the horizon is measured in days, and a forecast for the same weekday next week has $h=7$. The notation $o+h\mid o$ is deliberate: it keeps the target time and the information time separate. Every later formula inherits that separation.

The forecast origin is the last time point whose target value is allowed to influence the forecast. It is a wall, not a suggestion. If we stand at the end of day $o$, the information set for a univariate baseline is

This set contains the past and nothing else. It does not contain (y_{o+1}), even if that value happens to be sitting in the same data frame. More generally, if we have covariates (x_s), the information set is

A forecast rule is a function

which maps the available time-stamped records into a real-valued forecast. The braces in $\mathcal I_o$ are shorthand for those records, retaining their times and repeated values. For a baseline, the function is simple. For a machine-learning model, the function may be a fitted regression or tree ensemble, but the domain is still the information set. The target (y_{o+h}) is not in (\mathcal I_o). This is the formal version of an everyday rule: we cannot use next week's sales to predict next week's sales.

The distinction is the time when the feature value became available. A value observed or released only after origin (o) cannot be used there, even if a retrospective table now contains it. For example, a feature “sales the day before the target” can be available for a one-day-ahead forecast made today once today’s sales have been released. For a seven-day-ahead forecast made at the same origin, it would require an unavailable future realization. A feature “sales on the target day” is never available at the origin. Calendar features such as day of week, month, and planned holidays can be known in advance because they are deterministic or pre-scheduled. Planned promotions can also be known in advance, while their future realized effects are unavailable until observed.

It helps to sort features by when their values become known:

• Known before the origin: calendar structure, scheduled holidays, contractual prices, and approved promotion plans.
• Observed by the origin: the latest observed sales, inventory, and traffic.
• Known only after the origin: future realized sales and footfall, weather revisions released after the origin, and aggregates of realized responses that include an unavailable target-day outcome.

Only the first two groups belong in (\mathcal I_o). Values in the third group stay outside that forecast's inputs; after release, they may be used for scoring or retrospective evaluation.

For a direct (h)-step model, a training row is often written as

where (x_{s,h}) must be constructed only from information available at origin (s). At fitting origin (o), the target (y_{s+h}) must also have been released already; with no reporting delay this requires (s+h\le o). Revised or delayed data require the historical vintage actually available then. The subscript (h) reminds us that the feature set can differ by horizon. For a recursive model, we forecast one step, append the forecast to the history, and forecast again. Recursive forecasts can propagate errors because later forecasts use earlier predictions as inputs. The closed-form baseline implementations here produce every horizon from the same observed history without feeding predictions back into the functions.

A forecast is evaluable only if the target has been observed. For a fixed maximum horizon (H), an origin (o) is valid when

The target block is then (y_{o+1},\ldots,y_{o+H}). This inequality is the basic guard against accidentally scoring a forecast whose target has not happened. It also explains why backtests shorten as (H) grows: the last usable origins must leave room for the entire target block.

The three baselines encode different assumptions about how the future relates to the past. Naive and seasonal naive are conditional means under the corresponding zero-mean innovation random walks. The endpoint drift rule plugs an estimated drift into a random-walk-with-drift forecast. These interpretations explain the mechanisms without promising accuracy for an arbitrary series.

The naive forecast uses the last observed value for every horizon:

Here $T$ is the forecast origin, $y_T$ is its latest observed value, and $h$ is the lead time. Under a random walk

with integrable innovations, iterated conditional expectation gives $\mathbb E[Y_{T+h}\mid\mathcal I_T]=y_T$. A trend or changing seasonal pattern can make this model unsuitable; the baseline remains a useful matched-horizon comparison.

The seasonal-naive forecast repeats the last observed seasonal cycle. Let $m$ be the seasonal period: $m=7$ for daily data with a weekly pattern, $m=12$ for monthly data with an annual pattern, and $m=4$ for quarterly data.

The floor term selects the correct cycle offset. Define

where (\lfloor\cdot\rfloor) is the floor function, which rounds down to the nearest integer. The seasonal-naive forecast is

The index $T+h-m(k+1)$ is the most recent observation that has the same position in the seasonal cycle as the target. For $h=1$, $k=0$, so the forecast is $y_{T+1-m}$, the same season one cycle earlier. For $h=m$, $k=0$, so the forecast is $y_T$, the last observed value, which is the same season as the target. For $h=m+1$, $k=1$, so the forecast is again $y_{T+1-m}$. This is the most recent observed value at the target's seasonal position, now two cycles before the target.

where:

• $\hat y^{\text{snaive}}_{T+h\mid T}$: the seasonal-naive forecast for target time $T+h$
• $m$: the seasonal period (e.g., 7 for a weekly cycle in daily data)
• $h$: the lead time in periods
• $k=\lfloor (h-1)/m \rfloor$: the index of the block of $m$ future horizons containing $h$, starting at block zero

With integrable innovations satisfying $\mathbb E[\varepsilon_t\mid\mathcal I_{t-1}]=0$, the method is the conditional mean forecast under a seasonal random walk

Its weakness is rigidity. Within a forecast block it repeats the last observed seasonal cycle exactly, without smoothing its noise or extrapolating a new level or trend. If the seasonal pattern is strong and stable, that rigidity is a strength. If a promotion or a new competitor changes the pattern, the same rigidity becomes a liability.

The drift baseline extends a straight line from the first observation to the last observation. It requires at least two observations, $T\ge2$. The estimated drift per period is

where:

• $\hat c$: the estimated drift per period
• $y_T$: the last observation in the history
• $y_1$: the first observation in the history
• $T$: the number of observations (so $T-1$ is the number of intervals between them)

The (h)-step forecast is

For $h=1$, we add one drift increment to the last observation. For $h=2$, we add two increments, and so on.

where:

• $\hat y^{\text{drift}}_{T+h\mid T}$: the drift forecast for target time $T+h$
• $y_T$: the last observed value at the origin
• $\hat c$: the estimated drift per period
• $h$: the lead time in periods

This plugs an estimated drift into the conditional-mean formula for a random walk with drift,

where:

• $y_t$: the observed value at time $t$
• $y_{t-1}$: the previous period's observation
• $c$: the constant drift per period
• $\varepsilon_t$: the innovation at time $t$, with conditional mean zero

The estimate $\hat c=(y_T-y_1)/(T-1)$ is simple but endpoint-sensitive. A single unusual first or last observation can change the slope substantially. An alternative is to estimate a slope by ordinary least squares on the training history, which uses interior observations and can reduce sensitivity to a noisy endpoint under an appropriate trend model, but then the method is closer to a linear trend model than to the canonical drift baseline. Either version should be stated explicitly when reporting results.

It is useful to remember the assumptions as a compact contrast:

• Naive: the future level is the last level.
• Seasonal naive: the future calendar position repeats the most recent occurrence of that position.
• Drift: the future continues a straight line whose slope is fixed by the observed endpoints.

Each baseline is a hypothesis about the data-generating process. Backtesting asks whether that hypothesis is good enough for the decision at hand.

A fixed-origin holdout forecast gives one forecast origin and one target block. That is often too little evidence to judge a model, and it is easy for one origin to flatter or punish a method by accident. Rolling-origin backtesting repeats the forecast exercise at a sequence of origins

with (o_k+H\le T) for every origin. At origin (o_k), the model sees information through (o_k), forecasts (H) future values, and is scored on (y_{o_k+1},\ldots,y_{o_k+H}). Then the origin advances and the process repeats. The backtest is a sequence of small, honest forecasting problems, not one large hindsight problem.

In an expanding-window backtest, the training history grows:

This is appropriate when older data remain relevant and when we want to mimic a system that accumulates history over time. The first origin has the smallest training set; later origins have more data. More history can improve estimates under a stable seasonal model, but expanding windows can also mix incompatible regimes.

In a rolling-window backtest, only the most recent (w) observations are used:

This is appropriate when the data-generating process changes and old observations would bias the forecast. The window length (w) is a hyperparameter, and it must be chosen without looking at the test targets. A rolling window throws away information, so it can increase variance, but it can reduce bias after a structural break.

The spacing between origins is also a choice. Using every possible origin maximizes the number of evaluations. Consecutive target blocks share exactly $H-1$ dates, so there is no target overlap when $H=1$. Spacing origins by $H$ periods makes the target blocks disjoint and generally reduces the number of evaluations; the count can be unchanged when $H=1$ or only one origin is evaluable. The number of forecast errors alone does not establish independence among them. If we evaluate 100 daily origins with a 28-day horizon, the target blocks overlap, and the errors can be dependent. We should report the number of origins, the horizon, the window type, and the step, not only the number of error rows.

Once the forecasts are frozen and the targets are revealed, we need a way to turn many forecast errors into evidence. The raw object is the forecast error. For an origin (o) and horizon (h), the forecast error is

The sign convention matters: a positive error means the actual value was above the forecast. If we reverse the sign, bias has the opposite interpretation.

where:

• $e_{o,h}$: the forecast error at origin $o$ and horizon $h$
• $y_{o+h}$: the realized target value at time $o+h$
• $\hat y_{o+h\mid o}$: the forecast made at origin $o$ for target time $o+h$

The sign convention gives a positive error when the actual value exceeds the forecast. The absolute error is $|e_{o,h}|$, and the squared error is $e_{o,h}^2$. Every metric below is a different way of summarizing these elementary errors.

Let (\mathcal O_h) be the set of origins for which horizon (h) is evaluable. The mean absolute error at horizon $h$ is

where:

• $\mathcal O_h$: the set of origins for which horizon $h$ is evaluable
• $|\mathcal O_h|$: the number of such origins
• $e_{o,h}$: the forecast error at origin $o$ and horizon $h$

The root mean squared error is

where:

• $\mathcal O_h$: the set of origins evaluable at horizon $h$
• $e_{o,h}$: the forecast error at origin $o$ and horizon $h$

MAE is in the original units and is easy to explain. RMSE penalizes large errors more heavily because the errors are squared before averaging. Neither is universally better; they answer different loss questions. The mean error, or bias, is

where:

• $\mathcal O_h$: the set of origins evaluable at horizon $h$
• $e_{o,h}$: the signed forecast error at origin $o$ and horizon $h$
• $\operatorname{Bias}_h$: positive values indicate under-forecasting, negative values over-forecasting

A model can have low MAE and still be systematically biased. Reporting bias alongside MAE and RMSE prevents that mistake.

Percentage errors are tempting because they are scale-free, but they have a zero problem. The mean absolute percentage error is often written as

where:

• $e_{o,h}$: the forecast error at origin $o$ and horizon $h$
• $y_{o+h}$: the realized target value at time $o+h$
• $\mathcal O_h$: the set of origins evaluable at horizon $h$

If (y_{o+h}=0), the ratio is undefined. If (y_{o+h}) is near zero, the percentage error can explode even when the absolute error is small. We should not silently drop those rows without reporting the exclusion, because dropping zeros changes the target population. A weighted absolute percentage error,

avoids division by each individual target. It is undefined when the sum of absolute targets is zero and can become unstable when that sum is near zero. Here $\mathcal O_h$ is the set of origins evaluable at horizon $h$, $e_{o,h}$ is the forecast error, and $y_{o+h}$ is the realized target. It is not a universal fix.

The mean absolute scaled error, or MASE, uses a seasonal-naive in-sample scale:

where:

• $\operatorname{MAE}_h$: the out-of-sample MAE at horizon $h$
• $T$: the last index of the training history
• $m$: the seasonal period
• $y_t-y_{t-m}$: the seasonal difference between observation $t$ and the same season one cycle earlier

The denominator is the average absolute seasonal difference in the training history. If (m=1), it is the average absolute first difference, which is the naive scale. The numerator is the out-of-sample MAE at horizon (h). A MASE below 1 means the forecast has lower MAE than the seasonal-naive scale on the training data; a MASE above 1 means it is worse on that scale. The scale must be computed without test data. In a rolling backtest, we can compute the scale once from the initial training window, or recompute it at each origin from that origin's history. The first choice keeps the scale fixed and comparable across origins. The second updates it from the history retained at each origin: expanding histories accumulate observations, while rolling histories replace older observations with newer ones. For a per-origin scale $a_o$, average the components $|e_{o,h}|/a_o$; this generally differs from dividing pooled MAE by an average scale. State the convention. If the seasonal differences are all zero, the denominator is zero and MASE is undefined.

Uncertainty should be reported by horizon. At origin $o$, a residual-based interval can use only earlier forecast errors whose targets have already been released. With no reporting delay, define

Each $e_{s,h}$ must come from a forecast frozen at its own origin $s$. Delayed labels additionally require their release date to be no later than $o$. When enough completed errors are available, the interval is

Here $p$ is a quantile probability and $\alpha$ is the desired miscoverage level. Error quantiles have the same units as the target. This heuristic assumes the future error distribution resembles the completed past errors. It gives no finite-sample coverage guarantee under arbitrary serial dependence. Evaluate interval coverage at later untouched origins, updating calibration only with errors completed by each origin. Pooling future errors before constructing earlier intervals would leak outcomes. Overlapping errors may be correlated, so row count alone does not determine uncertainty in the reported coverage fraction.

Reporting should include the forecast origins, the horizon, the seasonal period, the window type, the model names, and the error scale. A single aggregate MAE over a mix of horizons can hide the fact that a model is good one step ahead and poor four weeks ahead. A table by horizon, plus a compact overall summary, is usually more informative. A skill score relative to a baseline is also useful:

where:

• $\operatorname{MAE}_{\text{model}}$: the mean absolute error of the model being evaluated
• $\operatorname{MAE}_{\text{baseline}}$: the mean absolute error of the chosen baseline, at the same origins, horizon, and error scale

A positive skill score means the model has lower MAE than the baseline; zero means it ties; negative means it is worse. The baseline and horizon must be stated. If baseline MAE is zero, this ratio is undefined. The skill score is most persuasive when the baseline is the strongest simple benchmark for the same decision, not the weakest straw man.

With integrable, conditionally mean-zero innovations, naive and seasonal naive are conditional means under their respective random-walk models. With known constant drift $c$, the conditional mean is $y_T+hc$; the implemented rule uses $\hat c$ instead. This is a plug-in estimate. Mean reversion, changing seasonality and other departures can make these models unsuitable.

The baselines are also linear functions of the observed history. The naive forecast is the identity function of the last observation and is constant across horizons at a fixed origin. The seasonal-naive forecast selects one of the last (m) observations according to the horizon. The drift forecast is an affine function of the first and last observations. This linearity makes their dependence on the observed history explicit, but also limits their flexibility. Finite inputs and valid history lengths are still required.

Expanding fitting windows are nested when history is appended without revision. Rolling fitting windows are generally not nested because older records leave the selected window. The full information available over time is a different object from the observations deliberately retained for fitting. More history can help in a stable model, while forgetting old regimes can help under change; neither window has a universal variance or accuracy advantage.

Forecast errors from overlapping origins can be dependent. If origin (o) and origin (o+1) both forecast target (o+28), their errors share the same future observation. Their retained fitting histories overlap according to the window policy: expanding histories are nested, while adjacent rolling windows of width $w$ share $w-1$ observations, including none when $w=1$. We can still average them, but we should not treat the average as if it came from independent samples. This is why a backtest with many overlapping origins can look more precise than it really is.

We generate a daily synthetic demand series with a slow trend, a weekly seasonal pattern, and noise. The baselines expose different parts of that structure: naive retains the latest level, seasonal naive repeats the last cycle, and drift extrapolates an endpoint slope. None is required to win.

We choose a forecast origin and reveal the next 35 days only for display and scoring. The forecasting functions receive history ending at that origin.

Before trusting a backtest loop, it is worth shrinking the problem until every number can be checked by hand. A small example exposes the bookkeeping: which observations are in the information set, which are targets, and how each baseline turns the same history into a different forecast.

The series is

We set the forecast origin at (T=8), so the observed history is (y_1,\ldots,y_8), and the target block is (y_9,\ldots,y_{12}). The seasonal period is (m=4), and the maximum horizon is (H=4). The history contains two full cycles and the target block one full cycle. It is also small enough that a reader can verify each forecast against the raw numbers.

For the naive forecast, every horizon repeats (y_8). In symbols,

So the naive path is (30,30,30,30), regardless of the upward movement in the history.

For the seasonal-naive forecast, we use (m=4). The floor formula gives (k=0) for (h=1,2,3,4), so the forecasts are (y_5,y_6,y_7,y_8). These are the last four observations, repeated in order. For (h=1), the target is (y_9), and the most recent same-season observation is (y_5). For (h=4), the target is (y_{12}), and the most recent same-season observation is (y_8). In this example the method carries the shape of the last cycle forward, including its local ups and downs, but it does not add the trend.

For the drift forecast, the estimated drift is

The forecasts are (y_8+h\hat c) for (h=1,2,3,4). The first observation and the last observation determine the slope. If either endpoint is unusual, the drift forecast changes. This is a useful feature to notice before applying drift to noisy operational data: the method is not estimating a trend from all the data, it is drawing a line between two boundary observations.

The code below computes exactly the same three forecasts from the same history. It does not look at the target block when constructing the predictions. The target block is attached only afterward for comparison.

A baseline win describes the evaluated target blocks under the chosen loss. A naive, seasonal-naive, or drift win can motivate further testing of recent level, recurring seasonal shape, or trend, respectively. A sample win alone does not establish that the corresponding pattern persists in the population. Compare a more complex model with the relevant baselines at the same origins and horizons before attributing an improvement to its added complexity.

A backtest is a bookkeeping device before it is a modeling device. The implementation below makes the bookkeeping explicit: it slices history at each origin, calls each forecaster with the same interface, records one error row per origin-model-horizon, and lets pandas do the aggregation. NumPy is the right level for the forecasting primitives because each baseline is a short array operation. pandas is useful for holding the time index, grouping errors by horizon, and producing readable summaries. We deliberately avoid a high-level forecasting package in this section so that the origin handling and leakage boundaries remain visible.

The function below accepts a series, a dictionary of forecasters, an initial training size, a horizon, a seasonal period, a step between origins, and a window type. An expanding window uses all observations up to the origin. A rolling window uses the most recent initial_train observations. The function returns a long error table with one row per origin, model, and horizon. The long format is deliberate: it preserves the origin and horizon of every error, so later summaries can be audited rather than trusted as opaque aggregates.

The horizon table makes the degradation of each baseline visible without a plot. A column that grows with the horizon shows increasing average absolute error on these origins; a flat column shows similar average error across the displayed horizons. Neither pattern alone establishes that a forecasting assumption holds or fails for the underlying process.

This section summarizes the key parameters documented above and the main functions used in the implementation. Below are the parameters that control the validation design and the baselines.

initial_train is the size of the first training window. It must be larger than the seasonal period so that seasonal-naive and MASE calculations are defined. In a rolling window, it also acts as the window length in this implementation. A larger initial window provides more fitting observations but can include stale regimes. On a shared origin schedule, a smaller window drops older observations no later and can increase estimation variance in fitted models; the naive and seasonal-naive rules here use only the latest observation or cycle once enough history is available. The choice is a bias-variance trade-off expressed in time, not in model capacity.

The parameter horizon is the maximum lead time. It determines which targets are evaluable and how the error table is grouped. If the business decision is a four-week order, the horizon should be 28 days, not one day. A model that is excellent one day ahead but poor 28 days ahead is not a solution to the four-week problem. The horizon should be chosen from the decision cadence, not from the desire to make the error curve look smooth.

The parameter step is the spacing between forecast origins. A step of 1 uses every possible origin; adjacent target blocks share $H-1$ dates. A step equal to the horizon creates disjoint target blocks and generally yields fewer evaluations, with no strict reduction required when $H=1$ or only one origin is evaluable. The step should be stated in the report because it affects the dependence between errors and the apparent sample size. A dense grid of origins is not automatically better evidence; the dependence among its errors depends on the horizon, history policy and data-generating process.

The parameter window chooses expanding or rolling history. Expanding history is appropriate when the process is stable and old data remain relevant. Rolling history is appropriate when the process changes. The choice should be made from training evidence, not from the test score. In practice, it is often worth reporting both windows as a sensitivity check, especially when a structural break is suspected.

The parameter season is the seasonal period used by seasonal naive and by the MASE scale. It should reflect the data frequency and the business calendar. Daily data with a weekly pattern usually use $m=7$. Monthly data with an annual pattern usually use $m=12$. If the seasonality is not stable, a fixed $m$ can be misleading. The seasonal period is a modeling assumption, and it should be defended in the same way as the choice of model.

The parameter scale is the denominator used for MASE. In this implementation it is computed once from the initial training window. A per-origin scale is also possible, but it should be documented. A zero or near-zero scale makes MASE unstable, just as zero targets make MAPE unstable. The scale should be treated as part of the reporting contract, not as an invisible implementation detail.

naive_forecast returns a constant array equal to the last observation. It is a simple last-value baseline and the standard random-walk benchmark. Comparing this baseline at matched origins and horizons tests the predictive value of the most recent level under the chosen loss. It is also the first place to look when a complex model appears suspiciously accurate, because an unavailable feature can produce an invalid comparison, even though a large legitimate improvement is also possible.

seasonal_naive_forecast tiles the last seasonal cycle until the requested horizon is filled. It is the standard seasonal random-walk benchmark. Comparing this baseline at matched origins and horizons tests the predictive value of repeating the latest seasonal pattern under the chosen loss. The function uses only the last season observations, so within that forecast block it cannot smooth their noise or extrapolate a new level; that is both its simplicity and its limitation.

drift_forecast estimates a slope from the first and last observations and adds that slope at each future step. It is the standard random-walk-with-drift benchmark. Because it uses endpoints, it is sensitive to outliers at the boundaries; a least-squares trend uses the interior observations and may be less sensitive to a noisy endpoint when a linear trend model is appropriate. The endpoint sensitivity is not a bug in the implementation; it is the defining assumption of the canonical drift baseline.

rolling_origin_backtest is the validation engine. It advances the origin, slices the available history, calls each forecaster, and records the realized error at every horizon. The slicing is the most important line in the function: it must never include the target block in the history. The function raises an error if a forecaster returns the wrong number of horizons, which prevents silent shape mismatches. The shape check catches an invalid forecaster return before its values enter the error table.

seasonal_scale computes the in-sample seasonal-naive scale for MASE. It uses only the training window passed to it. Keeping the scale function separate makes the leakage boundary explicit: the scale is estimated from training history, then used to normalize out-of-sample errors. summarize_errors groups by window and model; the separate horizon table groups by model and horizon. The grouping is where we turn forecast errors into a report that a decision-maker can interpret. Because the long table retains origins and horizons, the summary can always be traced back to the individual forecasts that produced it.

Baselines should be part of every forecasting workflow, not a routine first row in a results table. The naive forecast tests the predictive value of the latest level; seasonal naive tests repetition of the observed seasonal pattern; drift tests endpoint-based trend extrapolation. If a more complex model cannot improve accuracy at the horizon that drives the operational decision, the simpler forecast may be the better choice once maintenance, monitoring, and failure modes are included. The comparison is most informative when the benchmark is the strongest simple method for that decision, not a deliberately weak one.

Temporal validation should mirror the deployment cadence. Match the issuance schedule and the latest response actually available. In the Monday-morning example, sales are final through Sunday: use Sunday response-index origins and score the next 28 daily targets, beginning Monday. For daily next-day forecasting with the current day finalized, use that day as the response-index origin and a one-day horizon.

• The origin schedule, maximum horizon, window type, seasonal period, and step between origins should be reported together, because a single aggregate score over a mix of horizons can hide whether a model is useful at the lead time the business plans for.

• Expanding windows are appropriate when old data remain relevant, rolling windows when the process changes; when a structural break is suspected, reporting both is a useful sensitivity check. Set initial_train larger than the seasonal period and long enough to cover several cycles; in a rolling window it also fixes the window length.

• A step of 1 maximizes evaluations, with $H-1$ shared target dates between adjacent blocks. A step equal to the horizon removes target-block overlap and generally reduces the evaluation count; serial dependence and shared training histories can remain.

Feature availability is one source of leakage. A feature is admissible at origin (o) only if its value is known at (o); rolling averages of realized responses must use only observations available at that origin. Windows over covariates known in advance may include future dates. Scalers, imputers, and any other preprocessing statistics must be fitted on the history available at that origin and then applied forward. Calendar features and approved promotion plans are usually safe because they are known in advance, while unreleased realized sales, traffic, and weather revisions are not. Missing timestamps and irregular gaps need the same treatment in backtesting as in production: filling a gap with a value released after the forecast origin creates leakage. A later observation already available at that origin can be used by the same documented production rule, but not retrospectively at an earlier origin where it was unknown. A training-window statistic is another option and can distort seasonality, so the handling should be documented.

Error metrics should match the decision and be reported by horizon. MAE is natural when the cost of a unit error is roughly linear, RMSE when large errors are disproportionately costly, and bias when under-forecasting and over-forecasting have different consequences. MAPE should be avoided or accompanied by an explicit exclusion rule when the target can be zero, and MASE is useful for scale-free comparison only when its denominator is computed from training history. A single aggregate MAE can conceal a model that is strong one step ahead and poor four weeks ahead, so horizon-specific tables and a compact overall summary are usually more informative than one number. When empirical intervals are reported from backtest residuals, their coverage should be checked by horizon; overlapping origins can induce dependence, so row count alone does not establish precision.

The following pitfalls can invalidate the comparison. Fitting a scaler with values unavailable at the origin, computing a rolling average of realized responses that includes the unavailable target-day outcome, using a promotion flag that was unavailable at the forecast origin, or computing the MASE scale with test targets makes the evaluation invalid for the stated information boundary and can bias it optimistically. Selecting the window, horizon, or model after inspecting the test score also invalidates an untouched-test interpretation, and treating correlated forecast errors as independent can misstate precision. Drift is sensitive to unusual endpoints, seasonal naive is rigid when the seasonal pattern changes, and naive is flat when a trend or promotion is present; these weaknesses should be diagnosed rather than hidden by an aggregate score.

Computational cost and deployment needs should be considered before the validation design is fixed. If deployment refits at every origin, backtest cost grows with the number of origins and the cost of each fit; with other settings fixed, a larger step cannot increase the evaluation count and may reduce computational cost. Adjacent target-block overlap cannot increase as the spacing grows and stays zero once the blocks are disjoint; weaker error dependence is not guaranteed. For long series or many models, origins can be parallelized or errors aggregated in a streaming table, but the origin, horizon, and history boundary must remain explicit in every run. In production, the origin cadence, horizon, window type, seasonal period, and MASE scale should be part of the monitoring contract, and performance should be tracked by horizon, bias, and interval coverage rather than only by a single aggregate. Baselines are useful as a fallback and as a live diagnostic: if a complex model stops beating the relevant simple benchmark, that is a signal to check for leakage, drift, or a changed process before retraining blindly.

Forecasting baselines are not a formality. Naive and seasonal naive have simple random-walk interpretations, while drift estimates the slope of a random walk with drift. They provide transparent benchmarks at matched origins, horizons and losses. The naive forecast repeats the last observation, the seasonal-naive forecast repeats the last seasonal cycle, and the drift forecast continues a linear trend. Each makes a specific assumption, and each fails in a specific way.

Temporal validation must respect the forecast origin, the horizon, and the information set. At origin (o), only information through (o) may be used. Rolling-origin backtesting advances the origin, produces forecasts for explicit future targets, and scores them after the targets are revealed. Expanding windows use all history; rolling windows forget old history. Overlapping target blocks can create dependent errors, so error-row count alone does not establish independence.

Forecast error should be reported by horizon, with MAE, RMSE, bias, and a scale-aware measure such as MASE. Percentage errors need explicit handling of zeros and near-zero targets. MASE needs a leakage-free seasonal scale. Empirical uncertainty intervals should be interpreted with the dependence of overlapping errors in mind. When these pieces are in place, a sophisticated forecasting model has a fair benchmark and a validation design that resembles the decision it is meant to support.

Ready to test your understanding? Take this quick quiz to reinforce what you've learned about forecasting baselines, temporal validation, and backtesting.