Generalized Additive Models › Practical Implications

The smoothing parameter $\lambda$ is a hyperparameter. Choose it by scoring fits on observations excluded from fitting those coefficients. An untouched test set then assesses the selected procedure. For dependent observations, choose folds that match the generalization question. To evaluate transfer to unseen subjects, keep each subject in one fold. Forecasting requires a temporal split that matches deployment, including forecasting within already observed subjects when that is the target. Random folds below are appropriate for our independently generated observations.

The basis belongs inside that procedure too. Patsy's bs and statsmodels' BSplines can learn interior knot positions from training-predictor quantiles. Constructing that basis before splitting gives the fit information about validation inputs. That is a leakage path, although the size and direction of its effect on a score depend on the problem. Keep the fitted knot specification and training centering transformation, then reuse them on validation, test, and production rows. Our synthetic bounds $[-3,3]$ were fixed by the data-generation design; these are not estimated from held-out inputs. Both libraries' B-spline transforms reject points outside their boundary knots rather than silently performing polynomial extrapolation.

We use five folds and a predeclared grid of nine smoothing parameters. Each fold constructs its own basis and centering constraint using only that fold's training rows. The fold errors are averaged to select a shared $\lambda$ for the two components. A separate synthetic test sample is generated only after selection, then scored once. The earlier full-sample component plots illustrate fitting; their training RMSE does not select the winning parameter.

The reported test RMSE describes this one synthetic sample from the stated generating model. It is not evidence that GAMs will attain that error on another dataset. A separate $\lambda_j$ per smooth allows different flexibility across predictors, at the cost of a larger tuning problem. Sharing a parameter is a simplifying restriction here, not a universal GAM default.

For a Gaussian linear smoother, generalized cross-validation (GCV) offers another selection criterion: $\mathrm{GCV}=n\|\mathbf{y}-\hat{\mathbf{y}}\|^2/(n-\operatorname{tr}\mathbf{H})^2$, provided the denominator is nonzero. Here $n$ is the training-sample size and $\operatorname{tr}\mathbf{H}$ is the total effective degrees of freedom for that fixed basis and penalty. GCV approximates leave-one-out assessment by replacing individual leverage values with their average; it is not identical to every held-out split. Packages also provide likelihood-based smoothness selection methods. Choose a criterion that respects the data structure and evaluation question.

For a dense implementation with $q$ identifiable coefficients and $n$ rows, forming the cross-product matrix costs $O(nq^2)$ and a direct solve costs $O(q^3)$, with $O(nq+q^2)$ storage when the design is retained. Within a fixed fold, the basis and its cross-products can be reused across smoothing candidates. For non-Gaussian models fitted by IRLS, each iteration solves a weighted penalized system; runtime and convergence depend on the data and link. Effective degrees of freedom describe model flexibility; neither their size nor training RMSE alone diagnoses generalization.

Our constraint makes each fitted component average zero over its training observations, so the intercept and smooths have a specified reference. At two values of $x_j$, the difference in $\hat f_j$ gives the change in the fitted additive predictor while the other inputs are held fixed. With an identity link, a rise of $0.5$ corresponds to a $0.5$ increase in the fitted conditional mean. With a logit link it multiplies fitted odds by $e^{0.5}\approx1.65$; with a log link it multiplies the fitted mean by that factor. It does not imply an additive change of $0.5$ in probability for a logit GAM. On the response scale, a nonlinear inverse link also makes the size of a mean change depend on the other predictors' values.

Component curves describe fitted associations. Interpreting them as causal effects needs additional identification assumptions and a design that supports those assumptions. Correlation between inputs can limit which combinations the data support. Concurvity is the more specific problem in which a smooth term can be approximated by other model terms: it can make the allocation among fitted components unstable even when total predictions look adequate. Exact overlap in unpenalized directions can leave a model unidentifiable despite centering. Check supported predictor combinations and basis conditioning before interpreting a component in isolation.

Inspect residuals for patterns involving pairs of predictors, then compare a plausible interaction model using the same validation protocol. Residual plots can suggest a failure of additivity; absence of a visible pattern does not prove additivity. A tensor-product term $f_{jk}(x_j,x_k)$ starts from the row-wise products of two marginal bases. Before constraints, it has $K_jK_k$ coefficients: two 12-function bases give 144 products. Identifiability constraints and removal of overlapping main-effect directions can reduce that count. The interaction needs its own penalties and evidence that the added flexibility helps. There are $p(p-1)/2$ possible pairs, so domain knowledge can help limit the candidates.

Boundary regions often contain less information than the interior. The displayed example has known finite support; production data need explicit handling of out-of-domain inputs and monitoring for distribution shifts. Natural cubic splines impose linear tails, whereas the B-spline transforms used here refuse points beyond their boundary knots. Neither policy establishes that predictions beyond observed support are reliable. Rebuilding a basis on a new training window changes the fitted model and requires validation again.

For uncertainty, component intervals require a covariance calculation matched to the fitted likelihood, penalty, and centering convention. The fixed-smoothing Gaussian covariance given earlier is one such conditional calculation. It omits smoothing-selection uncertainty and does not turn an association into a causal effect. These limits matter when a curve is used to explain a prediction as well as when it is used for inference.