Generalized Additive Models › Disadvantages

GAMs are not free. Their main limitation is that the additive structure itself is a strong assumption. If the true relationship involves an interaction between two predictors, say the effect of $x_1$ flips sign depending on $x_2$, an additive model cannot represent it. We can add explicit tensor-product smooths $f_{12}(x_1, x_2)$ to handle interactions, but each interaction term costs us both flexibility and sample size, and the resulting components are much harder to plot and interpret. In high dimensions the number of potential pairwise interactions explodes.

Second, GAMs depend on several tuning choices: the basis dimension and knot arrangement, the penalty order, and the smoothing parameter $\lambda$ per component. Choosing those badly produces either an overfit curve that wiggles through noise or an over-smoothed curve that misses real structure. We can automate the choice with cross-validation or generalized cross-validation, but their estimates have sampling variability. Neither criterion guarantees that the selected model will generalize well; strong overlap between smooth terms can also complicate interpretation.

Third, extrapolation is dangerous in a way that is easy to forget. Out-of-domain behavior depends on the basis implementation: it may refuse prediction or impose an extrapolation rule. In particular, the Patsy and statsmodels B-spline transforms used here reject points beyond their boundary knots. A GAM should not be used to predict far outside the training support of any predictor.

A spline expansion also widens the design matrix relative to a linear model of the original predictors. Dense cross-products and coefficient solves then grow with the total basis dimension; specialized implementations can exploit sparsity or low-rank structure. We give the dense costs in Practical Implications.

Component plots show the fitted terms on the additive-predictor scale. They need care when predictors are dependent, because varying one predictor while holding the others fixed can leave the region supported by the data. Overlapping smooth-term spaces can also make individual components unstable.