Quantile Regression

A courier company wants to promise a delivery time. A forecast of the average duration does not tell it what fraction of parcels will arrive late. Quoting a much longer time may reduce late deliveries but make the service less attractive. A conditional 90th percentile addresses the operational question: what duration has at least 90 percent of deliveries at or below it for a route with these predictors? An estimated percentile still needs validation before it becomes a service promise.

Ordinary least squares answers a different question: it fits a linear conditional-mean model when that mean is correctly specified. Fitting the coefficients does not require constant conditional variance. A constant-width prediction band is an additional modeling choice, not a restriction imposed by least squares. Quantile regression instead fits a conditional quantile at a chosen level $\tau$. For a continuous conditional distribution with a unique quantile, the true 0.9 quantile has 90 percent of probability at or below it. Fitted linear quantiles need not achieve that exact conditional fraction in a finite sample.

The mechanism is a change of loss function. Squared error $\rho(u) = u^2$ is minimized by the conditional mean. The pinball loss (an asymmetric absolute-value loss) is minimized by the conditional quantile. Once we make that swap, the familiar linear model machinery (design matrices, gradients, regularization) mostly carries over, but the geometry changes in important ways. The objective becomes piecewise linear and non-differentiable at zero, so we solve a linear program rather than a linear system. The fitted curves are also no longer guaranteed to be ordered across $\tau$.

This chapter is the last stop in our tour of Part V: Linear and Generalized Linear Models. Everything we built for multiple linear regression, the design matrix, the intercept column, the loss-plus-optimizer framing from gradient descent and numerical optimization, reappears here with a different loss. What we gain is a model that speaks directly about risk, spread, and tail behavior, which is exactly the language we will need when we move to time-to-event data in Part VI.