Quantile Regression › Disadvantages

The main practical annoyance is quantile crossing. In the population the quantile function is nondecreasing in $\tau$, so $Q(0.4 \mid x) \le Q(0.6 \mid x)$ for every $x$. However, each $\tau$ is estimated by its own optimization, so nothing in the procedure forces the fitted curve at $\tau = 0.4$ to lie below the fitted curve at $\tau = 0.6$. In finite samples the curves can cross; the risk depends on sample size, the design, the predictor domain, and functional-form misspecification. At a fixed predictor value, narrowing a nested pair of target levels cannot increase its population quantile gap, leaving less separation for estimation error to overcome. A crossing fitted value is not just undesirable. It cannot be the ordered quantile function of a conditional distribution and can reverse the endpoints of a prediction band. We will see how to diagnose this, and how rearrangement repairs it, later in the chapter.

The second limitation is that efficiency at the center is worse than least squares when least squares is appropriate. For $n$ independent, identically distributed Gaussian observations with positive variance $\sigma^2$, the sample mean has variance $\sigma^2/n$ while the sample median has asymptotic variance $\pi\sigma^2/(2n)$, so their relative efficiency (the ratio of variances) is $(\sigma^2/n)/(\pi\sigma^2/(2n)) = 2/\pi \approx 0.64$ for the median, meaning we need roughly $1/(2/\pi) = \pi/2 \approx 1.57$ times as much data for the same precision. The same $2/\pi$ asymptotic relative-efficiency factor applies to correctly specified median regression with independent homoscedastic Gaussian errors and a regular design. Tail precision depends on the conditional density and design as well as $\tau$: normal tails have low density and can be difficult to estimate, but uncertainty need not increase monotonically toward the tails for every distribution.

Third, the objective is not smooth, so the fitted coefficients need not change smoothly as the data change; jumps can occur when an optimum switches between non-unique solutions. The general multivariable problem has no least-squares-style matrix formula, although special cases have quantile shortcuts. Ordinary differentiation is insufficient at zero residuals; we use a linear program. Its runtime depends on the sample size, predictor count, matrix representation, conditioning, and solver settings, and the piecewise-linear structure can produce non-unique optima when the data are degenerate (duplicate covariate patterns, for example), which complicates inference. Common regular asymptotic standard-error estimates use resampling or estimates of the conditional density; they do not automatically follow the OLS formula $\sigma^2 (X^\top X)^{-1}$.

Fourth, and most often confused, a fitted quantile band is not a confidence interval. The band $[\hat Q(0.05 \mid x), \hat Q(0.95 \mid x)]$ targets coverage of individual responses $Y$ (here $\hat Q(\tau \mid x)$ is the estimated conditional $\tau$-quantile), a statement about the spread of the data; a confidence interval is a statement about the uncertainty of an estimated parameter such as a regression coefficient. The first can be wide because the data are noisy; the second can be narrow because the sample is large. Both need to be reported, and they answer different questions.