Quantile Regression › Example: Small Numbers and Heteroscedastic Predictions

The formulas are compact, but the logic is easiest to trust when we can compute a few quantiles by hand. We start with a single median, then a slope, and finally the heteroscedastic case that motivates the entire method. The numbers are small enough to check on paper.

Take five response values $y = (2, 3, 5, 8, 10)$ and ask for the median, $\tau = 0.5$. The counting rule says we need a value $q$ with at most $0.5 \times 5 = 2.5$ observations strictly below it and at least $2.5$ at or below it. The value $q = 5$ has two observations below it $(2, 3)$ and three at or below it $(2, 3, 5)$, so it satisfies the rule. Let us confirm by evaluating the average pinball loss at three candidates.

• At $q = 3$ the residuals are $u = (-1, 0, 2, 5, 7)$. With $\tau = 0.5$, $\rho_{0.5}(u) = 0.5|u|$, so the losses are $(0.5, 0, 1.0, 2.5, 3.5)$ and their average is $7.5/5 = 1.5$.
• At $q = 5$ the residuals are $u = (-3, -2, 0, 3, 5)$, losses $(1.5, 1, 0, 1.5, 2.5)$, average $6.5/5 = 1.3$.
• At $q = 6$ the residuals are $u = (-4, -3, -1, 2, 4)$, losses $(2, 1.5, 0.5, 1, 2)$, average $7/5 = 1.4$.

These values compare the same average loss at three candidate medians.

Here the pinball loss at $\tau = 0.5$ is $\rho_{0.5}(u) = 0.5|u|$, and the residual is the signed distance from each observation to the candidate value $q$.

where:

• $u$: the residual $y_i - q$, the signed distance from each observation to the candidate value $q$
• $\rho_{0.5}(u)$: the pinball loss at $\tau = 0.5$, which equals $0.5|u|$
• $q$: the candidate value being evaluated as an estimate of the sample median

The minimum is at $q = 5$, which is the sample median. Moving $q$ up from 5 adds loss on the three points below and removes loss on the two above; because there are more points below than above, that trade is unfavorable. The balance point is exactly where the counts cross, which is the counting rule.

Now consider a one-parameter model with no intercept, $Q(\tau \mid x) = \beta x$, fitted to four observations, where $\beta$ is the single slope to be estimated and $x$ is the predictor:

Because all $x_i > 0$ and the residual in observation $i$ is $y_i - \beta x_i = x_i(r_i - \beta)$, the pinball loss is positively homogeneous of degree one: for $c > 0$, $\rho_\tau(cu) = c\,\rho_\tau(u)$. Writing $y_i - \beta x_i = x_i(r_i - \beta)$ with $x_i > 0$, the objective becomes

$$\sum_{i=1}^{4} \rho_\tau\left(y_i - \beta x_i\right) = \sum_{i=1}^{4} x_i\, \rho_\tau\left(r_i - \beta\right).$$

where:

• $r_i = y_i / x_i$: the ratio of response to predictor for the $i$-th observation
• $x_i$: the $i$-th predictor value, which also serves as the weight
• $\beta$: the single slope parameter being estimated
• $\rho_\tau$: the pinball loss at level $\tau$, which is positively homogeneous of degree one

This is a weighted pinball loss in the single unknown $\beta$, and its minimizer is the weighted $\tau$-quantile of the ratios $r_i$ with weights $x_i$. Order the ratios and accumulate weight: $1.50$ with weight 2, then $2.00$ with weight 1 (cumulative 3), then $3.00$ with weight 3 (cumulative 6), then $3.25$ with weight 4 (cumulative 10). For $\tau = 0.5$ the target cumulative weight is $0.5 \times 10 = 5$. The cumulative weight jumps from $3$ (after $2.00$) to $6$ (after $3.00$), so it crosses the target of $5$ at the ratio $3.00$; by the weighted-median convention we take the smallest ratio whose cumulative weight is at least the target, giving $\hat\beta(0.5) = 3$.

Checking the loss: at $\beta = 3$ the fitted values are $(3, 6, 9, 12)$ and the residuals are $(-1, -3, 0, 1)$, giving an average loss of $0.5(1 + 3 + 0 + 1)/4 = 0.625$. At $\beta = 2.75$ the residuals are $(-0.75, -2.5, 0.75, 2)$, average loss $0.5(0.75 + 2.5 + 0.75 + 2)/4 = 0.75$. At $\beta = 3.25$ the residuals are $(-1.25, -3.5, -0.75, 0)$, average loss $0.5(1.25 + 3.5 + 0.75 + 0)/4 = 0.6875$. Both neighbours are worse, confirming the minimum. Note also that this weighted-median minimizer applies because all slopes share a single parameter $\beta$ multiplying positive weights $x_i$; the objective is a weighted pinball loss in $\beta$ whose minimizer is the weighted $\tau$-quantile of the ratios.

This shortcut is worth remembering. For a single slope, no intercept, and positive predictors, the objective reduces to a weighted $\tau$-quantile of the ratios $r_i=y_i/x_i$ with weights $x_i$. At $\tau=0.5$ this is a weighted median. That is why quantile regression sometimes feels like a ranking operation rather than a fitting operation.

Now the case that matters. Take three illustrative quantile lines with the coefficients below. These are chosen for hand arithmetic, not claimed solver output from the later simulated dataset. Over $0\le x\le10$ their ordering gives a widening nominal 10th-to-90th-percentile band:

The fitted curves are $Q(0.1 \mid x) = 1.0 + 0.6x$, $Q(0.5 \mid x) = 2.0 + 1.1x$, and $Q(0.9 \mid x) = 3.6 + 2.0x$, where:

• $Q(\tau \mid x)$: the fitted conditional $\tau$-quantile at predictor value $x$
• $\beta_0(\tau)$: the intercept of the fitted line at level $\tau$
• $\beta_1(\tau)$: the slope of the fitted line at level $\tau$

Evaluate them at three values of $x$ and record the width of the central 80 percent band, which is defined as $W(x) = Q(0.9 \mid x) - Q(0.1 \mid x)$, where $Q(\tau \mid x)$ is the fitted conditional $\tau$-quantile and $W(x)$ is the resulting band width in response units:

• At $x = 1$: the three values are $1.6$, $3.1$, $5.6$, so $W(1) = 5.6 - 1.6 = 4.0$.
• At $x = 5$: the values are $4.0$, $7.5$, $13.6$, so $W(5) = 13.6 - 4.0 = 9.6$.
• At $x = 10$: the values are $7.0$, $13.0$, $23.6$, so $W(10) = 23.6 - 7.0 = 16.6$.

The width is not constant: because both bounding curves are linear, the width is exactly linear in $x$. Subtracting the two fitted lines $\hat Q(0.9 \mid x) = \beta_0(0.9) + \beta_1(0.9)\,x$ and $\hat Q(0.1 \mid x) = \beta_0(0.1) + \beta_1(0.1)\,x$ term by term,

$$\begin{aligned} W(x) &= \left[\beta_0(0.9) - \beta_0(0.1)\right] + \left[\beta_1(0.9) - \beta_1(0.1)\right] x \\ &= (3.60 - 1.00) + (2.00 - 0.60)\,x \\ &= 2.60 + 1.40\,x, \end{aligned}$$

which reproduces $4.0$, $9.6$, and $16.6$ at $x = 1, 5, 10$ (for example, $W(5) = 2.60 + 1.40 \times 5 = 9.60$). The slope difference $1.40$ is the rate at which the conditional spread grows: each additional unit of $x$ widens the 10-to-90 range by 1.40 units. An additional constant-variance normal residual model would produce a constant-width band. Least squares itself does not force that variance model.

We can also price individual predictions with the pinball loss. Suppose a new observation arrives with $x = 5$ and $y = 13.0$. The $\tau = 0.9$ prediction is $13.6$, so the residual is $u = 13.0 - 13.6 = -0.6$. Because $u < 0$ we use the lower branch: $\rho_{0.9}(-0.6) = (0.9 - 1)(-0.6) = (-0.1)(-0.6) = 0.06$. Overpredicting this response at a high quantile is cheap because the lower branch of $\rho_\tau$ has slope $-(1-\tau) = -0.1$ (in magnitude $0.1$), so a negative residual of size $0.6$ is penalized by only $0.1 \times 0.6 = 0.06$; a true 0.9 quantile of a continuous distribution has 90 percent of its probability below it. Our illustrative coefficients carry no empirical calibration guarantee; the small penalty here follows from the chosen loss. Now take $y = 7.0$ at the same $x$ and score it against the $\tau = 0.1$ line, whose prediction is $4.0$. The residual is $3.0 > 0$, so $\rho_{0.1}(3.0) = 0.1 \times 3.0 = 0.30$. The two residuals, $-0.6$ and $+3.0$, are not of comparable size, so this comparison mainly illustrates the pricing rule: the pinball loss charges $\tau$ per unit on the positive side and $1 - \tau$ per unit on the negative side, with both rates evaluated at the relevant $\tau$.